Activation and optimization of a night mode in camera monitor applications

Abstract

The invention relates to a method for operating an image data processing unit (16) of a vision system (10) of a vehicle (20). The method comprises receiving image data acquired by at least one image acquisition unit (12), receiving a status signal indicating a day or night state, and processing the received image data depending on the received status signal. The invention further relates to a vision system (10) comprising an image data processing unit (16) configured to carry out the method, and a vehicle (20) comprising the vision system (10).

Description

The invention relates to a method for operating an image data processing device of a vision system of a vehicle, to a vision system having a corresponding image data processing device and to a vehicle having a corresponding vision system.It is known in the art to provide vehicles with camera monitor systems (CMS) to facilitate vehicle operation. Such camera-monitor systems typically have a monitor device which is connected to at least one camera device. The at least one camera device is usually designed to capture image data of the vehicle surroundings, which are then displayed on the monitor device arranged in the vehicle.The image quality settings, i.e. the settings of the image signal processor (ISP) of camera monitor systems, are generally optimized for daylight scenarios. Using the same ISP settings for night driving often results in moderate image quality (e.g., high color noise, low dynamic range, over- or under-exposure), which increases the risk of accident due to fatigue, distraction or misassessment of the situation.One known approach to addressing this problem is to change the operating settings of the monitor device, e.g., by reducing the brightness of the monitor device at night (dimming). By reducing the power of an LED backlight behind the display, the driver can be prevented from being fatigued or distracted by a bright screen at night. However, the image content itself is generally not improved or adapted. Consequently, the image quality factors (key performance indicators) such as color noise, dynamic range, exposure and / or white balance remain insufficient.The object of the invention is to provide an improved method for providing image data, e.g. to a driver of a vehicle, wherein the disadvantages of conventional methods are preferably avoided. It is preferably an object of the invention to provide image data of better image quality, in particular adapted to night conditions and / or the current situation.These objects are achieved in each case by the subject matter of the independent claims. Preferred embodiments and applications of the invention are covered by the dependent claims.According to a first independent aspect, a method for operating an image data processing device is provided. The image data processing device is preferably an image data processing device of a vision system (e.g. of a mirror replacement system) of a vehicle.The method includes receiving image data captured by at least one image capture device. The at least one image capturing device can be part of the vehicle and / or of the vision system of the vehicle. For example, the at least one image capturing device can have one camera or a plurality of cameras which are mounted on the vehicle, for example. The image data preferably comprise rearward vision image data that have been recorded (e.g. laterally) next to and / or behind the vehicle. For example, the at least one image acquisition device can be configured to acquire a class II field of view, a class IV field of view and / or a class V field of view according to the rules 2003 / 97 / EG and 2007 / 38 / EG.The method further comprises receiving a state signal, preferably indicative of a day state or a night state. The state signal can be generated by a state device of the vehicle and / or the vision system of the vehicle. For example, the state device can be a brightness and / or light sensor. The brightness and / or light sensor may be configured to output a first state signal value indicating a day state when the brightness of the vehicle environment is above a predefined value and to output a second state signal value indicating a night state when the brightness of the vehicle environment is below this predefined value.The method further comprises (e.g. automatically) processing (e.g. post-processing) the received image data (e.g. by the image data processing device) in dependence on the received state signal. Consequently, the manner of processing the received image data may be different depending on the received state signal, in particular depending on whether the state signal indicates the day state or the night state. In this context, the step of processing may comprise changing and / or changing image processing settings based on the received state signal. For example, the step of processing can comprise adapting a contrast and / or a brightness of the received image data. The method preferably further comprises outputting the processed image data, e.g. to a display device.In contrast to known concepts, in which only the display properties (e.g. the intensity of an LED backlight) of a monitor device are adapted (e.g. dimmed) to the night conditions, the current solution advantageously adjusts the image content itself by correspondingly processing the captured image data. This advantageously allows a substantially greater flexibility in the active modification of the image data by the image data processing device in comparison with a simple adaptation of the display appearance via the monitor device. As a result, the image quality (e.g. automatically) can be optimized to a greater extent for the respective situation, which enables a better adaptation of the provided image data to the current needs of the driver.According to a first aspect, the step of processing may be different depending on whether the received state signal indicates the day state or the night state. For example, the step of processing may include a day mode and a night mode. Preferably, the image processing settings used to process the received image data differ depending on whether the day state (e.g., day mode) or the night state (e.g., night mode) is satisfied or present. For example, a contrast, a digital amplification and / or a white balance of the received image data can be changed depending on the received state signal and / or the present (e.g. day or night) mode. Advantageously, the image data can thereby be adapted to the current lighting conditions.According to a further aspect, the step of processing can comprise adjusting (e.g. amplifying and / or reducing) a contrast of the received image data. The term contrast may refer to the difference between bright and dark areas of the image data, for example. The contrast can be adjusted globally (e.g. by influencing the entire tone scale of the image data) and / or locally (e.g. by independently influencing areas of the image data).Additionally or alternatively, the step of processing may comprise adjusting (e.g. increasing and / or decreasing) a digital gain of the received image data. The term digital amplification may refer to a factor by which the image data is amplified after the digitizing process of the at least one image capturing device, for example. In contrast, the term analog gain or gain may refer to the magnitude of the gain of an output voltage of the photodetectors of the at least one image capturing device prior to being sampled by an analog-to-digital converter.Additionally or alternatively, the step of processing may comprise adjusting (e.g. increasing and / or decreasing) noise filtering of the received image data. This can be done, for example, by using a denoiser algorithm known per se, such as a spatial domain filter and / or a Wiener filter. In this context, the term noise filtering may refer to, for example, removing noise from the image data, in particular, restoring the "true" image. Preferably, adjusting the noise filtering (e.g., depending on the received state signal) comprises a change in a degree to which the noise filtering is used and / or whether it is used at all.According to a further aspect, the method can furthermore comprise outputting (e.g. by the image data processing device) a control signal for the at least one image capturing device as a function of the received state signal. In this case, the control signal can cause the at least one image capturing device to adapt at least one image capturing setting (e.g. an image rate, an (analog) amplification, an exposure time and / or an exposure threshold value) of the at least one image capturing device. For example, if the received state signal indicates the post-state, the output control signal may cause the at least one image capture device to decrease the frame rate (e.g., from 50 fps to 25 fps) and / or increase the exposure time. In addition to the aforementioned adjustment of the image processing settings, this aspect advantageously allows for an active change in the manner in which the (e.g., raw) image data is captured, thereby further increasing flexibility for adjusting / optimizing the image quality to the current conditions.According to a further aspect, the method can further comprise receiving a situation signal. The situation signal can be generated by a situation recognition device of the vehicle and / or the vision system of the vehicle. The situation signal may indicate a current speed of the vehicle, a gear selection of the vehicle (e.g., forward or reverse), and / or an operating state of the vehicle (e.g., maneuvering, cornering, and / or highway driving). The step of processing is further dependent on the received situation signal. Preferably, the processing of the received image data can thus depend on the received state signal and the received situation signal. For example, processing in the night mode may depend on whether the vehicle is parking backwards or on a highway. While in the first case, for example, the contrast and brightness of the image data (e.g. in the night mode / night settings) can be increased by the image data processing device (e.g. in order that the driver can recognize as many details as possible), in the second case the contrast and brightness of the image data (in the night mode / night settings) can be lowered (e.g. in order that the driver is less distracted). Advantageously, the image quality can thereby be optimally adapted to the respective driving situation, so that the perception of the situation of the driver for different scenarios can be improved.According to a further aspect, the step of processing (e.g. the received image data) may comprise reducing a contrast, a digital amplification and / or a brightness of the received image data if the received situation signal indicates a long-distance travel of the vehicle. The corresponding situation signal can be generated, for example, when the vehicle is travelling at a speed of more than 60 km / h. This has the advantage that the driver is not fatigued and / or distracted by the displayed image data.Additionally or alternatively, the step of processing (e.g. the received image data) may also comprise increasing a contrast, a digital gain and / or a brightness of the received image data if the received situation signal indicates a maneuvering (e.g. a parking operation) of the vehicle. The corresponding situation signal can be generated, for example, when the vehicle is travelling backwards and / or at a speed below 25 km / h. This has the advantage that the driver can very well recognize objects and / or obstacles in the displayed image data.According to a further aspect, the method further comprises selecting an image data processing mode from predefined possible image data processing modes depending on the received situation signal. For example, the predefined possible image data processing modes may include at least three, preferably at least four (e.g. different) predefined possible image data processing modes. The predefined possible image data processing modes may include, for example, a reverse travel mode, a highway travel mode, a maneuvering mode and / or an in-situ travel mode. Each of the predefined possible image data processing modes may preferably be associated with a particular type of processing of the received image data and / or particular image processing settings. Here, the step of processing may be performed according to the selected image data processing mode. This advantageously enables optimum adaptation of the image quality to the respective driving situation.According to a further aspect, the predefined possible image data processing modes can have a reverse travel mode. The reverse drive mode may include, for example, generating a wide-angle view of the received image data. Preferably, in the reverse mode, only reverse image data is processed during the processing step. This can be achieved by selecting the rearward image data from the received image data (e.g. only) and / or by controlling the at least one image capturing device such that rearward image data (e.g. only) is received by the image data processing device. This has the advantage that the driver can get a large overview of the rear area, so that he can easily recognize possible objects or obstacles.Additionally or alternatively, the predefined possible image data processing modes may include a highway driving mode. The highway driving mode may include, for example, generating a zoom view of the received image data. Preferably, the zoom view focuses on a certain area of the captured image data, e.g. on a remote area of the vehicle environment. For example only, the step of processing according to the highway driving mode may include processing the received image data such that the class II field of view is increased and / or the class V field of view is decreased. This has the advantage that the relevant area of interest can be displayed to the driver without additional distraction information about the immediate environment.Additionally or alternatively, the predefined possible image data processing modes can have a marshalling mode. The marshalling mode may include, for example, generating a wide-angle view of the received image data. Preferably, the step of processing according to the marshalling mode may comprise processing the received image data such that the information about the lateral area adjacent to the vehicle is magnified and / or highlighted. For example only, the step of processing according to the marshalling mode may include processing the received image data to increase the class II field of view while decreasing the class IV field of view. This has the advantage that the driver receives a large overview of the lateral region, so that he can easily recognize possible objects or obstacles, for example during a turn process.According to a further aspect, the predefined possible image data processing modes may also have a default mode (e.g. an in-situ driving mode). The default mode can be activated by default, for example, unless the received situation signal indicates a specific operating state of the vehicle, e.g., reversing, highway driving and / or maneuvering. In this context, the aforementioned reverse travel mode may preferably have a higher contrast, a higher brightness and / or a higher digital gain compared to the standard mode. Additionally or alternatively, the above mentioned highway driving mode may preferably have a lower contrast, a lower brightness and / or a lower digital gain compared to the standard mode. Additionally or alternatively, the aforementioned ranging mode may have a higher contrast, a higher brightness and / or a higher digital gain compared to the standard mode. This advantageously makes it possible to avoid distraction or fatigue in driving situations in which little additional image information is required, but to provide easily recognizable image information in parking and / or turning situations.In another aspect, the received image data may include data relating to different fields of view. For example, the received image data may include a class II field of view, a class II field of view, a class IV field of view, a class V field of view, and / or a class VI field of view (e.g., according to Policy 2003 / 97 / EG and 2007 / 38 / EG). The different fields of view preferably correspond to different directions of view and / or views of the vehicle environment of a driver seated in the driver's seat of the vehicle in a normal driving position. Additionally or alternatively, the different fields of view preferably correspond to views of the driver that would actually be displayed in (e.g., legally prescribed) mirror devices of the vehicle. In this context, the step of processing (e.g., the received image data) may preferably comprise merging the different fields of view into a combined view. For example, the various fields of view may be combined and / or joined together. The combined view preferably provides a, preferably continuous, (possibly distorted) visualization of the various fields of view in a single image. This has the advantage that the driver can be provided with a single representation of the environment of the vehicle, which display has the relevant information for safe vehicle guidance.In another aspect, the relative ranges of the different fields of view in the combined view are varied depending on the selected image data processing mode. Preferably, the step of merging thus comprises varying the relative ranges of the different fields of view in the combined view depending on the selected image data processing mode. The term "relative range" can correspond to the relative ranges that the different views are intended to assume during the display (e.g. on the display device).For example, in highway driving mode, the class V field of view or a portion of the class V field of view may be reduced (e.g., reduced) as compared to the default mode. In other words, the various fields of view may be merged in highway driving mode such that the class V field of view occupies a lesser area of the combined view than in the default mode. Additionally or alternatively, in highway driving mode, the class IV field of view and / or the class II field of view may be increased (e.g., increased and / or increased shown) compared to the default mode.Additionally or alternatively, in the marshalling mode, the class V field of view or a range of the class V field of view may be increased (e.g., increased and / or increased) compared to the default mode. In other words, the different fields of view may be merged in the marshalling mode such that the class V field of view occupies a larger area of the combined view than in the standard mode. Additionally or alternatively, in the maneuver mode, the class IV field of view and / or the class II field of view may be reduced (e.g., reduced) compared to the default mode. This advantageously enables the driver to make the relevant information about the vehicle environment available in a situation-appropriate manner, so that the areas which are particularly relevant for the current situation can be displayed larger.According to another aspect, the method may further include receiving a display brightness signal from a display device. The display brightness signal may indicate, for example, the brightness of a display (e.g., LCD display) of the display device, in particular the brightness of a backlight of a display of the display device. Preferably, the display brightness signal indicates a brightness value (e.g. manually adapted or adjustable) of the display. In this connection, the step of processing may be performed in response to the received display brightness signal. Preferably, the step of processing is performed in dependence on the received display brightness signal, such that a contrast of the received image data is increased if the display brightness signal indicates an increase in the brightness of the display and / or a contrast of the received image data is decreased if the display brightness signal indicates a decrease in the brightness of the display. Thus, the processing and / or image processing settings may be coupled to a (e.g., manual) brightness adjustment of the display of the display device. This has the advantage that synergistic interaction between the brightness of the backlight and the image processing settings is possible.According to another aspect, the method may further comprise receiving a user request signal. The user request signal can be generated by a control device of the vehicle and / or the vision system of the vehicle. For example, the control device can be arranged on the display device and / or integrated into the display device. For example, the control device can be designed as a touch-sensitive surface of the display device. The user request signal can preferably indicate a command and / or a request of a user. In this context, the method may further include adjusting the image processing settings (e.g., the image data processing device) used, e.g., for processing the received image data, depending on the received user request signal. This can comprise, for example, selecting another (e.g. currently unselected) image data processing mode and / or adapting the (e.g. currently) selected image data processing mode depending on the received user request signal. Advantageously, the image data processing mode can thereby be actively selected and / or modified by a user. Additionally or alternatively, the method and / or the step of adjusting the image processing settings may comprise adjusting a contrast, a digital gain and / or a noise filtering of the received image data depending on the received user request signal. Advantageously, the user can thereby actively adapt the processing of the received image data. Additionally or alternatively, the method and / or the step of adjusting the image processing settings may comprise adjusting a portion of the received image data to be displayed in dependence on the received user request signal. For example, the user may switch from a zoom view to a wide-angle view and vice versa.According to a further aspect, the method may further comprise outputting the processed image data (e.g. by the image data processing device). For example, the image data processing device can have a corresponding output at which the processed image data can be provided. Preferably, the processed image data is output for display on a display device. For example, the processed image data can be output in a data format readable by the display device. The display device can be part of the vehicle and / or of the vision system (e.g. of the mirror replacement system) of the vehicle. The processed image data may include image data of the class II field of view, the class IV field of view, and / or the class V field of view. The display device can be configured for simultaneous display of the class II field of view, the class IV field of view and / or the class V field of view, e.g. on the same monitor and / or the same display of the display device.According to a further aspect, the method can further comprise generating a zoom view and / or a wide-angle view of the received image data in dependence on the received situation signal. The zoom view preferably focuses on a specific area of the recorded image data, while the wide-angle view preferably represents an entire capture area of the at least one image capture device. This advantageously allows situation-related observation of the received image data.According to a further independent aspect, a vision system is provided. Preferably, the vision system is a vision system for a vehicle (e.g., a car or truck).The vision system has at least one image capturing device (e.g. a camera) which is designed to capture image data. Preferably, the image data includes rearward view image data captured (e.g., laterally) adjacent to and / or behind the vehicle. The at least one image capturing device can be part of the vehicle and / or be arranged on the vehicle. For example, the at least one image capturing device can be mounted in or on a camera arm which extends laterally outwards from the vehicle body. Particularly preferably, the at least one image capturing device is designed such that it captures one or more legally prescribed fields of view, e.g. a class II field of view, a class IV field of view and / or a class V field of view according to the rules 2003 / 97 / EG and 2007 / 38 / EG.The vision system further comprises a state device (e.g. a brightness and / or light sensor). The state device is configured to generate a state signal which preferably indicates a day state or a night state. The state device can be part of the vehicle and / or be arranged on the vehicle.The image processing system further includes an image data processing device (e.g., a processor). The image data processing device is configured to receive the captured image data and the generated state signal. For this purpose, the image data processing device can be connected to the at least one image capturing device and the state device, e.g. via corresponding wired or wireless signal connections. The image data processing device is further configured to process the received image data in dependence on the received state signal. For example, the image data processing device can be configured to perform a method as described in this document. Consequently, the aspects described herein in connection with the method are also to be disclosed and claimable in connection with the vision system just described, and the same is also to be applied vice versa.According to one aspect, the vision system can further comprise a display device. The display device may be configured to display the processed image data. For example, the display device can have a display (e.g. an LCD display) which is preferably designed to display the processed image data. The display device can be part of the vehicle and / or be arranged on the vehicle. For example, the display device can be arranged in a driver's cab of the vehicle and / or in a cockpit of the vehicle, preferably in the field of vision of the driver. The display device can be connected to the image data processing device, for example via a wired or wireless signal connection. As described above, the display device may be configured to display different fields of view, e.g., the class II field of view, the class IV field of view, and / or the class V field of view, simultaneously, e.g., on one and the same display of the display device.According to a further independent aspect, a vehicle (e.g. motor vehicle) is provided. Preferably, the vehicle may be a commercial vehicle, such as a truck or a bus. For example, due to its construction and equipment, the vehicle may be designed specifically for the transport of goods, for the transport of persons and / or for towing one or more (e.g. agricultural) trailer vehicles. The vehicle includes a vision system as described herein. Consequently, the aspects described herein in connection with the vision system and the method are also to be disclosed and claimable in connection with the vehicle, and correspondingly applies in reverse.According to one aspect, the at least one image capturing device can be configured to capture different fields of view, preferably a class II field of view, a class IV field of view and / or a class V field of view (e.g. according to the 2003 / 97 / EG and 2007 / 38 / EG directive). The at least one image capturing device can have, for example, a plurality of cameras which are mounted, for example, in or on a common camera arm. The plurality of cameras may be oriented differently and / or each have different viewing angles. In this context, the image data processing device can be designed to select image data of a subset of the plurality of cameras and / or to combine / combine the image data of the plurality of cameras.Additionally or alternatively, the image processing system can be part of a mirror replacement system of the vehicle, in particular of an external mirror replacement system. The term mirror replacement system can relate, for example, to a (e.g. fundamentally known) driver assistance system in which (e.g. legally prescribed) mirror devices (e.g. an external rear mirror) of the vehicle are replaced by one or more image capturing devices, wherein the captured images of these image capturing devices are displayed on a display device in the vehicle. In this case, the image capturing devices in particular capture a section of the vehicle environment which would actually be displayed in the respective (e.g. legally prescribed) mirror apparatus. For example, the at least one image capturing device may be configured to capture image data corresponding to a main exterior mirror (e.g. field of view II), a wide-angle exterior mirror (e.g. field of view IV) and / or a short-range or ramp mirror (e.g. field of view V). Consequently, the at least one image capturing device can have and / or be at least one mirror replacement image capturing device.Additionally or alternatively, the image processing system may be capable of displaying a surround view of the vehicle environment on the display device. The term surround view, which can also be referred to as a bird-view view, can relate, for example, to a (e.g., fundamentally known) driver assistance system in which a (e.g., artificial) image of the vehicle environment and / or of the vehicle from the bird's eye view is provided to the driver. In this case, the at least one image capturing device can have a plurality of surroundings cameras (e.g. including a front camera, a rear camera and / or at least one side camera), wherein the surroundings cameras can preferably be oriented in different directions. Additionally or alternatively, the at least one image capturing device can be part of a combination of two or more image capturing devices, the image data of which are combined / combined to form a bird's eye view of the vehicle. In this context, the at least one image capturing device can preferably have at least four image capturing devices (e.g. all-round view cameras).The aspects and features of the disclosure described above may be combined in any desired manner. Further details and advantages are described below with reference to the attached figures. The figures show: FIG. 1 is a schematic illustration of a vehicle according to an embodiment; FIG. 2 illustrates two schematic representations of processed image data in the default mode, using different image processing settings, according to an embodiment; FIG. 3 is two schematic representations of processed image data in highway driving mode using different image processing settings; and FIG. 4 shows schematic representations of the processed image data in the standard mode, the maneuvering mode and the motorway travel mode.The embodiments shown in FIGS. 1, 2, 3 and 4 correspond at least partially, so that similar or identical parts are provided with the same reference numerals and for their explanation reference is also made to the description of the other embodiments or figures in order to avoid repetitions.FIG. 1 illustrates a vehicle 20 according to an embodiment of the disclosure. For example only, the vehicle 20 is shown in FIG. 1 as a truck. However, the vehicle 20 may be any type of vehicle, such as a car, a bus, a construction machine, an agricultural machine, or the like.The vehicle 20 has at least one image capturing device 12, a state device 14 and an image data processing device 16.The aforementioned components may be connected to each other. For example, the image data processing device 16 can be connected to the at least one image capturing device 12 and the state device 14, e.g. by corresponding signal connections (dashed lines). Each of the signal connections may be a cable connection and / or a wireless connection.Furthermore, the at least one image capturing device 12, the state device 14 and the image data processing device 16 can form a vision system 10 and / or be part of a vision system 10 (e.g. of a mirror replacement system) of the vehicle 20. Thus, the vehicle 20 can have a vision system 10 which comprises the at least one image capturing device 12, the state device 14 and the image data processing device 16.The at least one image capturing device 12 can be configured to capture image data (e.g. in real time). The at least one image capturing device 12 can have, for example, one camera or a plurality of cameras. The one or more cameras may be mounted in or on a camera arm that preferably extends laterally outward from the vehicle body (e.g., laterally outward from a cab of the vehicle). The at least one image capturing device 12 can have at least one image sensor (e.g. a CMOS image sensor and / or a CCD image sensor). The at least one image capturing device 12 can have one or more optical elements (e.g. an objective, a mechanical shutter, an electrical shutter and / or a diaphragm).The at least one image capturing device 12 can be configured to capture optical information about the environment of the vehicle. In particular, the at least one image capturing device 12 can be configured to capture forward-directed image data (of the vehicle environment) in front of the vehicle 20, to capture lateral image data (of the vehicle environment) next to (e.g. left and / or right next to) the vehicle 20 and / or to capture rearward-directed image data (of the vehicle environment) behind the vehicle 20. The at least one image capturing device 12 can preferably be designed to capture a rear-facing side view of the environment of the vehicle (along the vehicle 20). This rear-facing side view may also have (e.g. due to an arrangement of the at least one image capturing device 12 in front of the windshield of the vehicle 20) at least partially areas of the surroundings of the vehicle which are (directly) in front of and / or laterally in front of the vehicle. The at least one image capturing device 12 can be configured to capture fields of view of the vehicle environment, which fields of view are legally prescribed, e.g. according to Directive 2003 / 97 / EG and 2007 / 38 / EG.For example, the at least one image capture device 12 may be configured to capture a class II field of view that may have a rear, remote view laterally along the vehicle 20. The class II field of view may correspond to a field of view typically detected by a main exterior mirror of the vehicle 20.Additionally or alternatively, the at least one image capturing device 12 can be configured to capture a class IV field of view, which can have a wide-angle (rear-facing) view of the lateral environment of the vehicle 20. As compared to the class II field of view, the class IV field of view may cover a shorter, but wider, lateral portion of the environment of the vehicle. The class IV field of view may correspond to a field of view typically detected by a wide-angle exterior mirror of the vehicle 20.Additionally or alternatively, the at least one image capture device 12 may be configured to capture a class V field of view that may include a close-up view of a lateral region adjacent (and possibly partially in front of) a cab of the vehicle 20, particularly a lateral region adjacent the passenger side of the vehicle 20 and / or the cab of the vehicle 20. The class IV field of view may correspond to a field of view typically detected by a ramp mirror and / or a side range safety mirror of the vehicle 20.Consequently, the (received) image data may include class II field of view image data, class IV field of view image data, and / or class V field of view image data. Additionally or alternatively, the (received) image data can also comprise class I field of view image data and / or class III field of view image data.Furthermore, the at least one image capturing device 12 can have a plurality of (e.g. four) image capturing devices 12. The plurality of image capturing devices 12 may be arranged in a distributed manner on the vehicle 20. The different image capturing devices 12 can be oriented differently and / or each have different observation fields. For example, the at least one image capturing device 12 can have a rear camera, a front camera and / or a side view camera. The at least one image capturing device 12 or the plurality of image capturing devices 12 may be part of a mirror replacement system of the vehicle 20 (e.g. replacement of an exterior rearview mirror) and / or a bird-view system of the vehicle 20.The state device 14 can be designed to generate and / or output a state signal. Preferably, the state signal may indicate a day state or a night state. Additionally or alternatively, the state signal can also indicate a light state of the environment of the vehicle. For example, the state signal may have a first state signal value if a brightness of the vehicle environment is above a predefined value and a second state if the brightness of the vehicle environment is below this predefined value. The state device 14 can have a brightness and / or light sensor in this context. Additionally or alternatively, the state device 14 can also have a clock, a timer and / or an interface which is designed to receive an external input which indicates a day state and / or a night state.The image data processing device 16 may be configured to receive inputs from the at least one image capturing device 12 and the state device 14. In particular, the image data processing device 16 can be configured to receive the captured image data from the at least one image capturing device 12 and the generated state signal from the state device 14. The image data processing device 16 may include a memory and / or at least one processor (e.g., an image signal processor-ISP) for processing images and / or image data. The memory may be configured to store instructions executable by the at least one processor.The image data processing device 16 can be configured to process the received image data. For example, the image data processing device 16 can be designed to modify the received image data (e.g. a quality and / or a performance characteristic of the received image data) by image processing. The image data processing device 16 can preferably be configured to adjust or change a contrast, a digital amplification, a noise filtering, a white balance, a luminance, a brightness and / or a sharpness of the received image data.In this case, the image data processing device 16 can be designed to process the received image data as a function of the received state signal. The image data processing device 16 can thus be configured to process the received image data differently depending on the received state signal, which is described in more detail in connection with FIGS. 2, 3 and 4. In particular, the processing may be different depending on whether the received state signal indicates the day state or the night state. For example, the received state signal can trigger a day or night mode of the image data processing device 16 for processing the received image data. Depending on the state signal and / or the respective mode (day or night mode), the image processing settings for processing the received image data may vary. For example, the image data processing device 16 may be configured to adjust the contrast, the digital amplification, the noise filtering, the white balance, the luminance, the brightness and / or the sharpness of the received image data depending on the received state signal and / or the mode. In particular, the image data processing device 16 can be configured to optimize the image quality for the day state and / or the night state. For example, the image data processing device 16 may be configured to increase the contrast and / or to decrease the brightness of the received image data if the received state signal indicates the night state.In addition, the image data processing device 16 can be configured to output a control signal to the at least one image capturing device 12. The control signal may cause the at least one image capturing device 12 to change and / or adjust at least one image capture setting of the at least one image capturing device 12. In this context, the term image capture setting may refer to (hardware or software) settings of the at least one image capture device 12, for example, that determine the manner in which the (e.g., raw) image data is captured by the at least one image capture device 12. For example, the at least one image recording setting may include an image rate, an (analog) gain, a black level, an exposure time and / or an exposure threshold of the at least one image capturing device 12. In this case, the frame rate can define a number of images per second that the at least one image capturing device 12 attempts to deliver. The black value may define a brightness level corresponding to the darkest value of the at least one image capture device 12. By adjusting the black level, a possible dark current can be compensated. The exposure time may define a period of time during which the at least one image capture device 12 collects photons for each image. The exposure threshold may define a level at which saturation of the at least one image capture device 12 is achieved. Additionally or alternatively, the at least one image recording setting can also have a focus and / or a diaphragm of the at least one image capturing device 12.In particular, the image data processing device 16 can be designed to output the control signal to the at least one image capturing device 12 as a function of the received signal. Thus, the image data processing device 16 can be configured to output the control signal differently depending on the received state signal, which is described in more detail in connection with FIGS. 2, 3 and 4. For example, the output may be made different depending on whether the received state signal indicates the day state or the night state.Furthermore, the vehicle 20 can have a situation recognition device 15. The situation recognition device 15 can be connected to the image data processing device 16, for example by a corresponding signal connection (dashed line). The situation recognition device 15 can be part of a vision system 10 of the vehicle 20. Thus, the vision system 10 can also have the situation recognition device 15.The situation recognition device 15 can be designed to generate and / or output a situation signal. The situation signal may indicate a (e.g., current) speed, gear selection, and / or operating state of the vehicle 20. For this purpose, the situation recognition device 15 can be connected to corresponding control units and / or sensor devices of the vehicle 20, e.g. a speed sensor, a gear position sensor and / or a steering angle sensor. Furthermore, the situation recognition device 15 can be configured to receive position data from a (e.g. GPS) positioning system of the vehicle. The situation recognition device 15 can be designed to generate and / or output the situation signal based on the information received from the control units, sensor devices and / or the positioning system.The situation signal generated by the situation recognition device 15 can have, for example, a first situation signal value which indicates a reverse travel of the vehicle 20, for example when a reverse gear of the vehicle 20 is engaged. The situation signal may have a second situation signal value indicative of highway driving of the vehicle 20, e.g., when the speed of the vehicle 20 exceeds 60 km / h or 100 km / h. The situation signal may have a third situation signal value indicating maneuvering of the vehicle 20, for example when the speed of the vehicle 20 is below 25 km / h and / or (strong) steering movements are present.The image data processing device 16 can furthermore be configured to process the received image data depending on the received situation signal. Thus, the image data processing device 16 can be designed to process the received image data differently depending on the received situation signal. Thus, the received situation signal can trigger, for example, different modes of image data processing of the image data processing device 16 for processing the received image data. Thus, for example, the night mode can (automatically) adapt to different situations and / or operating states of the vehicle 20 (at night) as a function of the situation signal. In this case, the image processing settings can (automatically) change for different travel speeds and travel directions (forward / backward).For example, various predefined image data processing modes can be stored and / or defined in the image data processing device 16. The predefined image data processing modes can have, for example, at least three, preferably at least four, predefined image data processing modes. The various predefined image data processing modes may define different image processing settings used for processing the received image data. The image data processing device 16 can be configured to select an image data processing mode from the predefined possible image data processing modes as a function of the received situation signal. This can be done via predefined rules or mapping tables that map different values of the situation signal to a corresponding image data processing mode. Additionally or alternatively, an optimization method (e.g. a gradient-based Newton method and / or a genetic optimization) can also be used. Furthermore, the image data processing device 16 can be configured to process the received image data in accordance with the selected image data processing mode. For example, the manner in which the received image data is processed and / or the image processing settings used may be different depending on the selected image data processing mode.The predefined image data processing modes may include, for example, a default mode, a reverse mode, a highway driving mode, and / or a marshalling mode.The default mode illustrated in FIG. 2 may be, for example, an in-lane driving mode. The default mode may be activated by default unless the situation signal indicates another operating state of the vehicle, e.g., reversing, highway driving, and / or maneuvering. Additionally or alternatively, the default mode may be selected by the image data processing device 16 when the received situation signal indicates an in-vehicle driving. For example, the situation signal may include a fourth situation signal value indicative of an in-vehicle driving, e.g., when the speed of the vehicle 20 is between 25 km / h and 60 km / h.In the default mode, the image data processing device 16 may be configured to generate a combined view of the received image data (in a single image). To this end, different fields of view contained in the received image data can be merged. In particular, as shown in FIG. 4, for example, the image data of the class II fields of view, the class IV fields of view, and the class V fields of view may be combined and / or concatenated into a single image. In this context, the image data or the processed image data may have (e.g. perspective) distortions and / or deformations.The reverse mode may be selected by the image data processor 16 when the received situation signal indicates reverse travel of the vehicle 20. In the reverse drive mode, the image data processing device 16 may be configured to generate a wide-angle view of the received image data. Preferably, the received image data may include rearward image data (e.g., captured by a rear view camera). Particularly preferably, only rearward image data is processed during the step of processing in the rearward travel mode. In the reverse driving mode, the image data processing device 16 can be further configured to increase a contrast, a brightness and / or a digital amplification of the received image data compared to a contrast, a brightness and / or a digital amplification in the standard mode.The highway driving mode illustrated in FIGS. 3 and 4 may be selected by the image data processor 16 when the received situation signal indicates highway driving of the vehicle 20. In the highway driving mode, the image data processing device 16 may be configured to generate a zoom view of the received image data. The term zoom view may refer to, for example, a view that focuses on a particular area of the captured image data, e.g., a particular area of the exterior environment of the vehicle 20. As shown in FIG. 4, in highway driving mode, the class V field of view may be reduced (e.g., reduced) compared to the default mode and / or the marshalling mode. The received image data may be processed to minimize the information about the surroundings of the front and side vehicles and maximize the information about the surroundings of the rear vehicle. In the motorway driving mode, the image data processing device 16 can be further configured to reduce the contrast, brightness and / or digital gain of the received image data compared to contrast, brightness and / or digital gain in the standard mode.The maneuvering mode can be selected by the image data processing device 16 if the received situation signal indicates maneuvering of the vehicle 20. In the marshalling mode, the image data processing device 16 may be configured to generate a wide-angle view of the received image data. In this case, as shown in FIG. 4, the class V field of view may be increased compared to the standard mode and / or the highway driving mode. Moreover, the class IV field of view and / or the class II field of view may be reduced (e.g., reduced) compared to the default mode and / or highway driving mode. Furthermore, the image data processing device 16 in the marshalling mode can be configured to increase the contrast, the brightness and / or the digital amplification of the received image data compared to contrast, brightness and / or digital amplification in the standard mode. In this case, the coupling can be asymmetrical, i.e. the contrast can be increased much more (during maneuvering) than it can be reduced (during highway and / or long-route driving).Furthermore, the vehicle 20 can have a display device 18. The display device 18 may be connected to the image data processing device 16, e.g. by a corresponding signal connection (dashed line). The display device 18 may be part of the vision system 10 of the vehicle 20. Thus, the vision system 10 can furthermore have the display device 18.The display device 18 may be configured to display the processed image data (e.g. in real time, i.e. the situation displayed on the display device 18 preferably corresponds to the actual situation outside the vehicle 20). The display device 18 can be configured to receive the processed image data from the image data processing device 16. The display device 18 can have a display (e.g. an LCD display) and / or a monitor. The display and / or monitor may be touch sensitive (e.g., configured as a touchscreen). The processed image data can be displayable on a monitor and / or display of the display device 18.The display device 18 may be disposed on an instrument panel of the vehicle 20. Alternatively, the display device 18 can also be arranged on an A-pillar of the vehicle 20. The display device 18 and / or the entire vision system 10 can be part of a mirror replacement system of the vehicle 20. Thus, the display device 18 may be configured to display an area of the vehicle environment that is normally visible via a mirror (not present) of the vehicle 20. The indicator 18 may be permanently active during operation of the vehicle 20. Additionally or alternatively, the display device 18 and / or the entire vision system 10 may be operable to display a surround view (or bird-view representation) of the vehicle environment.The display device 18 may include a dimmer to adjust a brightness of the display and / or monitor of the display device 18. The dimmer can be designed, for example, as a rotary knob and / or button on the display device 18. The dimmer can, however, also be designed as a (virtual) button which is displayed on the (touch-sensitive) display and / or monitor. The display device 18 can be designed to generate and / or output a display brightness signal. The display brightness signal can indicate a (current) brightness of the display and / or of the monitor of the display device 18. For example, the display brightness signal may be proportional to a displacement or setting of the dimmer. The display brightness signal can indicate, in particular, a brightness of a backlight of a display and / or of a monitor of the display device 18. In particular, the image data processing device 16 can be designed to process the received image data as a function of the received display brightness signal. Thus, the image data processing device 16 can be designed to process the received image data differently depending on the received display brightness signal. For example, the image processing settings may be coupled to the dimmer, i.e., the manual brightness adjustment of the display. In particular, the image data processing device 16 can be configured to increase the contrast of the received image data if the display brightness signal indicates an increase in the brightness of the display and / or to decrease the contrast of the received image data if the display brightness signal indicates a decrease in the brightness of the display. Additionally or alternatively, the display brightness signal may be asymmetrically coupled to the image processing settings.Furthermore, the display device 18 and / or the vehicle 20 can have a control device 17. The control device 17 can have, for example, one or more touchscreens, a button or a switch. The control device 17 can be arranged on the display device 18 and / or integrated therein. Thus, the controller 17 may be part of the vision system 10 of the vehicle 20.The control device 17 can be configured to generate and / or output a user request signal, e.g. based on inputs and / or actuations of the control device 17. the user request signal can indicate a command and / or a request of a user, e.g. in connection with the (post-)processoring of the received image data. For example, the user request signal may indicate a selection and / or adjustment of an image data processing mode, a change in image processing settings, and / or a change and / or selection of a view (e.g., a zoom or wide angle view).In this case, the image data processing device 16 can be designed to process the received image data as a function of the received user request signal. Thus, the processing of the image data processing device 16 can be actively changed and / or adapted by a user. The image data processing device 16 can be configured to adapt the image processing settings depending on the received user request signal. In addition to or as an alternative to automatic adjustment, the driver may thus preferably also control the scope of the image processing settings manually. For example, the night mode image processing settings may be manually adjustable by the driver.For example, the image data processing device 16 can be designed to adapt the selected image data processing mode in dependence on the received user request signal and / or to select another image data processing mode in dependence on the received user request signal. For example only, the user may request a switch from the reverse mode to the maneuver mode (manually), e.g., by executing a corresponding control action on the controller 17 (e.g., by pressing a maneuver mode button of the controller 17). The control device 17 can then generate and output a corresponding user request signal which indicates the aforementioned change of the image data processing mode, and the image data processing device 16 can be configured to change the respective image data processing mode on the basis of the received user request signal.Additionally or alternatively, the image data processing device 16 can be configured to adjust the contrast, the digital amplification, the noise filtering, the white balance, the luminance, the brightness and / or the sharpness of the received image data depending on the received user request signal. For example, the user may (manually) request a change in the contrast of the (displayed) image data, e.g. by carrying out a corresponding control action on the control device 17 (e.g. by actuating a contrast key of the control device 17). The control device 17 can then generate and output a corresponding user request signal which indicates the aforementioned change in contrast, and the image data processing device 16 can be configured to change the contrast of the received image data based on the received user request signal.Additionally or alternatively, the image data processing device 16 can also be designed to adapt a region of the received image data which is to be displayed as a function of the received user request signal. For example, the image data processing device 16 can be configured to adapt an image region generated from the received image data and / or a view generated from the received image data for display depending on the received user request signal. In particular, the image data processing device 16 can be configured to generate a zoom view or wide-angle view of the received image data in dependence on the received user request signal. By way of example, the user may (manually) request a change in the view of a currently (displayed) zoom view to a wide-angle view, e.g., by executing a corresponding control action on the controller 17 (e.g., by pressing a wide-angle view key of the controller 17). The controller 17 may then generate a corresponding user request signal indicative of the aforementioned change in view, and the image data processor 16 may be configured to generate a wide angle view of the received image data based on the received user request signal.FIG. 2 shows an example of processed image data (e.g., displayed on the display 18) in the default mode. The left and right images show the same recording situation or the same received image data (raw data), but with different image processing settings during the processing of the received image data. For reasons of displayability, the differences relate primarily to the image brightness, which is not intended to mean a restriction, however. As can be seen, both processed image data exemplarily show a view of a road of an domestic city at night, in which exemplary case the received image data comprises rearward image data. By way of example, parts of the side surface of the vehicle 20 can be seen on the respective left parts of the left and right images. If the respective images are compared, it can be seen that the left image shows substantially only the road 22 which is illuminated by another vehicle 21 behind the vehicle 20, while the environment of the road 22 is relatively dark. In the right image, the image processing settings are changed so that the overall brightness of the image is increased and more details in the vicinity of the road 22 (e.g., parking cars) can be better recognized. By default, only one default setting of the image processing settings may be used (e.g., the settings that result in the left or right image). However, it may be possible for the driver to manually adjust the image processing settings, e.g., via the vehicle controller and / or vision system, which in this exemplary case results in the respective other image (right or left). Moreover, it should be mentioned again that other image processing settings, such as a contrast, a digital amplification and / or a sharpness of the received image data, can also be changed, but these can only be represented with difficulty in a line drawing.FIG. 3 shows an example of processed image data (e.g., displayed on the display 18) in highway driving mode. Here again, the left and right images show the same captured situation or received image data (raw data), but with different image processing settings that were used in processing the received image data. As in the case of Fig. 2, the different image processing settings for the sake of displayability mainly relate to the image brightness, which is not intended to restrict the subject matter in any way, however. As can be seen, both processed image data show, by way of example, a zoom view of a nightly freeway situation, wherein, in this exemplary case, the received image data comprise rearward image data. When comparing the respective images, it can be seen that another vehicle 21 traveling behind the vehicle 20 is seen in the left image, but the driver is not excessively blinded by the light of the other vehicle 21 because the overall brightness is reduced. In contrast, in the right image, the image processing settings are changed so as to increase the overall brightness of the image, resulting in a very bright image being displayed on the display device 18. By default, a default setting of the image processing settings may be used (e.g., the settings that result in the darker left image). However, it may be possible for the driver to manually adjust the image processing settings, e.g., via the vehicle controller and / or vision system, resulting in the right image in this example case. Moreover, it should be mentioned again that other image processing settings, such as a contrast, a digital amplification and / or a sharpness of the received image data, can also be changed, but these can only be represented with difficulty in a line drawing.FIG. 4 shows schematic representations of processed image data in the standard, maneuvering mode and in the highway driving mode, wherein the processed image data have different fields of view. In this example, the processed image data (and the received image data) includes image data relating to a class II field of view, a class IV field of view, and a class V field of view of the environment of the vehicle according to the rules 2003 / 97 / EG and 2007 / 38 / EG. In the processing step, the different fields of view may be merged into a combined view, wherein the relative ranges of the different fields of view in the combined view may vary depending on the respective image processing mode. For example, in the marshalling mode, the range of the class V field of view may be increased compared to the default mode. Consequently, the (identical) image data of the class V field of view or the class V field of view can be displayed larger on the display device 18 than in the standard mode. In addition, the range of the class IV field of view and the class II field of view can be reduced as compared with the standard mode. Thus, the class II and class IV fields of view can be displayed on the display 18 smaller than the standard mode. Additionally or alternatively, in highway driving mode, the range of the class V field of view may be reduced (e.g., reduced) compared to the default mode. Therefore, here, the image data of the class V field of view or the class V field of view can be displayed smaller on the display device 18 than in the standard mode.Although the invention has been described with reference to certain embodiments, it will be apparent to those skilled in the art that various modifications may be made and equivalents may be substituted without departing from the scope of the invention. Accordingly, the invention is not intended to be limited to the disclosed embodiments, but to include all embodiments falling within the scope of the appended claims. In particular, the invention also claims protection for the subjects and features of the dependent claims independently of the related claims.List of reference characters10 Vision system 12 Image capturing device 14 State device 15 Situation recognition device 16 Image data processing device 17 Control device 18 Display device 20 Vehicle 21 Other vehicle 22 Road

Claims

A method of operating an image data processing device (16) of a vision system (10) of a vehicle (20), comprising: receiving image data captured by at least one image capturing device (12), wherein the image data preferably comprises rear view image data captured laterally and / or behind the vehicle (20); receiving a state signal indicating a day state or a night state, preferably generated by a state device (14), e.g. a brightness sensor; and processing the received image data, preferably comprising adjusting a contrast and / or a brightness of the received image data depending on the received state signal.The method of claim 1, wherein the step of processing differs depending on whether the received state signal indicates the day state or the night state.The method of claim 1 or 2, wherein the step of processing comprises: adjusting a contrast of the received image data; and / or adjusting a digital gain of the received image data; and / or adjusting noise filtering of the received image data.Method according to one of the preceding claims, further comprising: outputting a control signal for the at least one image capturing device (12) depending on the received state signal, wherein the control signal causes the at least one image capturing device (12) to adapt at least one image capturing setting, preferably an image rate, a gain, an exposure time and / or an exposure threshold value, of the at least one image capturing device (12).Method according to any of the preceding claims, further comprising: receiving, preferably from a situation detector (15), a situation signal indicative of a current speed, gear selection and / or operating state of the vehicle (20); and wherein the step of processing further depends on the received situation signal.The method according to claim 5, wherein the step of processing comprises: reducing a contrast, a digital gain and / or a brightness of the received image data if the received situation signal indicates a long-distance travel of the vehicle (20); and / or increasing a contrast, a digital gain and / or a brightness of the received image data if the received situation signal indicates a maneuvering, preferably a parking process, of the vehicle (20).The method according to claim 5 or 6, further comprising: selecting an image data processing mode from, preferably at least three, more preferably at least four, predefined possible image data processing modes depending on the received situation signal; and wherein the step of processing is performed according to the selected image data processing mode.The method of claim 7, wherein the predefined possible image data processing modes comprise: a reverse drive mode, preferably comprising generating a wide-angle view of the received image data; and / or a highway drive mode, preferably comprising generating a zoom view of the received image data; and / or a marshalling mode, preferably comprising generating a wide-angle view of the received image data.The method of claim 8, wherein: the predefined possible image data processing modes further comprise a default mode, preferably an in-situ driving mode; wherein: the reverse driving mode comprises a higher contrast, a higher brightness and / or a higher digital gain than the default mode; and / or the highway driving mode comprises a lower contrast, a lower brightness and / or a lower digital gain than the default mode; and / or the marshalling mode comprises a higher contrast, a higher brightness and / or a higher digital gain than the default mode.The method of any preceding claim, wherein: the received image data comprises data relating to different fields of view, preferably a class II field of view, a class IV field of view and / or a class V field of view; and the step of processing comprises merging the different fields of view into a combined view.The method of claim 10, when dependent on any one of claims 7 to 9, wherein: relative ranges of the different fields of view in the combined view are varied depending on the selected image data processing mode, preferably wherein: in the highway driving mode, the class V field of view is decreased compared to the default mode; and / or in the ranging mode, the class V field of view is increased compared to the default mode.The method according to any one of the preceding claims, further comprising: receiving a display brightness signal from a display device (18), wherein the display brightness signal indicates a brightness of a display of the display device (18), preferably a brightness of a backlight of a display of the display device (18); wherein the step of processing is performed in dependence on the received display brightness signal, preferably such that a contrast of the received image data is increased if the display brightness signal indicates an increase in the brightness of the display and / or that a contrast of the received image data is decreased if the display brightness signal indicates a decrease in the brightness of the display.The method according to any of the preceding claims, further comprising: receiving a user request signal; and adjusting the image processing settings depending on the received user request signal, preferably: a) selecting another image data processing mode and / or adjusting the selected image data processing mode depending on the received user request signal; and / or b) adjusting a contrast, a digital gain and / or a noise filtering of the received image data depending on the received user request signal; and / or c) adjusting a section to be displayed of the received image data depending on the received user request signal.Vision system (10) for a vehicle (20), comprising: at least one image capturing device (12) which is designed to capture image data which preferably comprises rearward vision image data which is captured laterally and / or behind the vehicle (20); a state device (14), preferably a brightness sensor, wherein the state device (14) is designed to generate a state signal which indicates a day state or a night state; an image data processing device (16), wherein the image data processing device (16) is designed to receive the captured image data and the generated state signal and is furthermore designed to process the received image data in dependence on the received state signal, wherein the image data processing device (16) is preferably designed to perform a method according to one of the preceding claims.The vision system (10) of claim 14, further comprising: a display device (18) configured to display the processed image data, wherein the display device (18) preferably comprises a display configured to simultaneously display a class II field of view, a class IV field of view, and a class V field of view.Vehicle (20), preferably commercial vehicle, comprising: a vision system (10) according to claim 14 or 15.The vehicle (20) according to claim 16, wherein: the at least one image capturing device (12) is configured to capture different fields of view, preferably a class II field of view, a class IV field of view and / or a class V field of view; and / or the vision system (10) is part of a mirror replacement system of the vehicle; and / or the vision system (10) is executable to display a surround view of the vehicle environment on the display device (18).

Images