System and procedure for changing an image acquisition setting

Abstract

Imaging system comprising: a rear-mounted camera for imaging a target on a trailer; and a controller set up to establish a reference point on the target and evaluate image data with respect to the reference point to determine whether a change to an image capture setting of the camera is required; wherein the controller further determines whether a change is required based on a state input indicating whether a vehicle rear light is activated; and wherein, if the state input indicates that the vehicle rear light is activated, the controller subtracts a vehicle rear light specific color cast from an image captured by the camera.

Description

[0001] The disclosure made here relates generally to driver assistance and active safety technologies in vehicles and in particular to changes made to the image capture settings of a camera.

[0002] Reversing a vehicle towing a trailer is very challenging for many drivers. This is especially true for drivers unfamiliar with reversing vehicles with trailers, which can include those who rarely tow (e.g., rent a trailer, rarely use a personal trailer, etc.). One reason for this difficulty is that reversing a vehicle with a trailer requires steering inputs that are the opposite of those used when reversing a vehicle without a trailer, and / or requires braking to stabilize the vehicle-trailer combination before it starts to slide sideways. Another reason is that small steering errors are amplified when reversing a vehicle with a trailer, causing the trailer to deviate from the intended path.

[0003] To assist the driver when steering a vehicle with a trailer attached, a trailer reversing assistance system must know the driver's intention. A common assumption in known trailer reversing assistance systems is that a driver of a vehicle with a trailer intends to reverse straight ahead, and the system either implicitly or explicitly assumes a path with zero curvature for the vehicle-trailer combination. Unfortunately, most real-world trailer reversing scenarios involve a curved path, and thus, assuming a path with zero curvature would significantly limit the system's usefulness. Some known systems assume that a path is known from a map or route planner.In this respect, some known trailer reversing assistance systems operate on the condition that the trailer's reversing path is known before reversing begins, for example, from a map or a route planning algorithm. Unfortunately, such trailer reversing assistance systems are known to have a relatively complex human-machine interface (HMI) for specifying the path, obstacles, and / or the destination of the reversing maneuver. Furthermore, such systems also require some way to determine how well the desired path is being followed and to know when the desired destination or stop and orientation have been achieved, using approaches such as cameras, inertial navigation, or high-precision global positioning systems (GPS). These requirements result in a relatively complex and expensive system.

[0004] Another reason why reversing a trailer can prove difficult is the need to steer the vehicle in a way that limits the possibility of a sideways roll condition. A trailer has reached a sideways roll condition when, while continuing to reverse, the camber angle cannot be reduced (i.e., made less acute) by applying maximum control input to the vehicle, such as moving the vehicle's steered front wheels to a maximum controlled angle at a maximum rate of control angle change. Should the sideways roll angle be reached, the vehicle must be pulled forward to reduce the camber angle and thus allow it to be controlled by manipulating the vehicle's steered wheels.In addition to the awkward position that requires the vehicle to be pulled forward when in a sideways position, this can also lead to damage to the vehicle and / or trailer if certain operating conditions of the vehicle regarding its speed, engine torque, acceleration, and the like are not recognized and counteracted. For example, if the vehicle is traveling in reverse at a reasonably high speed and / or is subjected to a reasonably high longitudinal acceleration when the sideways position is reached, the relative movement of the vehicle with respect to the trailer can cause contact between the vehicle and trailer, thus damaging the trailer and / or the vehicle.

[0005] From the prior art, in particular DE 10 2011 083 786 A1, a camera system for a vehicle is known that uses information about the illumination status of the vehicle's lighting to color-correct recorded image data. It also takes into account information about whether a trailer is attached to the vehicle, which might reflect the light from the rear vehicle's lighting.

[0006] The present invention is based on the objective of providing an improved imaging system and imaging method.

[0007] This problem is solved with the subject matter of claims 1 and 4.

[0008] The invention relates to an imaging system comprising: a camera mounted on the rear of a vehicle for imaging a target on a trailer; and a controller configured to set a reference point on the target and to evaluate image data with respect to the reference point in order to determine whether a change to an image acquisition setting of the camera is required; wherein the controller further determines whether a change is required based on a state input indicating whether a vehicle rear light is activated; and wherein, if the state input indicates that the vehicle rear light is activated, the controller subtracts a vehicle rear light-specific color cast from an image captured by the camera.

[0009] The invention further relates to an imaging method comprising the following steps: using a camera to image a target provided on a trailer; providing a control for: setting a reference point on the target and evaluating image data with respect to the reference point to determine whether a change to an image acquisition setting of the camera is required; and further determining whether a change is required based on a state input indicating whether a vehicle taillight is activated, wherein, if the state input indicates that the vehicle taillight is activated, subtracting a vehicle taillight-specific color cast from an image captured by the camera.

[0010] These and other aspects, objects and features of the present invention will be understood and recognized by a person skilled in the art upon consideration of the following patent specification, claims and accompanying drawings.

[0011] The drawings contain: Fig. 1. A block diagram illustrating a system for changing an image acquisition setting of a camera used in a trailer reversing assistance system according to one embodiment; and Fig. 2 a flowchart of a routine for changing an image acquisition setting of a camera used in a trailer reversing assistance system according to one embodiment.

[0012] While various aspects of the invention are described with reference to a specific explanatory embodiment, the invention is not limited to such embodiments, and additional modifications, applications, and embodiments can be implemented without departing from the invention. The same reference numerals are used in the figures to represent the same components. Those skilled in the art will recognize that the various components listed here can be modified without departing from the scope of protection of the invention.

[0013] Referring to Fig. Figure 1 shows an imaging system 2600 according to one embodiment and is intended for use with a trailer reversing assistance system. The imaging system 2600 comprises a camera 2602 with an image sensor 2604 that detects light and converts it into image data. The camera 2602 can be mounted on the rear of a towing vehicle 2606 and positioned to image a target 2608 located on a trailer 2610 attached to the vehicle 2606. The target 2608 can be arranged in a rectangular array with a checkerboard pattern that is detectable by the camera 2602. In one embodiment, the checkerboard pattern alternates between a first color and a second color that differs from the first color. In one array, the first color is green and the second color is red. In another array, the first color is white and the second color is green.However, it should be understood that other target shapes, sizes, patterns and color schemes can be used.

[0014] The imaging system 2600 further comprises a control unit 2612, which may be integrated with the camera 2602 or located separately from it. The control unit 2612 may include circuitry such as a processor 2614 and memory 2616. A routine 2618 for changing an image acquisition setting, such as the white balance and exposure of the camera 2602, may be stored in memory 2616 and is executed by the processor 2614. In one embodiment, the control unit 2612 is configured to set a reference point corresponding to an area of ​​the target 2608 or pendant 2610 that has a known color.By knowing how the reference point should appear in a captured image, the controller 2612 can evaluate image data received from the camera 2602 and change the white balance and exposure of the camera 2602 to compensate for changing light conditions, such as when the vehicle 2606 and the trailer 2610 move from a sunny area to a shaded area.

[0015] In the illustrated embodiment, the controller 2612 can also communicate with a positioning device, represented as a GPS-enabled device 2620, for receiving input regarding the geographic location of the vehicle 2606 and trailer 2610. The GPS-enabled device 2620 can be any suitable device capable of communicating with the controller 2612. In one embodiment, the GPS-enabled device 2620 is such an onboard device. In another embodiment, the GPS-enabled device 2620 is a portable electronic device such as, but not limited to, a portable GPS device or a GPS-enabled smart device, both of which are capable of wireless communication with the controller 2612 via Bluetooth®, WiFi, the like, or a combination thereof.Since the lighting conditions can change depending on the geographical location of the vehicle, the control unit 2612 can take into account the location input supplied by the GPS-enabled device 2620 when deciding whether a change in the white balance and / or exposure of the camera 2602 is required.

[0016] Since lighting conditions can also change depending on the current time, date, and weather conditions, the controller 2612 can additionally receive time and date information via input 2622 and weather information via input 2624. One or both of these can be considered by the controller 2612 when deciding whether an adjustment of the white balance and / or exposure of the camera 2602 is necessary. For example, the light intensity in Florida during a clear summer afternoon will generally be higher than the light intensity in Michigan during a cloudy summer morning. Thus, by providing this type of information, the controller 2612 can predict certain characteristics regarding the light captured by the image sensor 2604 of the camera 2602 and change the image capture settings of the camera 2602 accordingly.Following the previous example, if a vehicle 2606 and trailer 2610 are located in Florida, the controller 2612 can choose to decrease the exposure of camera 2602 and select a white balance setting suitable for higher color temperatures, whereas if the vehicle 2606 and trailer 2610 are located in Michigan, the controller 2612 can choose to increase the exposure of camera 2602 and select a white balance setting suitable for lower color temperatures. It is considered that the controller 2612 can receive the time and date information via the GPS-enabled device 2620, a portable electronic device, the electronic control module (ECM) of the vehicle 2606, or any other suitable means. The weather information can be supplied to the controller 2612 via an application running on a portable electronic device, an onboard device, or any other suitable means.

[0017] In addition to the information mentioned above, the controller 2612 can receive input from one or more devices 2626 located on the vehicle 2606 and / or the trailer 2610. These devices may include, but are not limited to, light sensors, speed sensors, inertial sensors, directional compasses, and / or other cameras, which can be provided in forward-, rearward-, and side-facing configurations. By effectively employing some or all of the devices 2626 with other devices and inputs described above, the controller 2612 can determine the orientation of the vehicle 2606 and the trailer 2610 with respect to a light source such as the sun.In one embodiment, the controller 2612 can monitor the position of the sun using input received from a sun-tracking application 2628, which may be stored on an onboard device or on an external device such as a portable electrical device (e.g., a smartphone). This information can be combined with date, time, and weather information received from inputs 2622 and 2624, respectively, to enable the controller 2612 to make changes to the white balance and / or exposure of the camera 2602 based on the relative position and expected luminous intensity of the sun.

[0018] Additionally, the controller 2612 can monitor the orientation and direction of travel of the vehicle 2606 and trailer 2610 with respect to the sun via input received from directional compasses, speed sensors, inertial sensors, the GPS-enabled device 20, and / or a GPS receiver 2630 attached to the trailer 2610. Since the dimensions of the vehicle 2606 and trailer 2610 are typically known, the controller 2612 can compare the orientation and direction of travel information with the position of the sun to predict possible changes in lighting conditions resulting from the vehicle 2606 and / or the trailer 2610 blocking direct sunlight, which may affect the amount of light detected by the image sensor 2604 of the camera 2602.For example, in some cases, the vehicle 2606 and the trailer 2610 may be oriented towards the sun in such a way that the camera 2602 and / or the imaged area is flooded with sunlight, resulting in a relatively large amount of light being captured by the image sensor 2604 of the camera 2602. In other cases, the vehicle 2606 and the trailer 2610 may be oriented towards the sun in such a way that the vehicle 2606 and / or the trailer 2610 block direct sunlight, resulting in relatively less light being captured by the light sensor 2604 of the camera 2602. In each of these cases, the controller 2612 can use the inputs and devices mentioned above to determine whether an adjustment of the white balance and / or exposure of the camera 2602 is necessary. Furthermore, the controller 2612 can effectively utilize image data received from other cameras provided on the vehicle 2606 and / or trailer 2610.Additionally or alternatively, the controller 2612 can use light information received from one or more light sensors on board the vehicle 2606 and / or trailer 2610. The controller 2612 can compare the image data and / or the light information with the image data received from the camera 2602. Any differences between them can be taken into account when determining whether a change in the white balance and / or exposure of the camera 2602 is required.

[0019] According to one embodiment, the imaging system 2600 is designed to compensate for changing light conditions caused by the activation of the vehicle's rear lights 2631 2606. The rear lights may include tail lights, brake lights, auxiliary lights, and other types of rear lighting. When activated, the rear lights can project light onto the imaged scene, causing a sudden change in the lighting conditions. If this is not taken into account, the imaging system 2600 may experience difficulties tracking the target 2608.

[0020] The controller 2612 can receive status information from a vehicle lighting system 2632 indicating whether any of the vehicle taillights 2631 have been activated. Since the light characteristics (e.g., color temperature and intensity) are known for each given vehicle taillight 2631, the controller 2612 can subtract the color cast projected by an activated vehicle taillight 2631 from an image captured by the camera 2602. Additionally, the light intensity associated with the activated vehicle taillight 2631 can be compensated for either by modifying the illumination of each pixel of the image sensor 2604 or by compensating the image after it has been captured. In this way, the imaging system 2600 can respond with the appropriate white balance and / or exposure settings of the camera 2602 when one or more vehicle taillights 2631 are activated.

[0021] Referring to Fig.Figure 2 shows routine 2618 for setting the white balance and exposure of camera 2602 according to one embodiment. Routine 2618 begins in step 2634 and proceeds to step 2636 to check whether any vehicle taillights 2631 have been activated. As already described, this can be achieved by providing status information regarding the activation state of each vehicle taillight 2631 to the controller 2612. If one or more taillights 2631 have been activated, routine 2618 proceeds to step 2638, and the controller 2612 makes an initial change to the image acquisition settings of camera 2602. In one embodiment, the controller 2612 subtracts the color cast projected by the activated vehicle taillights 2631 from the captured image.Additionally or alternatively, the control unit 2612 compensates for the intensity associated with the activated vehicle taillights 2631 by either modifying the exposure of each pixel or by compensating for a captured image.

[0022] Once step 2638 is completed, or if no vehicle taillights 2631 have been activated (step 2636), routine 3618 proceeds to step 2640, and the controller 2612 receives one or more types of input. As previously described, the input may include image data provided by the camera 2602, location input received from the GPS-enabled device 2620 and the GPS receiver 2630, time and date information provided by the time and date input 2622, weather information provided by the weather input 2624, various inputs provided by one or more vehicle devices 2626, and sun tracking information provided by the sun tracking application 2628. Routine 2618 then proceeds to step 2642, and the controller 2612 determines whether or not to make a second change to the image acquisition settings of the camera 2602.When making such a determination, the controller 2612 can use some or all of the inputs mentioned above. If the controller 2612 decides that no change is needed, routine 2618 returns to step 2636. Alternatively, if the controller 2612 decides that a change is needed, routine 2644 proceeds, and the controller 2612 makes the second change, which may involve changing the white balance and / or exposure of camera 2602. After the second change, routine 2618 returns to step 2636.

[0023] It is understood that changes and modifications can be made to the above-mentioned structure without deviating from the concepts of the present invention, and it is further understood that such concepts are to be covered by the following claims, unless these claims expressly state otherwise by their language.

Claims

[1] Imaging system comprising: a rear-mounted camera for imaging a target on a trailer; and a controller set up to establish a reference point on the target and evaluate image data with respect to the reference point to determine whether a change to an image capture setting of the camera is required; wherein the controller further determines whether a change is required based on a state input indicating whether a vehicle rear light is activated; and wherein, if the state input indicates that the vehicle rear light is activated, the controller subtracts a vehicle rear light specific color cast from an image captured by the camera. [2] Imaging system according to claim 1, wherein the image acquisition setting comprises at least one of a white balance and an exposure of the camera. [3] Imaging system according to claim 1, wherein, when the state input indicates that the vehicle rear light is activated, the control performs one of modifying an exposure of each pixel of the image sensor and compensating an image after its acquisition by the camera. [4] Imaging method comprising the following steps: using a camera to image a target provided on a trailer; providing control for: setting a reference point on the target and evaluating image data with respect to the reference point to determine whether a change to an image capture setting of the camera is required; and further determining whether a change is required based on a state input indicating whether a vehicle rear light is activated, wherein, if the state input indicates that the vehicle rear light is activated, subtracting a vehicle rear light specific color cast from an image captured by the camera. [5] Imaging method according to claim 4, wherein the image acquisition setting comprises at least one of a white balance and an exposure of the camera.

Images