Control system for rear view device in vehicle

Abstract

A control system for a rear view device in a vehicle. A vehicle control system (10) and corresponding method for controlling a rear view device, such as a mirror (12), monitoring a favorable viewing point (14) and the movement of a driver (11) with a driver monitoring system (13) to detect features and classify behaviors / features as, for example, tilting towards or away from the mirror (12). Detection of motion is used to dynamically deflect the mirror to adjust its field of view. Such adjustments are useful in situations such as before overtaking (e.g., seeing blind spots) and upon parking (e.g., automatically changing the field of view to a lower angle to see the rear wheels and their close curbs). The system may alternate between multiple modes, such as from a default driving mode in which the long-term posture of the driver is monitored to determine whether fine adjustments to the rearview mirror are required to compensate for the driver's tilt over time.

Description

Technical Field

[0001] The present disclosure relates to a control system for a rearview device in a vehicle, and more particularly to an adaptive system for controlling the display angle of an object in a rearview mirror or display relative to a driver. Background Art

[0002] In road vehicles, rear-view mirrors and side-view mirrors are critical for safe driving, ensuring that the driver has an adequate view of any other vehicles and / or other road users behind or immediately adjacent to his vehicle. However, once the mirrors are initially adjusted by the driver (e.g., for relative height and comfort of the mirrors), the field of view (FOV) in the mirrors typically remains fixed in the vehicle's reference frame. As a result, visibility in the mirrors is adequate if the driver's viewing perspective remains within a limited range of motion within the vehicle's cabin, but visibility through the mirrors becomes suboptimal once the driver changes his driving position. The driver may wish to readjust the mirrors to obtain a better view, but this is not necessarily easy or safe to do during driving.

[0003] Another limitation of having the mirror field of view fixed in the vehicle reference frame is that the standard or default adjustment of the mirror is effective only for driving in "normal" driving situations, where the driver's vantage point remains the same as during the mirror adjustment prior to driving. For example, it is expected that the driver will require visibility in a forward direction for normal driving on a street or highway, and the driver tends to adjust the mirror position relative to the middle position of their head when stationary.

[0004] However, if the road is not straight, i.e., when driving on roads with high curvature or rapidly changing slopes, the mirrors may not provide adequate visibility of critical objects. Some visibility limitations are also related to the geometry of the car, such as the well-known rear quarter blind spot. As mentioned above, attempts at manual adjustments (including activating the directional buttons of the electric mirror adjustment system) are dangerous while driving because attention is diverted from the road.

[0005] Limited visibility in vehicle mirrors has been addressed in the prior art. For example, in one set of solutions, it is proposed to better estimate the vehicle's own driving scene / surroundings, i.e. by determining whether the vehicle is driving on a curve or on a hill and making a predefined adjustment to the mirror position with a subsequent effect on the viewing angle. This adjustment is achieved by monitoring signals available from onboard sensors of the vehicle, such as steering wheel angle, yaw rate, state of turn indicators or GPS coordinates.

[0006] A second method for dynamically adjusting mirrors during driving attempts to monitor the driver's vantage point based on available data about the driver's body position. This can be done indirectly, for example by measuring the orientation of various components of the driver's seat. Examples of existing patent publications relevant to the background of the present invention include: US7571041, US8297763 and US2010 / 0177413. Summary of the invention

[0007] In light of the foregoing, there is a need to provide further adjustment options for mirror systems in vehicles.At a minimum, the present invention seeks to provide an alternative automatic rear view device control system for drivers and vehicle manufacturers to choose from.

[0008] According to a first aspect, a control system is provided for adapting a mirror or other rear-view display device to a variety of driving scenarios for a vehicle. The vehicle system includes: at least one rear-view device (e.g., a reflector or video display); a camera for capturing image data associated with an occupant within the vehicle (e.g., a driver's vantage point); and at least one processor configured to monitor characteristic deviations and / or movement of at least a portion of the occupant's body based on the captured image data, and to cause a change in the field of view of the rear-view imaging device in response to detecting the characteristic deviations and / or movement. In a particular form, the adjustment in the FOV is proportional to the detected behavior / motion, as opposed to, for example, a predetermined adjustment.

[0009] Thus, a solution is provided for adjusting the field of view (FOV) in one or more rearview mirrors based on the user's behavior / characteristics and movements (e.g., changing vantage point). In other words, the rearview image can be adjusted for an optimal FOV based on the driver's needs. The vehicle system is able to determine the vantage point of the occupant (e.g., eye height) and decide whether the vantage point can be improved, i.e., by adjusting the rearward-facing FOV.

[0010] In the context of this disclosure, the term "rear view device" or "image device" is a safety device used to enable the driver to see behind his seat, whether it is through a reflective mirror (e.g., the classic "wing" rearview mirror or a central "rear view" rearview mirror) or a video display for displaying an image associated with a rear-facing camera, which can be located on a side or center console. Typically, the rear view image will be a mirror image, so that moving objects behind the observer can be intuitively tracked by seeing the image of the moving object in front of the observer.

[0011] In other words, the proposed interaction method is compatible not only with physically actuated mirrors, but also with camera-based rearview mirrors (electronic mirrors) that are becoming more common. In fact, although the mirror is virtual, by changing the image perceived by the driver according to the principles of classical ray optics, the disclosure herein can make eMirror technology more intuitive and therefore easier for drivers to adopt. For example, in one form, when the driver leans toward the screen, the rearview camera can enlarge the image, and when the driver leans back, the image is reduced to have a wider view of the surrounding environment.

[0012] In an embodiment, the vehicle system is configured to operate in at least two modes. The first mode may correspond to monitoring the behavior and / or movement of at least a portion of the occupant's body over an extended period of time. In this way, changes in the driver's posture may be detected and the FOV (viewing angle) may be adjusted to account for changes in posture over time.

[0013] The second mode may correspond to monitoring the behavior and / or movement of at least a part of the occupant's body for more urgent (i.e., within a short period of time) movements. In this way, a change in the driver's position, such as a tilt in a specific direction, may be detected and the FOV (viewing angle) temporarily adjusted to an enhanced FOV. For example, if the driver leans to one side, this indicates a desire to see a greater angle relative to the opposite side of the vehicle, such as for overtaking. Preferably, the adjustment returns to a pre-adjustment position corresponding to straight road driving, in which the driver's body is in a neutral position.

[0014] In an embodiment, if the monitored behavior / motion exceeds a predetermined threshold, the processor generates a control signal. The control signal may activate a motor associated with the rearview mirror to angularly adjust the mirror, or in the case of an "eMirror", to move the camera or otherwise angularly adjust its FOV for corresponding display on the mirrored rearview display.

[0015] In an embodiment, the processor is configured to track the position and / or orientation of the driver's head from the image data captured by the image data. In this way, it can be determined whether the driver: (i) looks forward, but their position changes over time (e.g., slides / sinks into their seat); (ii) looks at the rearview mirror; and / or (iii) the driver leans toward or away from the rearview display.

[0016] In this way, the system enables the vehicle mirrors to be readjusted continuously (or periodically or suddenly at intervals) relative to the driver's current viewing point. In particular, the present invention recognizes the additional requirements for in-cabin cameras and driver monitoring in new cars. The system recognizes and utilizes advanced driver monitoring systems that are already able to track the position and orientation of the driver's head and the direction of their gaze. The collected data can be processed to directly measure the driver's vantage point. Position data can also be evaluated to detect the driver's intention to check a specific area around the car.

[0017] In particular, available driver monitoring systems (DMS) with 3D tracking capabilities report the spatial transformation (translation and rotation) of the driver's head in an arbitrary coordinate system defined in the vehicle's reference coordinate system. This enables the derivation of an estimate of the point from which the driver is viewing the car and its surroundings (the vantage point). For simplicity, it is generally assumed here that the viewpoint is the midpoint between the driver's eyes.

[0018] The disclosure herein enables different use case scenarios, such as adjusting rearview mirrors to eliminate blind spots, adjusting during parking, highlighting objects, etc. The present invention is embodied by algorithms, systems, and use cases / patterns.

[0019] A method according to the present invention is outlined; for example, comprising the following steps: determining a driver's vantage point relative to a rear view imaging device based on data captured by a driver monitoring system; determining whether behavior corresponding to movement of at least a portion of the driver's body captured by the driver monitoring system causes a deviation from the determined vantage point; and adjusting the field of view of the rear view imaging device in response to the deviated vantage point.

[0020] In this way, the driver's vantage point can be determined and, if necessary, the FOV of the rearview mirror can be adjusted to improve the vantage point. Compensation for deviations in the vantage point is a possible mode of operation of the control method, i.e., suitable for conventional forward driving situations. Another mode is to use the deviation as a trigger for other types of mirror adjustments in anticipation of the driver's intention, such as amplifying deviations towards a blind spot.

[0021] According to a third aspect, there is provided a non-transitory computer readable medium containing instructions, which when executed by one or more processors perform the aforementioned method. According to another aspect, there is provided a computer software product containing instructions, which when executed by one or more processors perform the aforementioned method. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Illustrative embodiments will now be described with reference to the accompanying drawings, in which:

[0023] Figure 1A schematic diagram showing the position of a vehicle mirror system (i.e., rearview mirror) relative to the driver's head;

[0024] Figure 2A and Figure 2B A schematic plan view and a detailed view, respectively, of a vehicle mirror system according to a first scenario; and

[0025] Figure 3A and Figure 3B Schematic diagrams of a vehicle mirror system according to a second scenario are shown, each schematic diagram having a plan view and a detail view. DETAILED DESCRIPTION

[0026] The following description provides exemplary embodiments and is accompanied by Figure 1 It is used to explain the principles of the present invention. However, the scope of the present invention is not intended to be limited to the precise details of the embodiments or to the exact consistency of all features and / or method steps, because the changes will be obvious to those skilled in the art and are considered to be covered by the specification. The terms used for components herein should be given a broad interpretation, which also includes equivalent functions and features. In some cases, several alternative terms (synonyms) for structural features have been provided, such as by using the " / " symbol to imply interchangeability, but such terms are not intended to be exhaustive. Descriptive terms should also be given the broadest interpretation; for example, the term "comprising" used in this specification means "at least partially composed of...", so that each statement including the term "comprising" in this specification is interpreted, and there may also be features other than the features at the beginning of the term. Related terms such as "including" and "comprising" are interpreted in the same way. Directional terms such as "vertical", "horizontal", "up", "down", "lateral", "above" and "below" are used for ease of explanation, usually with reference to the forms shown in the illustrations, and if equivalent functions can be achieved with alternative sizes and / or directions, they are not intended to be ultimately limited. All directional terms are related to each other.

[0027] The description herein relates to embodiments with a specific combination of steps or features, however, it is contemplated that further combinations and cross-combinations of compatible steps or features between the embodiments will be possible. In fact, separate features can function as the present invention independently of other features, and are not necessarily required to be implemented as a complete combination.

[0028] It should be understood that the illustrated embodiments show applications for illustrative purposes only. In fact, the present invention can be applied to many different configurations, where the embodiments are straightforward to implement for those skilled in the art.

[0029] Figure 1A rearview mirror system is shown. This represents the general concept of the invention, simplified to two dimensions. In particular, the system 10 operates based on the spatial relationship between the driver's vantage point, known from the coordinate system {H} of the driver's head 11, and the surface of the mirror 12, which has its own coordinate system {M}. The reflective surface is described by a function of the general form:

[0030] f m (x,y,z|θ,φ)

[0031] Here, θ and φ are the azimuth angles of the rearview mirror 12 controlled by the actuator.

[0032] Assume that the most extreme point P of a predetermined target area T , when viewing the mirror from the vantage point of the driver's head 11, the mirror should be adjusted so that the reflection from that position appears on the edge of the mirror 12. The spatial transformation between the coordinate system {C} of the DMS camera 13 and the mirror coordinate system {M} must be known in order to find the relative position of the driver's vantage point in the mirror coordinate system, which is essential for calculating the angle of incidence of the line 14 originating from the vantage point at the driver's head 11 and reflected at the mirror 12 surface.

[0033] The basic principle described above can be implemented as dynamic mirror adjustment with respect to at least two automatic behaviors. For example, the driver's FOV in the rearview mirror is enhanced for the following two situations: (1) regular driving situations, where the vehicle moves forward for a long time without lateral maneuvers, but the driver may change position unintentionally, and (2) lane change / turn situations, where the driver needs to carefully check for any surrounding vehicles to the left and right of the ego vehicle and make conscious / voluntary movements. A third behavior can be observed during parking, where the driver wants to see the low angle, where the curb can be seen relative to the rear wheels, again through intentional movements.

[0034] refer to Figure 2A and Figure 2B The first mode, ie, the forward driving situation, is outlined.

[0035] An exemplary first mode is intended to track the average vantage point over a long period of time based on the long term distribution of vantage points measured by the DMS system and adjust the mirrors accordingly This embodiment helps maintain optimal adjustment of the mirrors in situations where the driver's position gradually changes over time (e.g., the driver may gradually hunch further into the driver's seat while driving long distances).

[0036] The vantage point distribution is modeled using a three-dimensional function, such as a Gaussian function, which is continuously updated using new measurements of the head position. The expected value of the model function is assumed to be the most typical vantage point. The vantage point is used to adjust the side view mirror 12 in such a way that the mirror 12 reflects the maximum portion of the predetermined target area 15. According to Figure 2A , shows the automatic adjustment of the side view mirror 12, wherein the dashed line represents the FOV before adjustment and the solid line represents the FOV after adjustment based on the average vantage point.

[0037] Compared with earlier DMS measurements, the most recent DMS measurements have a greater impact on the vantage point distribution monitoring. In this way, when calculating the distribution, measurements earlier than the predetermined threshold period are ignored. The rejection threshold can be a tunable parameter that enables a good compromise between the responsiveness of the system (i.e., its ability to always provide the best possible FOV) and the stability of the mirror. It is worth noting that the fast / unsteady adjustment of the mirror may be undesirable or even disorienting for the driver. Therefore, the mirror adjustment in the first mode is actually imperceptible to the driver, and a gradually proportional movement is achieved according to the current head position of the driver.

[0038] It should be clear from the foregoing that body and / or head position is monitored over time to enable the mirror control system to achieve fine readjustment that provides the optimal FOV for normal forward driving scenarios. Figure 2B A schematic representation of a driver's head is shown with a distribution of position points (e.g., a bell curve) plotted, such as the distribution of the nearest vantage points along two dimensions of space, indicating that the system creates and continually updates a model of vantage points to determine the most typical vantage point and all instantaneous deviations from the most typical vantage point. In reality, the distribution will be a three-dimensional function, and what is shown is a schematic representation of it. This is simply to help monitor and determine if the head has deviated from the middle position where the mirror is set, indicating a new vantage point and the need to adjust the mirror's FOV. It is worth noting that both mirrors can be adjusted simultaneously to account for position changes in all modes.

[0039] A variation of the above scenario is also useful when multiple drivers of different body sizes share the same car, as there is no need to (remember to) manually re-adjust the mirrors each time a driver changes. Mirror adjustment for a new driver can be achieved in a shorter period of time, i.e. starting when the new driver first starts operating the vehicle, the DMS can detect the size of the new driver / body and immediately adjust the rearview device FOV accordingly.

[0040] refer to Figure 3A and Figure 3BA second mode corresponding to a lateral maneuvering scenario (blind spot peek / mitigation) is shown.

[0041] First, by way of background, it is important to note that with conventional (i.e., effectively fixed in a semi-permanent position until manually adjusted) mirror usage, the driver may be observed to expand the FOV slightly toward the left or right side of the vehicle by tilting in the opposite direction. However, such an expanded field of view is generally not sufficient to allow the driver to see vehicles in the rear quarter blind spot. For example, Figure 3A The conventional situation prior to adjustment according to the invention or with fixed mirrors is shown, wherein leaning to the right while looking at the left mirror provides only a slight enhancement of blind spot visibility indicated by the solid line of area 16, compared to the dashed line of the driver in the normal forward driving position (not leaning, but just looking at the mirror).

[0042] However, when properly detected using the camera 13, the tilting motion itself is a useful indicator of the driver's intent to check the blind spot and can be used as monitored by the DMS to trigger temporary deflection of the mirror 12 and help the driver see the desired area (e.g. Figure 3B 17 in the blind spot, which in the illustration includes a behavior event of a second vehicle that was previously in the blind spot.

[0043] According to this mode, the act of leaning to the right triggers a deflection of the mirror (the edge closest to the driver pivots toward the driver and the mirror rotates clockwise as a whole), which amplifies the change in FOV caused by the movement of the head tilt compared to what would normally be possible with a fixed mirror, thereby enabling the driver to see a second vehicle in the blind spot.

[0044] In an exemplary form, the mirror may be adjusted by detecting a head rotation toward the mirror (because it is assumed that the driver is looking at the mirror for driving assistance) and a tilting motion. Detection of one or both of these motions indicates that the driver wishes to obtain a better view / vantage point. Thus, upon detection of the described behavior, the mirror is temporarily deflected to provide a better view until the end of the motion is detected (i.e., the head returns to the middle / centered / facing forward) and / or until a predetermined period of time has passed, such as 3 to 8 seconds.

[0045] In accordance with the foregoing, the mirror / rearview display angle control system of the present invention may have a first mode and a second mode, the first mode involving a normal driving condition with the driver's head in a generally neutral forward facing position, the second mode involving activation of a "blind spot peek" feature in response to the DMS system detecting a significant momentary deviation from the average vantage point toward the left or right side of the vehicle, or up or down. Such deviation triggers the second mode and changes the FOV of the mirror (e.g., proportionally) as if it magnifies the normally occurring change in viewing angle due to tilt / extension.

[0046] As described above, after activating the second mode, the mirror may return to its original position after a predetermined (i.e. short) period of time, or when the DMS system detects that the measured vantage point has returned to near its average position (e.g. the driver is no longer leaning to one side).

[0047] The illustrated embodiment shows an example of a left side mirror on a left-hand drive vehicle, however, it is apparent that the right side mirror can similarly automatically deflect upon detecting a tilt or similar motion event. The same principles apply to right-hand drive vehicles to mitigate the driver's side blind spot. Furthermore, both mirrors can be adjusted simultaneously upon detecting a motion event.

[0048] The principles of the invention may also be applied to digital mirrors, where the camera FOV and corresponding display are modified based on DMS observations, ie in response to motion events of the driver's body and / or head to provide the driver with an optimal field of view.

[0049] Additional or supplementary modes of the control system may implement prior art features, such as monitoring the curvature of the road and / or the position of the driver's seat to obtain additional data for determining the vantage point.A variation may use activation of a turn signal as a trigger for FOV adjustment.

[0050] Of particular note with respect to the concepts disclosed herein is the ability to activate modes such as overtaking through intentional motion, such as a driver-initiated action that is a predefined, intuitive / natural, and obvious movement of the body. The detected motion to switch modes should be effortless and not interrupt the driving process, as would manual adjustment of a mirror. The FOV can be adjusted without the need for an additional triggering event, such as activation of an indication signal.

[0051] The proposed interaction method disclosed here is compatible not only with physically actuated mirrors, but also with camera-based rearview mirrors (electronic mirrors). In fact, the present concept can make the technology more intuitive and therefore easier to adopt by drivers, i.e., although the mirror is virtual, it changes the image perceived by the driver according to classical light optics principles.

[0052] In another example, a rear-view camera can zoom in on an image when the driver leans toward the screen, and zoom out on an image when the driver leans back, providing a wider view of the surrounding environment.

[0053] Specific use case scenarios for implementing the concepts disclosed herein include: adjusting mirrors to eliminate blind spots and adjusting during parking (e.g., by detecting the driver's upward reaching motion to make themselves taller / shorter and obtain a higher / lower angle view with a better vantage point of the vehicle's rear tires to avoid hitting / scratching the curb. This is particularly useful for adjusting the passenger side mirror, as this side mirror is often the side adjacent to the curb and the side farthest from the driver during parallel parking (where it is more difficult to gauge the width of the vehicle).

[0054] Changes in FOV may have a corresponding effect on objects that are identified / highlighted in the digital image display.

[0055] From the above it should be clear that actions and movements can be detected in three-dimensional space, for example, leaning in one direction and / or when lifting up from the driver's seat to give a greater height, which can trigger complementary / amplified (e.g., proportional) mirror adjustments that are also three-dimensional, such as yaw in opposite directions and tilt downward simultaneously.

[0056] Generally, the present invention can be summarized as an improvement in vehicle rearview control systems, with particular application to reflective and electronic mirror technology.

[0057] Broadly, a vehicle's digital mirror system includes a camera located at a position that simulates a rearward / side facing mirror.

[0058] In summary, a vehicle control system as described herein utilizes a driver monitoring system to monitor, for example, a vantage point associated with the driver's movements to detect and classify behaviors / features such as leaning toward or away from a mirror. Detection of changes in vantage points can be used to dynamically deflect the mirror to adjust its field of view. Such adjustments are useful in situations such as before overtaking (e.g., to see a blind spot) and when parking (e.g., automatically changing the field of view to a lower angle to see the rear wheels and their proximity to the curb). The movement can be proportional or non-proportional. For example, the mirror can be deflected by an amount proportional to the amount of tilt, even if the deflection is amplified; alternatively, a slight tilt above a threshold is sufficient to indicate an intention to obtain a better vantage point, and the mirror can be deflected by a maximum or appropriate amount.

[0059] The system can alternate between multiple modes, such as from a default driving mode in which the driver's long-term posture is monitored to determine whether fine adjustments to the mirrors are needed to compensate for the driver's lean over time, and one or more modes that respond to more immediate, voluntary, and intentional movements.

[0060] Essentially, the occupant's vantage point is monitored to classify features based on any deviations (i.e., changes from the driver's starting position) from the occupant's vantage point. If a feature is identified, the system may choose to adjust the rear-view device (e.g., display or mirror).

[0061] In embodiments, additional input data may be collected, such as back-up signals, indicators, etc. The feature may be categorized as a gradual departure over time, a parking maneuver, overtaking, or other suitable driving circumstances.

[0062] In certain forms, the system may include a training function, such as to help improve the use of the system over time. For example, the driver / passenger can provide feedback, such as confirming the successful implementation of a classification feature. For safety reasons, such training can be performed while the vehicle is stationary or in motion.

Claims

1. A vehicle system, comprising: at least one rear-view device; sensors for capturing data related to a vantage point of an occupant within the vehicle; as well as at least one processor, Wherein the processor is configured to determine a vantage point feature from the captured data and generate a signal for adjusting the field of view of the at least one rear-view device based on a classification of the determined vantage point feature.

2. The vehicle system according to claim 1, wherein: The processor is configured to determine the classification by reference to an average or median vantage point of the occupant determined from the captured data.

3. The vehicle system of claim 2, comprising a forward travel detection mode, wherein: The processor is configured to classify the vantage point features according to deviations from an average or median vantage point corresponding to posture changes over an extended period of time.

4. A vehicle system according to any one of claims 1 to 3, comprising a temporary motion detection mode, wherein: The processor is configured to classify the vantage point feature based on a temporary change in the occupant's vantage point.

5. The vehicle system according to claim 4, wherein: The processor is configured to restore the field of view of the at least one rear-view device to a state prior to detecting the temporary change.

6. The vehicle system according to claim 5, wherein: Restoration of the field of view occurs after a predetermined time or when the end of the temporary movement is detected.

7. The vehicle system according to any one of claims 4 to 6, wherein: The temporary change in the occupant corresponds to an attempt to lean and / or increase / decrease height relative to the at least one rear-view device.

8. The vehicle system according to any one of claims 1 to 7, wherein: The at least one rear-view device is a mirror, and the vehicle system further includes an actuator configured to adjust the mirror in response to the signal.

9. The vehicle system according to any one of claims 1 to 7, wherein: The at least one rear-view device is a visual display for displaying an image associated with a rear-facing camera mounted on the vehicle.

10. The vehicle system of claim 9, configured to cause a zoom-in of the image based on the classified vantage point characteristics.

11. The vehicle system according to any one of claims 1 to 10, wherein: The processor accepts additional input from the occupant for one or more of: execution of a training function, a reverse function, activation of an indicator.

12. A method of controlling a rear view function in a vehicle, the method comprising the steps of: determining a driver's vantage point relative to a rearview device based on data captured by a driver monitoring system; classifying vantage point features corresponding to the determined changes in vantage points; as well as A field of view of the rearview device is adjusted in response to the determined behavior.

13. The method according to claim 12, wherein: The vantage point characteristic corresponds to a change in the driver's vantage point over an extended period of time.

14. The method according to claim 12, wherein: The vantage point feature corresponds to a temporary change in the driver's vantage point for a short period of time, and wherein the adjustment of the field of view is temporary and returns to a pre-adjustment state after a predetermined period of time and / or at the end of a movement.

15. A non-transitory computer readable medium containing instructions which, when executed by one or more processors, perform the method of any one of claims 12 to 14.

Images