System and method for automatically starting vehicle turn lights

Abstract

A method for automatically starting a turn light includes monitoring a vehicle occupant of a vehicle with a driver monitoring system. The driver monitoring system includes at least one camera and is configured to determine a position of at least one eye of a vehicle occupant, a head orientation of the vehicle occupant, and a shoulder movement. The method includes detecting whether a line of sight of a vehicle occupant faces a side view mirror of the vehicle or detecting a shoulder movement of the vehicle occupant, and in response, automatically activating a turn light of the vehicle.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to systems and methods for automatically activating a vehicle turn signal. BACKGROUND

[0002] This introduction generally introduces the background of the present disclosure. To the extent that the descriptions in this introduction describe the work of the present inventors, and to the extent that the described work was not previously published, the present inventors hereby place it in the public domain. To the extent that the descriptions in this introduction describe work by others, this work was made available to the public by the date and time of the filing of this patent application, and is not admitted to be prior art against the present disclosure by virtue of being included in this background.

[0003] Vehicle drivers do not always activate turn signals when changing lanes. Therefore, it is desirable to develop a system and method for automatically activating a turn signal when a vehicle operator changes lanes. SUMMARY

[0004] The present disclosure describes a method for automatically activating a vehicle turn signal. The method further includes monitoring a vehicle occupant of a vehicle with a driver monitoring system. The driver monitoring system includes at least one camera and the driver monitoring system is configured to determine a position of at least one eye of the vehicle occupant, a head orientation of the vehicle occupant, and a shoulder movement. The method further includes detecting whether a line of sight of the vehicle occupant is directed toward a side mirror of the vehicle. The method further includes automatically activating a turn signal of the vehicle in response to determining that the line of sight of the vehicle occupant is directed toward the side mirror of the vehicle. The method described in this paragraph improves vehicle technology by automatically activating a turn signal.

[0005] In an aspect of the present disclosure, the method can include monitoring a steering wheel of the vehicle for movement. The method can include monitoring a steering wheel angle of the vehicle steering wheel. The method can include determining that the steering wheel angle is greater than a predetermined angle threshold. The turn signal of the vehicle is automatically turned on in response to: (a) detecting that the line of sight of the vehicle occupant is directed toward the side mirror of the vehicle and / or the rearview mirror of the vehicle, and (b) determining that the steering wheel angle is greater than the predetermined angle threshold. The method can include determining that a movement of the steering angle with respect to time is greater than a predetermined rate threshold. The turn signal of the vehicle is automatically turned on in response to: (a) detecting that the line of sight of the vehicle occupant is directed toward the side mirror of the vehicle, and (b) determining that the movement of the steering angle with respect to time is greater than the predetermined rate threshold. The method can include monitoring a head pose of the vehicle occupant. The method can include determining that the head pose of the vehicle occupant indicates that the vehicle occupant is gazing at the side mirror. The turn signal of the vehicle is automatically turned on in response to: (a) detecting that the line of sight of the vehicle occupant is directed toward the side mirror of the vehicle, and (b) determining that the head pose of the vehicle occupant indicates that the vehicle occupant is gazing at the side mirror.

[0006] In another aspect of the disclosure, a method for automatically activating turn signals includes monitoring, with a camera, a vehicle occupant of a vehicle and detecting a shoulder movement of the vehicle occupant (i.e., a shoulder check). The movement can be referred to as a shoulder check. A shoulder check is a quick turn of the head to the left or right to see what is outside the side window. The shoulder check (sometimes called a head check) is an important way to check your blind spots. The method also includes determining that the shoulder movement indicates that the vehicle occupant is looking at an adjacent lane, and automatically activating a turn signal of the vehicle in response to determining that the shoulder movement indicates that the vehicle occupant is looking at an adjacent lane. The method described in this paragraph improves vehicle technology by automatically activating turn signals.

[0007] In an aspect of the disclosure, the method can include monitoring a movement of a steering wheel of the vehicle. The method can include monitoring a steering wheel angle of the steering wheel of the vehicle. The method can include determining that the steering wheel angle is greater than a predetermined angle threshold. The turn signal of the vehicle is automatically turned on in response to: (a) determining that the shoulder movement indicates that the vehicle occupant is looking at an adjacent lane, and (b) determining that the steering wheel angle is greater than the predetermined angle threshold. The method can include determining that a movement of the steering angle with respect to time is greater than a predetermined rate threshold. The turn signal of the vehicle is automatically turned on in response to: (a) determining that the shoulder movement indicates that the vehicle occupant is looking at an adjacent lane, and (b) determining that the movement of the steering angle with respect to time is greater than the predetermined rate threshold.

[0008] A system for automatically activating turn signals includes a camera positioned to monitor a vehicle occupant and a controller in communication with the camera. The controller is programmed to perform the method described above.

[0009] Further areas of applicability of the present disclosure will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description and specific examples, while indicating the preferred embodiment, are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure.

[0010] The above features and advantages and other features and advantages of the presently disclosed systems and methods are readily apparent from the detailed description, including claims and exemplary embodiments, when taken in connection with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0011] The present disclosure will become more fully understood from the detailed description and the accompanying drawings, wherein:

[0012] Figure 1 is a schematic side view of a vehicle including a system for automatically activating turn signals.

[0013] Figure 2 is a schematic top view of a vehicle traveling along a roadway.

[0014] Figure 3 is a schematic top view of a vehicle traveling along a roadway. Figure 1A flowchart of a method of controlling a turn signal of a vehicle. DETAILED DESCRIPTION

[0015] Reference will now be made in detail to several examples of the present disclosure shown in the accompanying drawings. Wherever possible, the same or like reference numbers will be used throughout the drawings and the written description to refer to the same or like parts or steps.

[0016] Reference Figure 1 and Figure 2 The vehicle 10 generally includes a vehicle body 12 and a plurality of wheels 14 coupled to the vehicle body 12. The vehicle 10 can be an autonomous vehicle. The vehicle 10 can be a sedan, a truck, a coupe, a sport utility vehicle (SUV), a recreational vehicle (RV). The vehicle 10 further includes a system 11 for automatically turning on one or more turn signals 16. The turn signals 16 can be lights and are coupled to the vehicle body 12. The vehicle 10 further includes a driver monitoring system (DMS) 17. The DMS 17 includes one or more cameras 18 positioned to monitor a vehicle occupant 19 (e.g., a vehicle driver). The DMS 17 includes infrared (IR) cameras, RGB cameras, and time-of-flight cameras. The IR cameras are primarily used for tracking the driver’s eyes and facial recognition. The IR cameras work well in low light conditions, making them effective for monitoring at night. The RGB cameras are standard color cameras used to detect the driver’s facial expressions and movements. The RGB cameras can recognize brightness and color, helping to assess the driver’s emotional state. The time-of-flight cameras measure distance to create 3D images, allowing for more accurate detection of the driver’s position and movements. The time-of-flight cameras are beneficial for tracking motion and analyzing posture. The IR cameras, RGB cameras, and time-of-flight cameras are used to monitor the driver’s attention, fatigue, and emotional state in real-time, supporting safer driving. The cameras 18 can be directly coupled to the vehicle body 12. The vehicle 10 further includes side mirrors 24 Figure 2 ).

[0017] Further, the system 11 includes a controller 34 in communication with the camera 18. Although the controller 34 is shown as being coupled to the vehicle 10, it is contemplated that the controller 34 can be external to the vehicle 10. The controller 34 is programmed to receive raw images from the camera 18 and includes at least one processor 44 and a non-transitory computer readable storage device or medium 46. The processor 44 can be a custom processor, a central processing unit (CPU), a graphics processing unit (GPU), a secondary processor in a plurality of processors associated with the controller 34, a semiconductor-based microprocessor (in the form of a microchip or chip set), a macroprocessor, a combination thereof, or generally a device for executing instructions. The computer readable storage device or medium 46 can include volatile and non-volatile storage devices such as read-only memory (ROM), random access memory (RAM), and a keep alive memory (KAM). The KAM is a persistent or non-volatile memory that can be used to store various operating variables when the processor 44 is powered off. The computer readable storage device or medium of the controller 34 can be implemented using a plurality of memory devices such as programmable read-only memory (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or other electric, magnetic, optical, or combination memory devices capable of storing data, some of which represent executable instructions used by the controller 34 in controlling the turn signal 16. The controller 34 is in communication with the DMS 17, the camera 18, and the turn signal 16.

[0018] The instructions, which can comprise one or more separate programs, each of which comprises an ordered listing of executable instructions for implementing logical functions. When the instructions are executed by the processor 44, the processor receives and processes signals from the camera 18, performs logic, calculations, methods, and / or algorithms for automatically controlling components of the turn signal 16, and generates control signals based on the logic, calculations, methods, and / or algorithms. Although Figure 1 A single controller 34 is shown in FIG. 1, but the system 11 can include multiple controllers 34 that are in communication through an appropriate communication medium or combination of communication mediums and cooperate to process sensor signals, perform logic, calculations, methods, and / or algorithms, and generate control signals that automatically control features of the system 11. The non-transitory computer readable storage device or medium 46 includes machine readable instructions (shown in FIG. 1) that, when executed by one or more processors, cause the processor 44 to perform the method 100 (shown in FIG. 1). Figure 3 Figure 3

[0019] ​​The vehicle 10 also includes a steering wheel 20 and a steering angle sensor (SAS) 22 coupled to the steering wheel 20. The steering wheel 20 can be rotated to control the steering of the vehicle 10. The SAS 22 is configured to directly or indirectly measure the steering wheel angle and / or steering wheel angular velocity in real-time, among other signals. The wheel angle can be measured by the engine control unit and transmitted to the controller 34. The controller 34 is in communication with the SAS 22.

[0020] Figure 3 is a flowchart of a method for automatically activating the turn signal 16 of the vehicle 10. The method 100 begins at blocks 102, 104, 106, and 108, where the controller 34 receives a plurality of inputs. At block 102, the controller 34 receives a transmission state (e.g., park, drive, reverse, neutral, drive, or low). At block 104, the controller 34 receives inputs from the relevant vehicle interface. At block 106, the controller 34 receives image data from the camera 18 of the DMS 17. At block 108, the controller 34 receives sensor data from the SAS 22. The sensor data generated by the SAS 22 indicates the movement of the steering angle with respect to time and / or the steering angle.

[0021] The method 100 then proceeds to block 110. At block 110, the controller 34 determines whether the transmission state is a drive state or a low state. If the transmission state is not a drive state or a low state, the method 100 returns to block 102. If the transmission state is a drive state or a low state, the method 100 proceeds to block 112.

[0022] At block 112, the system 11 for automatically turning on one or more turn signals 16 is activated. The system 11 can be referred to as an intelligent turn signal system. The method 100 then proceeds to block 114. At block 114, the controller 34 performs system interface diagnostics of the relevant vehicle interface inputs. The method 100 then proceeds to block 116. At block 116, the controller 34 determines whether the system 11 passes the system interface diagnostics. If not, the method 100 proceeds to block 116. At block 116, the controller 34 disables the system 11. If the system 11 passes all of the system interface diagnostics, the method 100 proceeds to block 118.

[0023] At block 118, the controller 34 uses the image data from the camera 18 to determine the line of sight and / or head pose of the vehicle occupant 19. In addition, the controller 34 can use the image data to determine whether the line of sight, head pose, and / or shoulder motion of the vehicle occupant 19 is toward the right side mirror 24 of the vehicle 10. To do so, the controller 34 determines whether the head rotation, line of sight, and / or shoulder motion of the vehicle occupant 19 is greater than a right side threshold. If the head pose and / or line of sight is not greater than the right side threshold, the method 100 proceeds to block 132. If the head pose and / or line of sight is greater than the right side threshold, the method 100 proceeds to block 120. As one non-limiting example, the right side threshold can be thirty degrees. However, it is contemplated that the right side threshold can be adjusted based on other system variables (e.g., vehicle speed) or based on the output of a neural network or another type of machine learning technique.

[0024] At block 120, the controller 34 records the time elapsed until the condition of block 118 is no longer true. The method 100 then continues to block 122.

[0025] At block 122, the controller 34 determines whether the elapsed time (determined in block 120) is greater than a time threshold. If the elapsed time is not greater than the time threshold, the method 100 proceeds to block 102. If the elapsed time is greater than the time threshold, the method 100 continues to block 124.

[0026] At block 124, the controller 34 determines that an occurrence has happened (i.e., that the vehicle occupant 19 is attempting to change lanes). In one embodiment, more than one consecutive occurrence is required to proceed. The occurrence can be calibrated or can be based on habits learned by the vehicle occupant using, for example, a neural network. For example, the controller 34 can track the line of sight toward the interior rearview mirror instead of (or in addition to) the side mirror 24. The neural network can also determine the number of times the occupant looks at the side mirror 24 and / or the rearview mirror. In addition, the neural network can determine whether the occupant performed a shoulder check prior to changing lanes. The method 100 then continues to block 126.

[0027] At block 126, the controller 34 determines whether the steering wheel angle is greater than a predetermined angle threshold and / or the movement of the steering angle with respect to time (i.e., steering wheel speed) is greater than a predetermined rate threshold. If the steering wheel angle is not greater than the predetermined angle threshold and / or if the movement of the steering angle with respect to time (i.e., steering wheel speed) is not greater than the predetermined rate threshold, the method 100 returns to block 102. If the steering wheel angle is greater than the predetermined angle threshold and / or if the movement of the steering angle with respect to time (i.e., steering wheel speed) is greater than the predetermined rate threshold, the method 100 proceeds to block 128. At block 128, the right turn signal 16 is automatically turned on. When the turn signal 16 is activated, the system 11 will provide a unique serial data signal for the vehicle 10 to distinguish between an automatic activation and a driver initiated turn signal activation. The unique serial data signal is consumed by other sub-functions, such as Lane Departure Warning and / or Lane Deep Assist, which allows the system 11 to seamlessly integrate with other active safety features. Then, the method 100 continues to block 130. At block 130, the controller 34 determines whether the steering wheel angle is greater than a predetermined angle threshold and / or the movement of the steering angle with respect to time (i.e., steering wheel speed) is greater than a predetermined rate threshold and the transmission state is either a drive state or a low state. If the steering wheel angle is not greater than the predetermined angle threshold and / or if the movement of the steering angle with respect to time (i.e., steering wheel speed) is not greater than the predetermined rate threshold and the transmission state is not either a drive state or a low state (Drive or Low), the method 100 returns to block 102. If the steering wheel angle is greater than the predetermined angle threshold and / or if the movement of the steering angle with respect to time (i.e., steering wheel speed) is greater than the predetermined rate threshold and the transmission state is either a drive state or a low state, the method 100 returns to block 128.

[0028] The method 100 also includes a block 132 for left face rotation and / or gaze detection. At block 132, the controller 34 uses image data from the camera 18 to determine the gaze and / or head pose of the vehicle occupant 19. The controller 34 can also identify the number of times (count) the vehicle occupant is looking at the left side mirror 16, the right side mirror 16, and / or the rearview mirror. In addition, the controller 34 can use the image data to determine whether the gaze, head pose, and / or shoulder movement of the vehicle occupant 19 is toward the left side mirror 24 of the vehicle 10. To do so, the controller 34 determines whether the head rotation, gaze, and / or shoulder movement of the vehicle occupant 19 is greater than a left side threshold. If the head pose and / or gaze is not greater than the left side threshold, the method 100 proceeds to block 102. If the head pose and / or gaze is greater than the left side threshold, the method 100 proceeds to block 134. As one non-limiting example, the left side threshold can be thirty degrees. However, it is contemplated that the left side threshold can be adjusted based on other system variables (e.g., vehicle speed) or based on the output of a neural network or another type of machine learning technique.

[0029] At block 134, the controller 34 records the time elapsed until the condition of block 132 is no longer true. The method 100 then continues to block 136.

[0030] At block 136, the controller 34 determines whether the elapsed time (determined in block 134) is greater than a time threshold. If the elapsed time is not greater than the time threshold, the method 100 proceeds to block 102. If the elapsed time is greater than the time threshold, the method 100 continues to block 138.

[0031] At block 138, the controller 34 determines that an occurrence has happened (i.e., the vehicle occupant 19 is attempting to change lanes). In one embodiment, more than one consecutive occurrence is required to proceed. The occurrence can be calibrated or can be based on habits learned by the vehicle occupant using, for example, a neural network. For example, the controller 34 can track the gaze toward the interior rearview mirror instead of (or in addition to) the side mirror 24. The method 100 then continues to block 140.

[0032] At block 140, the controller 34 determines whether the steering wheel angle is greater than a predetermined angle threshold and / or the movement of the steering angle with respect to time (i.e., steering wheel speed) is greater than a predetermined rate threshold. If the steering wheel angle is not greater than the predetermined angle threshold and / or if the movement of the steering angle with respect to time (i.e., steering wheel speed) is not greater than the predetermined rate threshold, the method 100 returns to block 102. If the steering wheel angle is greater than the predetermined angle threshold and / or if the movement of the steering angle with respect to time (i.e., steering wheel speed) is greater than the predetermined rate threshold, the method 100 proceeds to block 142. At block 142, the left turn signal 16 is automatically turned on. When the turn signal 16 is activated, the system 11 will provide a unique serial data signal for the vehicle 10 to distinguish between an automatic activation and a driver initiated turn signal activation. The unique serial data signal is consumed by other sub-functions, such as lane departure warnings and / or lane depth assist, which allows the system 11 to seamlessly integrate with other active safety features. Then, the method 100 continues to block 144. At block 144, the controller 34 determines whether the steering wheel angle is greater than a predetermined angle threshold and / or the movement of the steering angle with respect to time (i.e., steering wheel speed) is greater than a predetermined rate threshold and the transmission state is either a drive state or a low speed state. If the steering wheel angle is not greater than the predetermined angle threshold and / or if the movement of the steering angle with respect to time (i.e., steering wheel speed) is not greater than the predetermined rate threshold and the transmission state is not either a drive state or a low speed state, the method 100 returns to block 102. If the steering wheel angle is greater than the predetermined angle threshold and / or if the movement of the steering angle with respect to time (i.e., steering wheel speed) is greater than the predetermined rate threshold and the transmission state is either a drive state or a low speed state, the method 100 returns to block 142.

[0033] While the above describes example embodiments, these embodiments are not intended to describe all possible forms of the claims. The words used in the specification are words of description rather than limitation, and it is understood that various changes can be made without departing from the spirit and scope of the disclosure. As previously described, features of various embodiments can be combined to form further embodiments of the presently disclosed systems and methods not explicitly described or illustrated. While various embodiments can have been described as providing advantages or being superior to other embodiments or prior art implementations, it will be appreciated that one or more features or characteristics of one or more embodiments can be incorporated into other embodiments or features to form further embodiments having similar or different advantages or that some features and characteristics of the embodiments can be omitted, replaced, or supplemented. As such, the embodiments described are not intended to be limited to the described embodiments, but are to be construed as including all alterations, modifications and combinations falling within the scope of the claims.

[0034] The drawings are in simplified form and are not drawn to precise scale. Directional terminology, such as top, bottom, left, right, up, down, over, above, below, under, posterior, anterior, posterior, anterior, etc., is used for convenience and clarity only. Such terminology is not intended to limit the scope of the disclosure to only a particular orientation.

[0035] Embodiments of the present disclosure are described herein. It is to be understood, however, that the disclosed embodiments are merely examples and other embodiments can take various alternative forms. The drawings are not necessarily to scale; some features can be exaggerated or minimized for clarity. Specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to employ the current disclosure in a variety of ways. As will be understood by those skilled in the art, various features shown and described with reference to any one figure can be combined with features shown and described in one or more other figures to produce embodiments that are not explicitly shown or described. The combination of features provides representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of the present disclosure can be desired for particular applications or implementations.

[0036] This description is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses. The broad teachings of the present disclosure can be implemented in a variety of forms. Therefore, while this disclosure includes particular examples, the true scope of the disclosure should not be limited to such examples.

Claims

1. A method for automatically activating turn signals, comprising: monitoring a vehicle occupant of a vehicle with a driver monitoring system, wherein the driver monitoring system includes at least one camera and the driver monitoring system is configured to determine a position of at least one eye of the vehicle occupant, a head orientation of the vehicle occupant, and a shoulder movement; detecting whether a line of sight of the vehicle occupant is directed toward a side mirror of the vehicle; and in response to determining that the line of sight of the vehicle occupant is directed toward the side mirror of the vehicle, automatically activating the turn signals of the vehicle.

2. The method of claim 1, further comprising monitoring a movement of a steering wheel of the vehicle.

3. The method of claim 2, further comprising monitoring a steering wheel angle of the steering wheel of the vehicle.

4. The method of claim 3, further comprising determining that the steering wheel angle is greater than a predetermined angle threshold. in response to (a) detecting that the line of sight of the vehicle occupant is directed toward the side mirror of the vehicle and (b) determining that the steering wheel angle is greater than the predetermined angle threshold, automatically turning on the turn signals of the vehicle.

5. The method of claim 4, wherein, 6. The method of claim 2, further comprising determining that a movement of a steering angle with respect to time is greater than a predetermined rate threshold. in response to (a) detecting that the line of sight of the vehicle occupant is directed toward the side mirror of the vehicle and (b) determining that the movement of the steering angle with respect to time is greater than the predetermined rate threshold, automatically turning on the turn signals of the vehicle.

7. The method of claim 6, wherein, 8. The method of claim 1, further comprising monitoring a head pose of the vehicle occupant.

9. The method of claim 8, further comprising determining that the head pose of the vehicle occupant indicates that the vehicle occupant is gazing at the side mirror. in response to (a) detecting that the line of sight of the vehicle occupant is directed toward the side mirror of the vehicle and (b) determining that the head pose of the vehicle occupant indicates that the vehicle occupant is gazing at the side mirror, automatically turning on the turn signals of the vehicle.

10. The method of claim 9, wherein, ​

Images