Methods, systems, and devices for e-mirror traffic lane identification

Abstract

To provide methods, systems, and devices for e-mirror traffic lane identification.SOLUTION: Methods, systems, and devices for assisting a driver of a vehicle are provided. A traffic lane identification system may include one or more displays located on or within the vehicle. The traffic lane identification system may further include an electronic control unit (ECU) coupled to the one or more displays. The ECU may transmit real-time video data to a display so as to display the field of view of a surrounding area of the vehicle. The ECU may further generate a lane identifier indicating a current traffic lane of the vehicle. The ECU may further transmit the lane identifier to the display so as to display the lane identifier overlaid within the field of view of the surrounding area of the vehicle.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to methods, systems, and / or apparatus for electronic mirror lane identification. [Background technology]

[0002] Modern vehicles are often equipped with numerous screens and cameras to assist vehicle users (e.g., the driver and / or passengers) in various vehicle maneuvers, such as lane changes, parking, and / or reversing. The screens may allow the driver to view live video captured by cameras around the vehicle. However, the driver may become confused or disoriented, especially when attempting to determine at a glance which location in the image is the vehicle's own lane or an adjacent lane. Poor weather conditions (e.g., rain, snow, sleet) may also contribute to the difficulty of judging or seeing lanes or lane markings. The risk of a driver being unable to correctly judge or see the lane arises particularly when the vehicle is new to the driver, when the driver is tired, in heavy traffic, when the driver / vehicle is changing lanes, when the screen image does not include parts of the vehicle's body, and / or when the camera is not precisely centered on the vehicle.

[0003] Therefore, it would be desirable to provide a method, system, and apparatus for electronic mirror lane identification. Summary of the Invention

[0004] Generally, aspects of the subject matter described in this disclosure may be realized by a lane identification system for a vehicle. The lane identification system may include one or more displays located on or within the vehicle. The lane identification system may further include an electronic control unit (ECU) coupled to the one or more displays. The ECU may be configured to transmit real-time video data to the displays for displaying a peripheral view of the vehicle via the displays. The ECU may be further configured to generate a lane identifier indicating a current lane of the vehicle. The ECU may be further configured to transmit the lane identifier to the displays for displaying the lane identifier overlaid on the view via the displays.

[0005] In one aspect, the object may be realized by a lane identification system for assisting a vehicle driver. The lane identification system may include a camera configured to capture real-time video data of the vehicle's peripheral field of view. The lane identification system may further include a display located on or within the vehicle. The lane identification system may further include an electronic control unit (ECU) coupled to the display and the camera. The ECU may be configured to transmit the real-time video data to the display for displaying the vehicle's peripheral field of view via the display. The ECU may be further configured to generate a lane identifier indicating the vehicle's current lane. The ECU may be further configured to transmit the lane identifier to the display for displaying the lane identifier overlaid on the field of view via the display.

[0006] In one aspect, the object may be realized by a method for assisting a vehicle driver. The method may include transmitting real-time video data to a display via an electronic control unit (ECU) of the vehicle to display a peripheral view of the vehicle via the display. The method may further include generating, via the ECU, a lane identifier indicating a current lane of the vehicle. The method may further include transmitting, via the ECU, the lane identifier to the display to display the lane identifier overlaid on the view via the display.

[0007] These and other embodiments may optionally include one or more of the following features.

[0008] In various aspects, the lane identification system may further include a camera coupled to the ECU and configured to capture real-time video data of the vehicle's peripheral field of view.

[0009] In various embodiments, the peripheral visibility of the vehicle includes the rear visibility relative to the vehicle.

[0010] In various embodiments, the display includes a digital rearview mirror configured to display a rear view and / or lane identifiers.

[0011] In various embodiments, the lane identifier comprises a marker that highlights the lane markings of the vehicle's current lane.

[0012] In various embodiments, the lane identifier includes a first zone and a second zone that include the vehicle's current lane.

[0013] In various embodiments, at least one of the color or brightness of the second zone is adjusted relative to the first zone.

[0014] In various aspects, the lane identifier may include a plurality of colors, each color of the plurality of colors being assigned to a respective lane of a plurality of lanes identified in the field of view.

[0015] In various embodiments, the ECU is further configured to determine that the vehicle is performing a lane change maneuver. In various embodiments, the ECU is further configured to remove the lane identifier from the display during the lane change maneuver. In various embodiments, the ECU is further configured to determine that the vehicle has completed the lane change maneuver. In various embodiments, the ECU is further configured to generate an updated lane identifier indicating the vehicle's new current lane. In various embodiments, the ECU is further configured to transmit the updated lane identifier to the display for displaying the updated lane identifier overlaid on the real-time view via the display. [Brief explanation of the drawings]

[0016] Other systems, methods, features, and advantages of the present disclosure will become apparent to those skilled in the art upon examination of the following drawings and detailed description. Components shown in the drawings are not necessarily to scale and may be exaggerated to better illustrate important features of the present disclosure. In the drawings, like reference numerals refer to like parts throughout the different views. [Figure 1] FIG. 1 is a block diagram of an example lane identification system for a vehicle, according to certain aspects of the present disclosure. [Figure 2] FIG. 2 is a schematic top view of an example vehicle including the example lane identification system of FIG. 1, according to certain aspects of the present disclosure. [Figure 3] FIG. 3 is a diagram of an example cabin of a vehicle including the example lane identification system of FIG. 1 according to certain aspects of the present disclosure. [Figure 4A-B] 4A and 4B illustrate a driver using an electronic mirror according to various embodiments of the present disclosure. [Figure 5A] FIG. 5A illustrates an electronic mirror displaying the rear view of a vehicle according to various embodiments of the present disclosure. [Figure 5B-E] 5B-E illustrate the electronic mirror of FIG. 5A displaying the rear view of the vehicle with various lane identifiers superimposed thereon, according to various embodiments of the present disclosure. [Figure 6] FIG. 6 is a flow diagram of an example process for controlling the lane identification system of FIG. 1 according to certain aspects of the present disclosure. [Figure 7] FIG. 7 is a flow diagram of an example process for controlling the lane identification system of FIG. 1 according to certain aspects of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0017] Disclosed herein are methods, systems, and / or apparatuses that implement a lane identification system. The lane identification system may include one or more displays located within the cabin of a vehicle. Particular embodiments of the subject matter described in this disclosure may be implemented to achieve one or more of the following advantages: The lane identification system may generate a lane identifier that indicates a current lane of a vehicle user (e.g., a driver and / or passenger). The lane identification system may enable a user to quickly and clearly identify and understand the vehicle's own lane as distinct from adjacent lanes. Furthermore, the lane identifier may quickly direct the user's attention to the vehicle's current lane. The lane identifier may be a graphical icon or marker displayed by at least one display (e.g., a rearview mirror). By indicating the vehicle's current lane, the lane identifier may eliminate or at least reduce the time a user spends looking at a display to determine the vehicle's travel lane, thereby improving safety and reducing user confusion or error.

[0018] Additionally, the lane identification system may further improve safety by assigning lane colors such that each lane maintains its assigned color when the lane identifiers are automatically and dynamically updated when a vehicle changes lanes. For example, when a vehicle performs a lane change maneuver to move from lane one to lane two, the lane identification system can update the lane identifier based on the new view from lane two. This updated lane identifier can maintain the previous lane color displayed from when the vehicle was in lane one, which may reduce user confusion.

[0019] As used herein, "electronic mirror" may refer to the use of a camera or image sensor to detect image data behind a vehicle and display the image data on a display screen within the vehicle. The display screen may be integrated into an existing mirror (e.g., a rearview or side mirror) or may be a separate mirror or screen. In this manner, the camera and display screen comprise an electronic mirror that acts and functions similarly to a conventional mirror.

[0020] FIG. 1 is a block diagram of an example lane identification system 100. The lane identification system 100, or portions thereof, may be retrofitted to, connected to, included in, or separate from a vehicle 102. The vehicle 102 may be a transportation capable of transporting people, goods, or permanently or temporarily stationary equipment. The vehicle 102 may be a self-propelled wheeled transportation such as a passenger car, sport utility vehicle, truck, bus, van, or other vehicle powered by a motor, battery, or fuel cell. For example, the vehicle 102 may be an electric vehicle, a hybrid vehicle, a hydrogen fuel cell vehicle, a plug-in hybrid vehicle, or any other type of vehicle having a fuel cell stack, a motor, an engine, and / or a generator. Other examples of vehicles may include a bicycle, a train, an airplane, a boat, or any other form of transportable transportation mechanism. The vehicle 102 may be semi-autonomous or autonomous. That is, the vehicle 102 may be self-operating and may navigate without human input. An autonomous vehicle may have and use one or more sensors and / or navigation units to operate autonomously.

[0021] The lane identification system and / or vehicle 102 may include a motor and / or generator 132, a battery 134, and / or a transmission 136. The motor and / or generator 132 may be located within an engine bay of the vehicle 102. The motor and / or generator 132 may be an internal combustion engine (ICE). In this regard, the motor and / or generator 132 may combust a mixture of air and fuel to provide power to the vehicle 102 and / or components of the vehicle 102 and / or the lane identification system 100. Thus, the motor and / or generator 132 may accelerate, decelerate, or maintain a desired speed of the vehicle 102. The motor and / or generator 132 may include a combination of an ICE and an electric motor, for example, in a hybrid vehicle application. In some embodiments, the motor and / or generator 132 may be an electric motor. In this regard, the motor and / or generator 132 may be an electric motor and electric generator that converts electrical energy into mechanical power, such as torque, and converts mechanical power into electrical energy. The motor and / or generator 132 may be electrically connected to the battery 134. The motor and / or generator 132 may convert energy from the battery 134 into mechanical power and may provide energy back to the battery 134, for example, via regenerative braking. The battery 134 may be electrically connected to the motor and / or generator 132 and may provide electrical energy to and receive electrical energy from the motor and / or generator 132. The battery 134 may provide electrical energy to a lane identification system.

[0022] Transmission 136 may be an automatic transmission (e.g., a 6-, 7-, 8-, 9-, or 10-speed automatic transmission) or a manual transmission (e.g., a 6-speed manual transmission) having at least one reverse gear, a neutral position, and at least one forward (or drive) gear. In some embodiments, transmission 136 may be a motor vehicle transmission (e.g., a 1- or 2-speed transmission). Transmission 136 may receive torque from motor and / or generator 132 and transfer the torque to one or more wheels of vehicle 102.

[0023] The lane identification system 100 may further include a network access device 114. The network access device 114 may be electrically connected to the ECU 106 and may include a channel or communication port, such as one or more of a Wi-Fi unit, a Bluetooth unit, a radio frequency identification (RFID) tag or reader, a DSRC unit, and / or a cellular network unit for accessing a network 140 (e.g., CDMA, GSM, 3G, 4G, 5G, etc.). The network access device 114 may transmit data to and receive data from systems and devices not directly connected to the vehicle 102. For example, the ECU 106 may communicate with an external display 120f (e.g., a mobile device, phone, tablet, laptop, etc.), one or more cameras 116, one or more sensors 118, one or more displays 120, and / or the memory 108 via the network access device 114. In some embodiments, the rear-facing camera 116a may communicate wirelessly with the ECU 106 through the network access device 114. This reduces or eliminates the need for wires running from the rear-facing camera 116a to the vehicle 102.

[0024] The lane identification system 100 may further include a memory 108. The memory 108 may be electrically connected to the ECU 106. In some embodiments, the memory 108 may be communicatively connected to the ECU 106 (e.g., via the network 140) so as to be remote from the ECU 106 and / or the vehicle 102. In other embodiments, the memory 108 may be electrically connected to the ECU 106, and a remote memory (e.g., a remote database) may be communicatively connected to the ECU 106, where the remote memory has similar, additional, and / or different functionality than the memory 108 (e.g., greater storage capacity, enabling over-the-air (OTA) updates, etc.). The memory 108 may store instructions for execution on the ECU 106 and may include one or more of random access memory (RAM) or other volatile or non-volatile memory. The memory 108 may be a non-transitory memory or data storage device, such as a hard disk drive, a solid-state disk drive, a hybrid disk drive, or other suitable data storage, and may further store machine-readable instructions that may be loaded and executed by the ECU 106. The memory 108 may store vehicle parameters (e.g., vehicle weight, vehicle length, vehicle width, vehicle height, transmission gear information, etc.). The memory 108 may further store one or more lane colors described herein. The one or more lane colors may be predetermined, set by the manufacturer of the vehicle 102, and / or set by the user of the vehicle 102 (e.g., via the user interface 120a).

[0025] FIG. 2 illustrates an example vehicle 102 including an embodiment of the lane identification system 100 of FIG. 1. Referring collectively to FIGS. 1 and 2, the lane identification system 100 further includes one or more cameras 116. The one or more cameras 116 may be coupled to the exterior 201 of the vehicle 102. In some embodiments, the one or more cameras 116 may be coupled to the interior of the vehicle 102 but face outward to capture the exterior of the vehicle 102. The one or more cameras 116 may include a rear-facing camera 116a configured to provide a rearward view behind the vehicle 102. While the rear-facing camera 116a is shown located at the center rear of the vehicle 102, it may be located at any other suitable location on the vehicle 102, such as the left side of the vehicle 102, the right side of the vehicle 102, the left mirror 120d of the vehicle 102, the right mirror 120e of the vehicle 102, or any other suitable location depending on the desired design of the vehicle 102. In various embodiments, rear-facing camera 116a may be located at a position offset from the center of vehicle 102. In some embodiments, one or more cameras 116 may include a side-facing camera, a forward-facing camera, or any other suitable camera, depending on the desired direction of view to provide to the user of vehicle 102.

[0026] The one or more cameras 116 may be digital cameras, infrared thermal cameras, and / or night vision cameras (e.g., using active illumination and / or image intensification). In some embodiments, the one or more cameras 116 may be and / or include a panoramic view monitor (PVM) system for the vehicle 102. The one or more cameras 116 may provide, capture, and / or record real-time video (or video data) including images and / or video of the surroundings of the vehicle 102. The rear-facing camera 116a may provide, capture, and / or record real-time video (or video data) including images and / or video of the area behind the vehicle 102. Each camera of the one or more cameras 116 may capture one or more fields of view that may include at least a portion of the surroundings of the vehicle 102. In some embodiments, each camera of the one or more cameras 116 may allow a user to change the zoom level (e.g., via user interface 120a), and / or the lane identification system may automatically change the zoom level of each camera based on the speed of the vehicle 102 (e.g., decreasing the zoom level as the speed of the vehicle 102 increases). In some embodiments, the one or more cameras 116 may include a trailer camera that can provide a rear view to a user of a vehicle behind the trailer.

[0027] The rear-facing camera 116a may be coupled to the exterior 201 and / or rear of the vehicle 102 (e.g., bumper, rear tailgate, etc.). In some embodiments, the lane identification system 100 may include multiple rear-facing cameras, each having a different field of view and / or direction. The rear-facing camera 116a may provide, capture, and / or record real-time video of the rear field of view 208 of the vehicle 102 (e.g., capturing a portion of the surroundings behind the vehicle 102). In some embodiments, the rear-facing camera 116a may be a wireless camera such that the real-time video of the rear field of view 208 may be transmitted wirelessly to the ECU 106. In some embodiments, the rear-facing camera 116a may transmit the real-time video of the rear field of view 208 to the ECU 106 via a wired connection.

[0028] The lane identification system 100 may further include one or more sensors (or one or more vehicle sensors) 118. The one or more sensors 118 may be integrated with the vehicle 102. The one or more sensors 118 may periodically and / or continuously measure, detect (or indicate), and / or determine one or more conditions (or one or more vehicle states) of the vehicle 102. The one or more conditions of the vehicle 102 may include gear position, vehicle speed, turn signal status, steering angle, camera status, and / or proximity to one or more objects. The one or more sensors 118 may be and / or include software, hardware, firmware, or a combination thereof for measuring, detecting, and / or determining one or more conditions of the vehicle 102. The one or more sensors 118 may include a transmission sensor (or shift sensor) 118a, a speed sensor 118b, a turn signal sensor 118c, a steering angle sensor 118d, and / or one or more distance sensors 118e. In some embodiments, the lane identification system may include two or more, three or more, four or more, or all of a transmission sensor 118a, a speed sensor 118b, a turn signal sensor 118c, a steering angle sensor 118d, and / or one or more distance sensors 118e.

[0029] The transmission sensor 118a may be located within the vehicle 102 and / or may be connected to the transmission 136 of the vehicle 102. The transmission sensor 118a may indicate, detect, and / or determine a shift position (or gear position or mode) of the vehicle 102 and / or the transmission 136. The shift position may include drive, reverse, and neutral. In some embodiments, the lane identification system 100 may include multiple transmission sensors 118a.

[0030] The speed sensor 118b may be located within the vehicle 102. The speed sensor 118b may measure, detect, and / or determine the vehicle speed of the vehicle 102. In some embodiments, the speed sensor 118b may measure, detect, and / or determine the vehicle speed based on the rotational speed of the output shaft of the transmission 136, the rotational speed of one or more wheels of the vehicle 102, and / or a GPS signal. In some embodiments, the lane identification system 100 may include multiple speed sensors 118b.

[0031] FIG. 3 illustrates a cabin 302 of a vehicle 102 including the lane identification system of FIG. 1. Referring collectively to FIGS. 1 and 3, a turn signal sensor 118c may be located within the vehicle 102 and / or may be coupled to an indicator stalk 308 and / or steering wheel 306 of the vehicle 102. The turn signal sensor 118c may indicate, detect, and / or determine a turn signal status of the vehicle 102. The turn signal status may include whether the left turn signal of the vehicle 102 is activated and / or whether the right turn signal of the vehicle 102 is activated. For example, the turn signal sensor 118c may detect when the driver of the vehicle 102 activates the left or right turn signal of the vehicle 102. In some embodiments, the turn signal sensor 118c may detect when the vehicle 102 activates a left or right turn signal, for example, during autonomous or semi-autonomous driving. In some embodiments, the lane identification system 100 may include multiple turn signal sensors 118c.

[0032] The steering angle sensor 118d may be located within the vehicle 102 and / or may be connected to the steering wheel 306 of the vehicle 102. The steering angle sensor 118d may measure, detect, and / or determine the steering angle of the vehicle 102 (e.g., the degree to which the steering wheel is turned left or right from a center position (or reference position)). In some embodiments, the lane identification system may include multiple steering angle sensors. The steering angle sensor 118d may indicate whether the vehicle 102 is turning (e.g., when the vehicle 102 turns left or right by more than 20 degrees) or changing direction slightly (e.g., when the vehicle 102 turns left or right between 0 and 20 degrees).

[0033] One or more distance sensors 118e may be coupled to the exterior 201 of the vehicle 102 (see FIG. 2). The one or more distance sensors 118e may be and / or include a camera, a sonar sensor, a radar sensor, and / or a lidar sensor. The one or more distance sensors 118e may measure, detect, and / or determine proximity to one or more objects in the vicinity of the vehicle 102. The one or more distance sensors 118e may measure, detect, and / or determine proximity of one or more objects by collecting spatial information of the one or more objects and constructing a point cloud. In some embodiments, the lane identification system 100 may include multiple distance sensors 118e.

[0034] 1, 2, and 3 collectively, the lane identification system 100 may further include one or more displays (or one or more displays and / or mirrors) 120. The one or more displays 120 may include a user interface (or information display) 120a, a gauge cluster 120b, a rearview mirror 120c, a left mirror 120d, and / or a right mirror 120e. The one or more displays 120 may be built into the vehicle 102 and / or may be retrofitted to the vehicle 102. The one or more displays 120 may include a liquid crystal display (LCD), a light emitting diode (LED) display, a segmented display, a holographic display, an electronic paper display, a laser color video display, and / or other types of displays. In some embodiments, the one or more displays 120 may include an external display 120f (e.g., a cell phone, laptop, tablet, etc.) that may have some or all of the functionality (e.g., receiving and displaying images) of the user interface 120a, the gauge cluster 120b, the rearview mirror 120c, the left mirror 120d, and / or the right mirror 120e. At least one display of the one or more displays 120 displays lane identifiers that identify the lanes surrounding the vehicle 102. In some embodiments, the rearview mirror 120c displays the lane identifiers.

[0035] The user interface 120a may be located within the cabin 302 of the vehicle 102 and / or may be coupled to the dashboard 310 (as shown by FIG. 3 ) of the vehicle 102. The user interface 120a may provide an interface to a user of the vehicle 102 to interact with and / or receive output from the electronic control unit (ECU) 106. The user interface 120a may include user interface elements, such as one or more screens and / or one or more touchscreens, including buttons, switches, microphones, speakers, gesture monitoring sensors, knobs, a graphical user interface (GUI) and / or other input / output devices electrically connected to the ECU 106, to provide for the input of information (or data) to and / or the output of information (or data) from the ECU 106. User interface 120a may display lane identifiers, rear view 208, and / or any other suitable view (e.g., left rear view, right rear view, second rear view, forward view, etc.).

[0036] Gauge cluster 120b may be located within the cabin 302 of vehicle 102 and / or may be coupled to the dashboard 310 of vehicle 102 (as shown in FIG. 3 ). Gauge cluster 120b may include one or more screens and / or physical indicators for displaying vehicle information such as a speedometer, a tachometer, energy and / or fuel gauges, a map, etc. In some embodiments, gauge cluster 120b may display a lane identifier, a rear view 208, and / or any other suitable view (e.g., left rear view, right rear view, second rear view, forward view, etc.).

[0037] The rearview mirror 120c may be located within the cabin 302 of the vehicle 102 and / or may be coupled to the ceiling 312 of the vehicle 102 (as shown in FIG. 3 ). The rearview mirror 120c may be an electronic mirror such that the surface 314 of the rearview mirror 120c may be and / or include one or more screens that may display lane identifiers, the rear view 208, and / or any other suitable view (e.g., a second rear view, etc.). In some embodiments, the rearview mirror 120c may comprise a conventional (or non-electronic) rearview mirror such that the surface 314 of the rearview mirror 120c may be or include a reflective surface that allows a user to see a reflection of at least a portion of the rear view. In some embodiments, the rearview mirror 120c may have an electronic mode (e.g., using one or more screens on the surface 314) and a conventional mode (using a reflective surface on the surface 314).

[0038] The left mirror (or left field mirror) 120d may be located within the cabin 302 of the vehicle 102 or may be coupled to the exterior 201 of the vehicle 102 (as shown in FIG. 2). The left mirror 120d may include an electronic (or digital) left field mirror having one or more screens that may display lane identifiers and the left rear view. The left mirror 120d may also include a conventional (or non-electronic) rearview mirror, such that the surface of the left mirror 120d may be or include a reflective surface that allows a user to see a reflection of at least a portion of the left rear view. In some embodiments, the left mirror 120d may have an electronic mode (e.g., using one or more screens to display the left rear view) and a conventional mode (using a reflective surface to show a reflection of at least a portion of the left rear view).

[0039] The right mirror (or right field of view mirror) 120e may be located within the cabin 302 of the vehicle 102 and / or may be coupled to the exterior 201 of the vehicle 102 (as shown in FIG. 2). The right mirror 120e may include an electronic (or digital) right field of view mirror having one or more screens that may display lane identifiers and a right rear view. The right mirror 120e may also include a conventional (or non-electronic) rearview mirror, such that the surface of the right mirror 120e may be or include a reflective surface that allows a user to see a reflection of at least a portion of the right rear view. In some embodiments, the right mirror 120e may have an electronic mode (e.g., using one or more screens to display the right rear view) and a conventional mode (using a reflective surface to show a reflection of at least a portion of the right rear view).

[0040] Returning briefly to FIG. 1 , the lane identification system 100 may further include one or more processors, such as an electronic control unit (ECU) 106. The ECU 106 may be implemented as a single ECU or as multiple ECUs. The ECU 106 may be electrically connected (e.g., via a controller area network (CAN) bus and / or other protocols) to some or all components of the vehicle 102 and / or the lane identification system. The ECU 106 may be electrically connected to one or more cameras 116, one or more sensors 118, one or more displays 120, memory 108, and / or network access device 114. The ECU 106 may include one or more processors (or controllers) designed specifically for controlling the operation of the vehicle 102, such as accelerating, braking, controlling a panoramic view monitor (PVM) (e.g., one or more cameras 116) of the vehicle 102, etc. In some embodiments, ECU 106 may be or include a sensor fusion ECU for an advanced driver assistance system (ADAS), a panoramic view monitor (PVM) ECU, an engine control module (ECM), a transmission control module (TCM), a telematic control unit (TCU), an in-vehicle infotainment (IVI) ECU, and / or a graphics processing unit (GPU).

[0041] 1-3 collectively, the ECU 106 may receive or determine one or more conditions of the vehicle 102, including shift position, vehicle speed, turn signal status, steering angle, camera status, and / or proximity to one or more objects, via one or more sensors 118. Based on the received or determined one or more conditions of the vehicle 102, the ECU 106 may control and / or activate one or more cameras 116 to capture real-time video data including a rear view 208 or any other desired view.

[0042] The ECU 106 may receive real-time video data from one or more cameras 116, including the rear view 208. The ECU 106 may transmit the received real-time video data to one or more displays 120, such that at least one of the one or more displays 120 may display the rear view 208. For example, the ECU 106 may transmit the real-time video data including the rear view 208 to a rearview mirror 120c.

[0043] FIG. 4A shows a driver 410 and rearview mirror 420, which is an electronic mirror. Rearview mirror 420 can be similar to rearview mirror 120c. Rearview mirror 420 has a first mode in which it reflects light and functions as a traditional mirror. The first mode is shown in FIG. 4A. In the first mode, rearview mirror 420 shows a reflection 404 of objects behind driver 410. As shown in FIG. 4A, passengers and following vehicles are shown in reflection 404 from rearview mirror 420. Rearview mirror 420 may be switched to a second mode by engaging button 406.

[0044] FIG. 4B shows a driver 410 and the electronic rearview mirror 420 of FIG. 4A. The rearview mirror 420 has a second mode that displays image data detected by a camera facing toward the rear of the vehicle (e.g., rear-facing camera 116a). The second mode may be activated by engaging a button 406. Engaging the button 406 may also switch between the first and second modes. While a button 406 is shown in FIGS. 4A and 4B, any other switch or trigger mechanism may be used, such as a touchscreen button, a switch, a microphone for detecting voice commands, or a sensor for detecting gestures. The rearview mirror 420 may automatically switch to the second mode when the rearview mirror (or camera) 420 detects an obstacle or partial obstacle.

[0045] When in the second mode, rearview mirror 420 may display objects behind the vehicle with greater clarity and detail than a conventional mirror or rearview mirror 420 operating in the first mode. As shown in Figure 4B, more of the vehicle is shown on display 408 compared to reflection 404 in Figure 1A. When the camera used by rearview mirror 420 is pointed rearward and located behind the last row of seats, objects or occupants within the vehicle may not be visible on display 408.

[0046] FIG. 5A illustrates a rearview mirror 520 displaying a rear view of the vehicle's surroundings. The rearview mirror 520 may be an electronic mirror. According to various aspects, the rearview mirror 520 may be similar to the rearview mirror 120c of FIGS. 1 and 3. With reference to FIGS. 1, 2, and 5A collectively, the ECU 106 may transmit real-time video data received from a vehicle camera, including the rearview view 208, to the rearview mirror 520. In this manner, the rearview mirror 520 may display a rear view of the vehicle's surroundings. The rearview view includes lane markings 511 (e.g., solid or dashed lines) that demarcate the boundaries of one or more lanes. The ECU 106 may generate a lane identifier (e.g., hash marks, a pattern, and / or a color) that indicates the vehicle's current lane. The ECU 106 may transmit the lane identifier to a display to display the lane identifier overlaid on the rearview view, for example, as shown in any of FIGS. 5B, 5C, 5D, and / or 5E.

[0047] 5B shows a rearview mirror 520 displaying a lane identifier 521 overlaid on the vehicle's surrounding rear view. The lane identifier 521 can include one or more digital markers that highlight the boundaries of the lane. In the illustrated embodiment, the lane identifier 521 includes multiple digital markers in the form of lines and / or colors that highlight the lane markings 511 on the road. The lane identifier 521 can highlight one or more of the lane markings 511 on the road to allow the driver to quickly determine the current lane 522. In this manner, the lane identifier 521 can include digital markers that highlight the lane markings 511 of the vehicle's current lane 522.

[0048] FIG. 5C illustrates a rearview mirror 520 displaying lane identifiers overlaid on the vehicle's surrounding rear view. The lane identifiers may include one or more digital markers that highlight lane boundaries. In the illustrated embodiment, the lane identifiers include a first zone 531 and a second zone 532 that include the vehicle's current lane 522. In at least one pattern, the color and / or brightness of the second zone 532 is adjusted relative to the first zone 531. For example, the second zone 532 (which may include the remaining view outside the first zone 531) may be dimmed or color adjusted (e.g., by biasing the color toward a predetermined base color, such as aqua), among other adjustments. In this manner, the vehicle's current lane 522 may be emphasized or made prominent relative to the remaining view, allowing the driver to quickly determine the vehicle's lane from adjacent lanes.

[0049] In one embodiment, the ECU 106 may dim the second zone 532 relative to the first zone 531 and / or increase the brightness of the first zone 531 relative to the second zone 532 so that the driver can quickly and clearly identify the vehicle's current lane.

[0050] FIG. 5D shows a rearview mirror 520 displaying lane identifiers overlaid on the vehicle's surrounding rear view. The lane identifiers may include one or more digital markers that highlight lane boundaries. In the illustrated embodiment, the lane identifiers include multiple zones, each of which overlaps a single lane and is assigned a unique color. For example, a first zone 541, which includes the vehicle's current lane 522, may be displayed without color adjustment. In this manner, the vehicle's current lane 522 may appear normal. A second zone 542, which includes the adjacent lane 523, may be assigned a first color (e.g., yellow or any other suitable color) that overlaps the first adjacent lane 523. That is, the first adjacent lane 523 may be highlighted with the first color. A third zone 543, which includes the second adjacent lane 524, may be assigned a second color (e.g., blue or any other suitable color) that overlaps the second adjacent lane 524. That is, the second adjacent lane 524 may be highlighted with the second color. Any additional lanes detected by ECU 106 may be assigned respective zones. For example, fourth zone 544, which includes third lane 525, and fifth zone 546, which includes fourth lane 526, may each be assigned a color that overlaps the respective lane.

[0051] In various embodiments, the ECU 106 can assign a color to each lane 522, 523, 524, 525, and 526 such that the same color is used to highlight each of the lanes 522, 523, 524, 525, and 526 regardless of the vehicle's current lane. For example, referring collectively to FIGS. 5D and 5E, a vehicle may perform a lane change maneuver to move into lane 523. FIG. 5E illustrates the rear view of the vehicle after the vehicle has moved into lane 523. In response to the ECU 106 detecting that the vehicle has moved into lane 523, the ECU 106 can disable highlighting the vehicle's current lane 523 and enable highlighting the lane 522 ahead of the vehicle. The ahead lane 522 (which was not previously highlighted) can be highlighted with a predetermined color, such as orange. If the vehicle moves back into lane 522, ECU 106 can disable highlighting of lane 522 and re-enable highlighting of lane 523 to highlight lane 523 in a first color, such as yellow. Accordingly, ECU 106 stores the lane color in memory 108. In this manner, ECU 106 may generate one or more lane identifiers by processing the lane identifier data stored in memory 108. ECU 106 may transmit the lane identifier data (or processed lane identifier data) to a display (e.g., rearview mirror 520) so that the one or more lane identifiers are displayed by the display. By maintaining the lane colors, driver confusion can be reduced or eliminated.

[0052] In various embodiments, the ECU 106 can identify lanes, horizons, and any other suitable features in the real-time video data using feature detection, pattern matching, image detection, estimation using camera packing parameters, machine learning, and / or any other suitable techniques. Examples of feature detection techniques include the Harris corner detector (), the Shi-Tomasi corner detector (), the Canny edge detector (), the Bold detector (), and the Scale-Invariant Feature Transform (SIFT). The ECU 106 can detect lane markings 511 in the image data to demarcate one or more lanes. In this manner, the ECU 106 can overlay digital markers in the vehicle's surrounding rear view in the correct locations and / or zones. The digital markers can be iteratively adjusted and / or updated by the ECU 106 as the vehicle moves along the road, changes lanes, and / or changes direction of travel.

[0053] In various embodiments, the ECU 106 can use GPS data and / or map information (e.g., HDMAP) to assist in identifying lane markings. For example, the HDMAP information can provide the number of lanes on the roadway of the vehicle 102.

[0054] 1, 3, and 5A-5E collectively, the ECU 106 may be configured to determine and / or predict whether the vehicle 102 is performing a lane change maneuver. The ECU 106 may be configured to determine and / or predict a lane change maneuver using sensor feedback indicative of one or more conditions of the vehicle 102. The one or more conditions may include a current shift position, a current vehicle speed, a current turn signal condition, a current steering angle, and / or a current camera condition. In some embodiments, the ECU 106 may determine or receive the one or more conditions of the vehicle 102 via one or more sensors 118.

[0055] In various embodiments, the ECU 106 determines and / or predicts a lane change maneuver by detecting movement of the lane markings 511 relative to the vehicle 102 using real-time video data received from the rear-facing camera 116a. For example, the ECU 106 may detect that the vehicle 102 is moving laterally or across a lane when the lane markings 511 move horizontally across the display. The ECU 106 may similarly determine and / or predict a lane change maneuver using real-time video data received from a forward-facing camera (e.g., lane departure warning).

[0056] In various embodiments, the ECU 106 determines and / or predicts a lane change maneuver by detecting that the vehicle's turn signals are activated. The ECU 106 may receive the turn signal status from the turn signal sensor 118c.

[0057] In various embodiments, the ECU 106 determines and / or predicts a lane change maneuver using the steering angle sensor 118d. The ECU 106 can determine and / or anticipate a lane change maneuver in response to the steering angle sensor 118d indicating that the steering wheel 306 is at a steering angle that indicates a lane change.

[0058] ECU 106 may use any suitable method to determine and / or anticipate a lane change maneuver, and the disclosure is not intended to be particularly limited in this respect.

[0059] 6 is a flow diagram of an example process 600 for controlling a lane identification system. One or more computers or one or more processing devices, such as the ECU 106 of the lane identification system of FIG. 1 , may perform the process 600. For ease of explanation, the process 600 is described below with reference to FIGS. 1-5E. However, the process 600 of the present disclosure is not limited to use with the exemplary lane identification system of FIGS. 1-5E.

[0060] The lane identification system may receive 602 real-time video data from one or more cameras 116. The real-time video data may include the rear view 208.

[0061] The lane identification system may generate 604 a lane identifier indicating the current lane of the vehicle 102. The lane identifier may include a marker that highlights the lane markings of the vehicle's current lane, for example, as described with reference to FIG. 5B. The lane identifier may include a first zone and a second zone that include the vehicle's current lane, where at least one of the color or brightness of the second zone is adjusted relative to the first zone, for example, as described with reference to FIG. 5C. The lane identifier may include multiple colors, each assigned to a respective lane of multiple lanes identified in the field of view, for example, as described with reference to FIGS. 5D and 5E.

[0062] The lane identification system may transmit the lane identifier to a display (e.g., rearview mirror 120c) for displaying the lane identifier overlaid on the field of view via the display (606). For example, ECU 106 can transmit the lane identifier to rearview mirror 120c for displaying the lane identifier overlaid on the field of view as shown in these example figures, including the combination of Figures 5B-5E.

[0063] In various embodiments, the lane identification system may determine that the vehicle is performing a lane change maneuver. In response to detecting the lane change maneuver, the lane identification system may temporarily disable lane identifiers from the display while the vehicle 102 is performing the lane change so as not to confuse the driver of the location of the lane. In response to detecting that the vehicle 102 has completed the lane change maneuver, the lane identification system may generate an updated lane identifier indicating the vehicle's new current lane. The lane identification system may transmit the updated lane identifier to the rearview mirror 120c to display the updated lane identifier superimposed within a real-time field of view via the rearview mirror 120c.

[0064] FIG. 7 is a flow diagram of an example process 700 for controlling a lane identification system. One or more computers or one or more processing devices, such as the ECU 106 of the lane identification system of FIG. 1 , may perform the process 700. For ease of explanation, the process 700 is described below with reference to FIGS. 1-5E. However, the process 700 of the present disclosure is not limited to use with the exemplary lane identification system of FIGS. 1-5E. Various aspects of the process 700 may be similar to the process 600 of FIG. 6.

[0065] The lane identification system may receive 602 real-time video data from one or more cameras 116. The real-time video data may include the rear view 208.

[0066] The lane identification system may assign a first color to the vehicle's first (current) lane and may assign a second color to the vehicle's second (adjacent) lane (703). For example, referring briefly to Figures 1 and 5D together, ECU 106 may assign lane 522 a first color (e.g., orange). ECU 106 may assign adjacent lane 523 a second color (e.g., yellow).

[0067] The lane identification system may generate 704a a lane identifier indicating the current lane of the vehicle 102. The lane identifier may include multiple colors, including the second color assigned in step 703, one color assigned to each lane of multiple lanes identified in the field of view, for example, as described with respect to Figures 5D and 5E.

[0068] The lane identification system may transmit (706a) the lane identifier to a display (e.g., rearview mirror 120c) for displaying the lane identifier overlaid on the field of view via the display. The lane identifier may include highlighting lane 523 in a second color. The lane identifier may omit highlighting lane 522. That is, the first color may be omitted for the vehicle's 102's current lane 522 (see FIG. 5D).

[0069] The lane identification system may determine 708 that the vehicle 102 has moved from lane 1 to lane 2. For example, the ECU 106 may detect that the vehicle 102 has moved from lane 522 (see FIG. 5D) to lane 523 (see FIG. 5E).

[0070] The lane identification system may generate 704b an updated lane identifier indicating the updated current lane of the vehicle 102. Step 704b occurs in response to the ECU 106 detecting a lane change. The lane identifier may include multiple colors, including the first color assigned in step 703, with each color assigned to a respective lane of multiple lanes identified in the field of view, for example, as described with respect to Figures 5D and 5E.

[0071] The lane identification system may transmit (706b) the updated lane identifier to a display (e.g., rearview mirror 120c) for displaying the updated lane identifier overlaid on the field of view via the display. The updated lane identifier may include highlighting lane 522 in a first color. The updated lane identifier may omit highlighting lane 523. That is, the second color may be omitted for the vehicle's 102's current lane 523 (see FIG. 5E).

[0072] Exemplary embodiments of the disclosure have been disclosed in an illustrative manner. Accordingly, the terminology used throughout should be read without limitation. While minor modifications to the teachings herein will occur to those skilled in the art, it should be understood that what is intended to be limited within the scope of the patent granted hereunder are all embodiments that reasonably fall within the scope of the advances to the art brought by this disclosure, and that the scope should not be limited except in light of the appended claims and their equivalents.

Claims

1. 1. A lane identification system for assisting a vehicle driver, comprising: a display located on or within the vehicle; an electronic control unit (ECU) connected to the display, transmitting real-time video data to the display for displaying a view around the vehicle via the display; generating a lane identifier indicating a current lane of the vehicle; transmitting the lane identifier to the display for displaying the lane identifier overlaid on the field of view via the display; An ECU configured as follows: A lane identification system comprising:

2. 2. The lane identification system of claim 1, further comprising a camera coupled to the ECU and configured to capture the real-time video data of the field of view of the surroundings of the vehicle.

3. 3. The lane identification system of claim 2, wherein the field of view of the periphery of the vehicle includes a rear field of view relative to the vehicle.

4. 4. The lane identification system of claim 3, wherein the display comprises a digital rearview mirror configured to display the rear view and / or the lane identifier.

5. 2. The lane identification system of claim 1, wherein the lane identifier comprises a marker that highlights a lane marking of the current lane of the vehicle.

6. 2. The lane identification system of claim 1, wherein the lane identifier includes a first zone that includes the current lane of the vehicle and a second zone.

7. 7. The lane identification system of claim 6, wherein at least one of the color or brightness of the second zone is coordinated with respect to the first zone.

8. 2. The lane identification system of claim 1, wherein the lane identifier comprises a plurality of colors, each color of the plurality of colors being assigned to a respective lane of a plurality of lanes identified in the field of view.

9. 2. The lane identification system according to claim 1, wherein the ECU: determining that the vehicle is performing a lane change maneuver; generating an updated lane identifier indicating a new current lane for the vehicle; transmitting the updated lane identifier to the display for displaying the updated lane identifier overlaid on the field of view via the display; A lane identification system configured to:

10. 10. The lane identification system of claim 9, wherein the ECU is further configured to remove the lane identifier from the display during the lane change maneuver.

11. 1. A lane identification system for assisting a vehicle driver, comprising: a camera configured to capture real-time video data of a peripheral field of view of the vehicle; a display located on or within the vehicle; an electronic control unit (ECU) connected to the display and the camera, transmitting the real-time video data to the display for displaying the view of the surroundings of the vehicle via the display; generating a lane identifier indicating a current lane of the vehicle; transmitting the lane identifier to the display for displaying the lane identifier overlaid on the field of view via the display; and an ECU configured as follows: A lane identification system comprising:

12. 12. The lane identification system of claim 11, the visibility of the surroundings of the vehicle includes a rear visibility relative to the vehicle; The display displays the rear view and / or the lane identifier.

1. A lane identification system including a digital rearview mirror configured to:

13. 12. The lane identification system of claim 11, wherein the lane identifier comprises a marker that highlights a lane marking of the current lane of the vehicle.

14. 12. The lane identification system of claim 11, the lane identifier includes a first zone that includes the current lane of the vehicle and a second zone; At least one of the color or brightness of the second zone is adjusted relative to the first zone. Lane identification system.

15. 12. The lane identification system of claim 11, wherein the lane identifier comprises a plurality of colors, each color of the plurality of colors being assigned to a respective lane of a plurality of lanes identified in the field of view.

16. 12. The lane identification system according to claim 11, wherein the ECU: determining that the vehicle is performing a lane change maneuver; removing the lane identifier from the display during the lane change maneuver; determining that the vehicle has completed the lane change maneuver; generating an updated lane identifier indicating a new current lane for the vehicle; transmitting the updated lane identifier to the display for displaying the updated lane identifier overlaid on the field of view via the display; A lane identification system configured to:

17. 1. A method for assisting a vehicle driver, comprising: transmitting real-time video data via an electronic control unit (ECU) to a display for displaying a peripheral view of the vehicle via the display; generating a lane identifier indicating a current lane of the vehicle via the ECU; transmitting the lane identifier via the ECU to the display for displaying the lane identifier overlaid on the field of view via the display; A method comprising:

18. 18. The method of claim 17, the visibility of the surroundings of the vehicle includes a rear visibility relative to the vehicle; The display displays the rear view and / or the lane identifier. The method includes:

19. 18. The method of claim 17, wherein the lane identifier comprises: a marker highlighting the lane markings of the current lane of the vehicle; a first zone including the current lane of the vehicle; and a second zone, wherein at least one of a color or brightness of the second zone is adjusted relative to the first zone; a plurality of colors, each color of the plurality of colors being assigned to a respective lane of a plurality of lanes identified in the visual field; The method includes at least one of the following:

20. 18. The method of claim 17, determining via the ECU that the vehicle is performing a lane change maneuver; Disabling the lane identifier from the display via the ECU during the lane change maneuver; determining via the ECU that the vehicle has completed the lane change maneuver; generating, via the ECU, an updated lane identifier indicating a new current lane for the vehicle; transmitting the updated lane identifier via the ECU to the display for displaying the updated lane identifier overlaid on the field of view via the display; The method further comprises:

Images