Method, system and apparatus for electron mirror traffic lane recognition

Abstract

The invention relates to a method, a system and equipment for electronic mirror traffic lane recognition. Methods, systems, and apparatus for assisting a driver of a vehicle. The traffic lane recognition system may include one or more displays located on or within the vehicle. The system may also include an electronic control unit (ECU) coupled to the one or more displays. The ECU may send the real-time video data to a display to display a field of view of a surrounding area of the vehicle. The ECU may also generate a lane identifier indicative of a current traffic lane of the vehicle. The ECU may also send the lane identifier to a display to display the lane identifier covered within a field of view of the surrounding area of the vehicle.

Description

Technical Field

[0001] This disclosure relates to methods, systems, and / or devices for traffic lane recognition using electronic mirrors. Background Technology

[0002] Modern vehicles are often equipped with numerous screens and cameras to assist vehicle users (e.g., drivers and / or passengers) in various vehicle maneuvers, such as changing lanes, parking, and / or reversing. One or more screens can allow the driver to view live video of the area surrounding the vehicle captured by one or more cameras. However, drivers can become confused or overwhelmed when trying to determine which part of the image belongs to their own lane or an adjacent lane, especially during a rapid scan. Inclement weather conditions (e.g., rain, snow, sleet, etc.) can also make it difficult to judge or see lanes or lane markings. The risk of a driver being unable to correctly judge or see their lane is particularly high when the driver is unfamiliar with the vehicle, is fatigued, is in traffic congestion, is changing lanes, the screen image does not include the vehicle body, and / or the camera is not precisely centered on the vehicle.

[0003] Accordingly, it is desirable to provide methods, systems, and devices for traffic lane recognition using electronic mirrors. Summary of the Invention

[0004] Generally, one aspect of the subject matter described in this disclosure can be implemented in a traffic lane recognition system for a vehicle. The traffic lane recognition system may include one or more displays located on or within the vehicle. The traffic lane recognition system may also include an electronic control unit (ECU) coupled to one or more displays. The ECU may be configured to send real-time video data to the displays to display a field of view of the area surrounding the vehicle. The ECU may also be configured to generate a lane identifier indicating the vehicle's current traffic lane. The ECU may further be configured to send the lane identifier to the displays to display the lane identifier overlaid within the field of view.

[0005] In one aspect, this subject matter can be implemented in a traffic lane recognition system for assisting a driver of a vehicle. The traffic lane recognition system may include a camera configured to capture real-time video data of the field of view of the area surrounding the vehicle. The traffic lane recognition system may also include a display located on or within the vehicle. The traffic lane recognition system may further include an electronic control unit (ECU) coupled to the display and the camera. The ECU may be configured to send real-time video data to the display so as to display the field of view of the area surrounding the vehicle. The ECU may also be configured to generate a lane identifier indicating the vehicle's current traffic lane. The ECU may further be configured to send the lane identifier to the display so as to display the lane identifier overlaid within the field of view.

[0006] In one aspect, this subject matter can be implemented in a method for assisting a driver of a vehicle. The method may include transmitting real-time video data via the vehicle's electronic control unit (ECU) to a display so as to show the field of view of the area surrounding the vehicle. The method may also include generating a lane identifier via the ECU indicating the vehicle's current traffic lane. The method may further include transmitting the lane identifier via the ECU to the display so as to show the lane identifier covering the field of view.

[0007] These and other embodiments may optionally include one or more of the following features.

[0008] In various aspects, the traffic lane recognition system may also include a camera coupled to the ECU and configured to capture real-time video data of the field of view of the area surrounding the vehicle.

[0009] In all respects, the field of view of the area surrounding the vehicle includes the field of view relative to the rear of the vehicle.

[0010] In various aspects, the display includes a digital rearview mirror configured to display the rear view and / or lane identifiers.

[0011] In various aspects, lane identifiers include lane markings that highlight the vehicle's current traffic lane.

[0012] In all aspects, lane identifiers include a first zone and a second zone, with the first zone including the vehicle's current traffic lane.

[0013] In all aspects, at least one of the colors or brightness of the second zone is adjusted relative to the first zone.

[0014] In all aspects, lane identifiers include multiple colors, each of which is assigned to a corresponding traffic lane among the multiple traffic lanes identified in the field of view.

[0015] In various aspects, the ECU is also configured to determine that the vehicle is performing a lane change maneuver. In various aspects, the ECU is also configured to turn off the display of lane identifiers during a lane change maneuver. In various aspects, the ECU is also configured to determine that the vehicle has completed a lane change maneuver. In various aspects, the ECU is also configured to generate an updated lane identifier indicating the vehicle's new current traffic lane. In various aspects, the ECU is also configured to send the updated lane identifier to a display for display overlaid on the field of view. Attached Figure Description

[0016] Other systems, methods, features, and advantages of this disclosure will become apparent to those skilled in the art upon examination of the following figures and detailed description. Component portions shown in the figures are not necessarily drawn to scale and may be exaggerated to better illustrate key features of this disclosure. In the figures, the same reference numerals denote the same parts throughout different views.

[0017] Figure 1 This is a block diagram of an example traffic lane recognition system for vehicles according to one aspect of this disclosure.

[0018] Figure 2 This is based on one aspect of the disclosure, including Figure 1 A schematic top view of an example vehicle in an example traffic lane recognition system.

[0019] Figure 3 This is based on one aspect of the disclosure, including Figure 1 An illustration of an example driver's cab of a vehicle with an example traffic lane recognition system.

[0020] Figure 4A and Figure 4B The illustration depicts a driver using an electronic mirror according to various aspects of this disclosure.

[0021] Figure 5A The illustration shows an electronic mirror displaying the rear view of a vehicle according to various aspects of this disclosure.

[0022] Figure 5B-5E The illustrations depict various aspects according to this disclosure. Figure 5A The electronic mirror displays the vehicle's rear view and is covered with various lane markings.

[0023] Figure 6 It is a control based on one aspect of this disclosure Figure 1 A flowchart illustrating an example process of a lane recognition system.

[0024] Figure 7 It is a control based on one aspect of this disclosure Figure 1 A flowchart illustrating an example process of a lane recognition system. Detailed Implementation

[0025] This document discloses methods, systems, apparatuses, and / or vehicles for implementing a traffic lane recognition system. The traffic lane recognition system may include one or more displays located in the driver's compartment of the vehicle. Specific embodiments of the subject matter described in this disclosure can be implemented to achieve one or more of the following advantages: The traffic lane recognition system can generate a lane identifier that indicates the current lane to the user of the vehicle (e.g., the driver and / or passengers). The traffic lane recognition system allows the user to quickly and clearly identify and understand their own traffic lane, which is separated from adjacent traffic lanes. Moreover, the lane identifier can quickly direct the user's attention to the vehicle's current traffic lane. The lane identifier can be a graphic icon or sign displayed by at least one display (e.g., a rearview mirror). By indicating the vehicle's current traffic lane, the lane identifier can eliminate or at least reduce the time spent by the user looking at a display to determine the vehicle's current traffic lane, thereby improving safety and reducing user confusion or error.

[0026] Furthermore, traffic lane recognition systems can further enhance safety by assigning lane colors, ensuring that each lane maintains its assigned color as lane identifiers are automatically and dynamically updated as vehicles change lanes. For example, when a vehicle performs a lane change maneuver to move from a first lane to a second lane, the traffic lane recognition system can update the lane identifier based on the new field of view in the second lane. This updated lane identifier can maintain the lane color displayed when the vehicle was in the first lane, thereby reducing user confusion.

[0027] As used herein, "electronic mirror" can refer to a device that uses a camera or other image sensor to detect image data behind a vehicle and display the image data on a screen inside the vehicle. The screen can be integrated into an existing mirror (e.g., a rearview or side mirror) or can be a separate mirror or screen. In this way, the camera and screen constitute an electronic mirror, which behaves and functions similarly to a conventional mirror.

[0028] Figure 1This is a block diagram of an example traffic lane recognition system 100. The traffic lane recognition system 100, or a portion thereof, may be modified, coupled to, included in, or contained within vehicle 102, or detached from vehicle 102. Vehicle 102 may be a vehicle capable of transporting people, objects, or permanently or temporarily fixed installations. Vehicle 102 may be a self-propelled wheeled vehicle, such as a car, SUV, truck, bus, van, or other vehicle powered by a motor, battery, or fuel cell. For example, vehicle 102 may be an electric vehicle, a hybrid vehicle, a hydrogen fuel cell vehicle, a plug-in hybrid vehicle, or any other type of vehicle with a fuel cell stack, motor, engine, and / or generator. Other examples of vehicles include bicycles, trains, airplanes, or ships, and any other form of transportable vehicle. Vehicle 102 may be semi-autonomous or autonomous. That is, vehicle 102 may operate and navigate autonomously without human input. Autonomous vehicles may have and use one or more sensors and / or navigation units to drive autonomously.

[0029] The traffic lane recognition system and / or vehicle 102 may include a motor and / or generator 132, a battery 134, and / or a transmission 136. The motor and / or generator 132 may be located in the engine compartment of vehicle 102. The motor and / or generator 132 may be an internal combustion engine (ICE). In this respect, the motor and / or generator 132 may burn an air-fuel mixture to power vehicle 102 and / or its components and / or traffic lane recognition system 100. Accordingly, the motor and / or generator 132 may accelerate, decelerate, or maintain a desired speed for vehicle 102. The motor and / or generator 132 may include a combination of an ICE and an electric motor, such as for hybrid vehicle applications. In an example, the motor and / or generator 132 may be an electric motor. In this respect, the motor and / or generator 132 may be an electric motor and generator that converts electrical energy into mechanical power (such as torque) and mechanical power into electrical energy. The motor and / or generator 132 may be electrically connected to battery 134. The motor and / or generator 132 can convert energy from the battery 134 into mechanical power and can supply energy back to the battery 134, for example, via regenerative braking. The battery 134 can be electrically connected to the motor and / or generator 132 and can supply electrical energy to and / or receive electrical energy from the motor and / or generator 132. The battery 134 can supply electrical energy to a traffic lane recognition system.

[0030] The transmission 136 may be an automatic transmission (e.g., a 6-speed, 7-speed, 8-speed, 9-speed, or 10-speed automatic transmission) or a manual transmission (e.g., a six-speed manual transmission) having at least one reverse gear, one neutral gear, and at least one forward gear (or drive gear). In the example, the transmission 136 may be an electric vehicle transmission (e.g., a single-speed or two-speed transmission). The transmission 136 may receive torque from the motor and / or generator 132 and may transmit torque to one or more wheels of the vehicle 102.

[0031] The traffic lane recognition system 100 may also include a network access device 114. The network access device 114 may be electrically connected to the ECU 106 and may include a communication port or channel, such as a Wi-Fi unit. The network access device 114 may include one or more of the following: a unit, a radio frequency identification (RFID) tag or reader, a DSRC unit, and / or a cellular network unit for accessing network 140 (e.g., CDMA, GSM, 3G, 4G, 5G, etc.). The network access device 114 can send and receive data to and from devices and systems not directly connected to vehicle 102. For example, ECU 106 may communicate via network access device 114 with external display 120f (e.g., mobile device, telephone, tablet, laptop, etc.), one or more cameras 116, one or more sensors 118, one or more displays 120, and / or memory 108. In this example, a rear-facing camera 116a may wirelessly communicate with ECU 106 via network access device 114. This can reduce or eliminate the wiring requirement from rear-facing camera 116a to vehicle 102.

[0032] The traffic lane recognition system 100 may also include a memory 108. The memory 108 may be electrically connected to the ECU 106. In one example, the memory 108 may be communicatively coupled (e.g., via network 140) to the ECU 106, such that the memory 108 is remote from the ECU 106 and / or the vehicle 102. In other examples, the memory 108 may be electrically connected to the ECU 106, and a remote memory (e.g., a remote database) may be communicatively coupled to the ECU 106, wherein the remote memory has similar, additional, and / or different functions than the memory 108 (e.g., larger storage capacity, enabling wireless updates, etc.). The memory 108 may store instructions that can be executed on the ECU 106 and may include one or more of random access memory (RAM) or other volatile or non-volatile memories. The memory 108 may be a non-transitory memory or data storage device, such as a hard disk drive, solid-state drive, hybrid disk drive, or other suitable data storage device, and may also store machine-readable instructions that can be loaded and executed by the ECU 106. The memory 108 may store vehicle parameters (e.g., vehicle weight, vehicle length, vehicle width, vehicle height, transmission gear information, etc.). The memory 108 may also store one or more traffic lane colors as described herein. One or more traffic lane colors may be predetermined and set by the manufacturer of the vehicle 102, and / or may be set by the user of the vehicle 102 (e.g., via user interface 120a).

[0033] Figure 2 The diagram includes Figure 1 Example vehicle 102 of example traffic lane recognition system 100. Combined reference. Figure 1 and Figure 2 The traffic lane recognition system 100 may also include one or more cameras 116. The one or more cameras 116 may be coupled to the exterior 201 of the vehicle 102. In the example, the one or more cameras 116 are coupled to the interior of the vehicle 102 but face outwards to capture the exterior of the vehicle 102. The one or more cameras 116 may include a rear-facing camera 116a configured to provide a rear view of the rear of the vehicle 102. Although shown as being located at the rear center of the vehicle 102, the rear-facing camera 116a may be located in any suitable position on the vehicle 102, such as the left side of the vehicle 102, the right side of the vehicle 102, the left mirror 120d of the vehicle 102, the right mirror 120e of the vehicle 102, or any other suitable position, depending on the desired design of the vehicle 102. In various aspects, the rear-facing camera 116a may be located off-center from the vehicle 102. In the example, the one or more cameras 116 may include a side-facing camera, a front-facing camera, or any other suitable camera, depending on the desired viewing direction provided to the user of the vehicle 102.

[0034] One or more cameras 116 may be digital cameras, infrared thermal cameras, and / or night vision cameras (e.g., utilizing active illumination and / or image enhancement). In the example, one or more cameras 116 may be and / or include a panoramic monitor (PVM) system comprising vehicle 102. One or more cameras 116 may provide, capture, and / or record real-time video (or video data) including images and / or video of the area surrounding vehicle 102. A rear-facing camera 116a may provide, capture, and / or record real-time video (or video data) including images and / or video of the area behind vehicle 102. Each of the one or more cameras 116 may capture one or more fields of view that may include at least a portion of the area surrounding vehicle 102. In the example, each of the one or more cameras 116 may allow a user to change the zoom level of each camera (e.g., via user interface 120a), and / or the traffic lane recognition system may automatically change the zoom level of each camera based on the speed of vehicle 102 (e.g., the zoom level decreases as the speed of vehicle 102 increases). In the example, one or more cameras 116 may include trailer cameras that can provide a rear view of the trailer to users of the vehicle.

[0035] A rear-facing camera 116a may be coupled to the exterior 201 and / or rear of vehicle 102 (e.g., rear bumper, tailgate, etc.). In the example, the traffic lane recognition system 100 may include multiple rear-facing cameras, each with a different field of view and / or orientation. The rear-facing camera 116a may provide, capture, and / or record real-time video relative to the rear field of view 208 of vehicle 102 (e.g., capturing a portion of the surrounding area behind vehicle 102). In the example, the rear-facing camera 116a may be a wireless camera, such that it can wirelessly transmit the real-time video of the rear field of view 208 to ECU 106. In the example, the rear-facing camera 116a may transmit the real-time video of the rear field of view 208 to ECU 106 via a wired connection.

[0036] The traffic lane recognition system 100 may also include one or more sensors (or one or more vehicle sensors) 118. One or more sensors 118 may be integrated with the vehicle 102. One or more sensors 118 may periodically and / or continuously measure, detect (or indicate), and / or determine one or more conditions (or one or more vehicle conditions) of the vehicle 102. One or more conditions of the vehicle 102 may include shift position, vehicle speed, turn signal condition, steering angle, camera condition, and / or proximity to one or more objects. One or more sensors 118 may be and / or include software, hardware, firmware, or a combination thereof for measuring, detecting, and / or determining one or more conditions of the vehicle 102. One or more sensors 118 may include a transmission sensor (or shift sensor) 118a, a speed sensor 118b, a turn signal sensor 118c, a steering angle sensor 118d, and / or one or more distance sensors 118e. In the example, the traffic lane recognition system may include two or more, three or more, four or more, or all of the following: a transmission sensor 118a, a speed sensor 118b, a steering signal sensor 118c, a steering angle sensor 118d, and / or one or more distance sensors 118e.

[0037] The transmission sensor 118a may be located within the vehicle 102 and / or coupled to the transmission 136 of the vehicle 102. The transmission sensor 118a may indicate, detect, and / or determine the shift position (or gear position or mode) of the vehicle 102 and / or the transmission 136. The shift position may include drive, reverse, and / or neutral. In this example, the traffic lane recognition system 100 may include multiple transmission sensors.

[0038] Speed ​​sensor 118b may be located within vehicle 102. Speed ​​sensor 118b can measure, detect, and / or determine the vehicle speed of vehicle 102. In this example, speed sensor 118b may measure, detect, and / or determine vehicle speed based on the rotational speed of the output shaft of transmission 136, the rotational speed of one or more wheels of vehicle 102, and / or GPS signals. In this example, traffic lane recognition system 100 may include multiple speed sensors 118b.

[0039] Figure 3 It includes Figure 1 Illustration of the driver's cab 302 of vehicle 102 in a traffic lane recognition system. (Combined reference) Figure 1 and Figure 3The turn signal sensor 118c may be located within the vehicle 102 and / or coupled to the indicator 308 and / or steering wheel 306 of the vehicle 102. The turn signal sensor 118c may indicate, detect, and / or determine the turn signal status of the vehicle 102. The turn signal status may include whether the left turn signal of the vehicle 102 is activated and / or whether the right turn signal of the vehicle 102 is activated. For example, the turn signal sensor 118c may detect when the driver of the vehicle 102 activates the left turn signal or the right turn signal of the vehicle 102. In this example, the turn signal sensor 118c may detect when the vehicle 102 activates the left turn signal or the right turn signal during, for example, autonomous driving or semi-autonomous driving. In this example, the traffic lane recognition system 100 may include multiple turn signal sensors 118c.

[0040] Steering angle sensor 118d may be located within vehicle 102 and / or coupled to steering wheel 306 of vehicle 102. Steering angle sensor 118d can measure, detect, and / or determine the steering angle of vehicle 102 (e.g., the degree by which steering wheel 306 is rotated left or right from a central (or reference) position). In this example, the traffic lane recognition system may include multiple steering angle sensors. Steering angle sensor 118d can indicate whether vehicle 102 is turning (e.g., when vehicle 102 turns 20 degrees or more to the left or right) or making a small change in direction (e.g., when vehicle 102 turns between 0 and 20 degrees to the left or right).

[0041] One or more distance sensors 118e can be coupled to the exterior 201 of the vehicle 102 (see [link]). Figure 2 One or more distance sensors 118e may be and / or include cameras, sonar sensors, radar sensors, and / or lidar sensors. One or more distance sensors 118e can measure, detect, and / or determine the proximity of one or more objects in the area surrounding the vehicle 102. One or more distance sensors 118e can measure, detect, and / or determine the proximity of one or more objects by collecting spatial information about the objects and constructing a point cloud. In this example, the traffic lane recognition system 100 may include multiple distance sensors 118e.

[0042] Combination Reference Figure 1 , Figure 2 and Figure 3The traffic lane recognition system 100 may also include one or more displays (or one or more displays and / or mirrors) 120. The one or more displays 120 may include a user interface (or infotainment display) 120a, an instrument panel 120b, a rearview mirror 120c, a left mirror 120d, and / or a right mirror 120e. The one or more displays 120 may be original to the vehicle 102 and / or may be retrofitted for the vehicle 102. The one or more displays 120 may include a liquid crystal display (LCD), a light-emitting diode (LED) display, a segmented display, a holographic display, an electronic paper display, a laser color video display, and / or other display types. In the example, the one or more displays 120 may include an external display 120f (e.g., a mobile phone, a laptop computer, a tablet computer, etc.) which may have some or all of the functions (e.g., receiving and displaying images) of the user interface 120a, instrument panel 120b, rearview mirror 120c, left mirror 120d, and / or right mirror 120e. At least one of the displays 120 can display lane identifiers that identify traffic lanes around the vehicle 102 (discussed in more detail below). In the example, the rearview mirror 120c displays the lane identifiers.

[0043] User interface 120a may be located in the driver's cab 302 of vehicle 102, and / or may be attached to the dashboard 310 of vehicle 102 (e.g., Figure 3 (As shown in the diagram). User interface 120a can provide an interface for users of vehicle 102 to interact with and / or receive output from electronic control unit (ECU) 106. User interface 120a may have user interface elements such as one or more screens and / or one or more input / output devices with buttons, switches, microphones, speakers, gesture monitoring sensors, knobs, graphical user interfaces (GUIs), and / or electrically connected to ECU 106 to provide input and / or output of information (or data) to ECU 106. User interface 120a may display lane identifiers, rear field of view 208, and / or any other suitable field of view (e.g., left rear field of view, right rear field of view, second rear field of view, forward field of view, etc.).

[0044] The instrument panel 120b may be located within the cab 302 of the vehicle 102, and / or may be coupled to the instrument panel 310 of the vehicle 102 (e.g., Figure 3 (As shown in the diagram). The instrument panel 120b may have one or more screens and / or physical indicators to display vehicle information, such as a speedometer, tachometer, energy and / or fuel level indicator, map, etc. In the example, the instrument panel 120b may display lane identifiers, rear field of view 208, and / or any other suitable field of view (e.g., left rear field of view, right rear field of view, second rear field of view, forward field of view, etc.).

[0045] The rearview mirror 120c may be located within the driver's cab 302 of the vehicle 102, and / or may be coupled to the ceiling 312 of the vehicle 102 (e.g., Figure 3 (As shown in the diagram). The rearview mirror 120c may be an electronic mirror. The rearview mirror 120c may include an electronic rearview mirror such that the surface 314 of the rearview mirror 120c may be or include one or more screens that can display lane markings, rear field of view 208, and / or any other suitable field of view (e.g., a second rear field of view, etc.). In the example, the rearview mirror 120c may be a conventional (or non-electronic) rearview mirror such that the surface 314 of the rearview mirror 120c may be or include a reflective surface that allows the user to see at least a portion of the rear field of view. In the example, the rearview mirror 120c may have an electronic mode (e.g., utilizing one or more screens of the surface 314) and a conventional mode (e.g., utilizing the reflective surface of the surface 314).

[0046] The left mirror (or left-side side mirror) 120d can be located inside the driver's cab 302 of the vehicle 102 or can be coupled to the exterior 201 of the vehicle 102 (e.g., Figure 2 (As shown in the diagram). The left mirror 120d may include an electronic (or digital) left-side mirror with one or more screens, which may display lane markings and a left rear view. The left mirror 120d may include a conventional (or non-electronic) rearview mirror, such that the surface of the left mirror 120d may be or include a reflective surface that allows the user to see at least a portion of the left rear view. In the example, the left mirror 120d may have an electronic mode (e.g., using one or more screens to display the left rear view) and a conventional mode (i.e., using a reflective surface to show reflections of at least a portion of the left rear view).

[0047] The right-side mirror (or right-side viewing mirror) 120e may be located inside the driver's cab 302 of the vehicle 102 and / or may be coupled to the exterior 201 of the vehicle 102 (e.g., Figure 2 (As shown in the diagram). The right-side mirror 120e may include an electronic (or digital) right-side mirror with one or more screens, which may display lane markings and a right rear view. The right-side mirror 120e may include a conventional (or non-electronic) rearview mirror, such that the surface of the right-side mirror 120e may be or include a reflective surface that allows the user to see at least a portion of the right rear view. In the example, the right-side mirror 120e may have an electronic mode (e.g., using one or more screens to display the right rear view) and a conventional mode (i.e., using a reflective surface to show reflections of at least a portion of the right rear view).

[0048] Brief Review Figure 1The traffic lane recognition system 100 may also include one or more processors, such as an electronic control unit (ECU) 106. The ECU 106 may be implemented as a single ECU or multiple ECUs. The ECU 106 may be electrically connected to some or all components of the vehicle 102 and / or the traffic lane recognition system (e.g., via a controller area network (CAN) bus and / or other protocols). The ECU 106 may be electrically connected to one or more cameras 116, one or more sensors 118, one or more displays 120, a memory 108, and / or a network access device 114. The ECU 106 may include one or more processors (or controllers) specifically designed to control the operation of the vehicle 102 (such as acceleration, braking, controlling the panoramic monitor (PVM) of the vehicle 102 (e.g., one or more cameras 116), etc.). In the example, the ECU 106 may be and / or include an advanced driver assistance system (ADAS) sensor fusion ECU, a panoramic monitor (PVM) ECU, an engine control module (ECM), a transmission control module (TCM), a telematics control unit (TCU), an in-vehicle infotainment (IVI) ECU, and / or a graphics processing unit (GPU).

[0049] Combination Reference Figure 1-3 The ECU 106 can receive or determine one or more conditions of the vehicle 102 via one or more sensors 118, including shift position, vehicle speed, steering signal status, steering angle, camera conditions, and / or proximity to one or more objects. The ECU 106 can control and / or activate one or more cameras 116 to capture real-time video data, including a rear field of view 208 or any other desired field of view, based on the received or determined state of the vehicle 102.

[0050] ECU 106 can receive real-time video data, including a rear field of view 208, from one or more cameras 116. ECU 106 can transmit the received real-time video data to one or more displays 120, such that at least one of the displays 120 can display the rear field of view 208. For example, ECU 106 can transmit the received real-time video data, including the rear field of view 208, to a rearview mirror 120c.

[0051] Figure 4A The illustration shows a driver's side mirror 410 and a rearview mirror 420, which is an electronic mirror. The rearview mirror 420 can be similar to a rearview mirror 120c. The rearview mirror 420 can have a first mode in which it reflects light and functions like a conventional mirror. This first mode... Figure 4A As shown in the diagram. In the first mode, the rearview mirror 420 shows a reflection 404 of an object behind the driver 410. (See diagram below.) Figure 4AAs shown, the passenger and following vehicle are displayed in reflection 404 from the rearview mirror 420. The rearview mirror 420 can be switched to a second mode by pressing button 406.

[0052] Figure 4B The illustration shows driver 410 and Figure 4A The rearview mirror 420 is an electronic mirror. The rearview mirror 420 has a second mode in which it displays image data detected by a rear-facing camera (e.g., rear-facing camera 116a). The second mode can be activated by pressing button 406. Further pressing of button 406 switches between the first and second modes. Although button 406 is... Figure 4A and Figure 4B The diagram shows the second mode, but any other switch or trigger mechanism can be used, such as a touchscreen button, a switch, a microphone for detecting voice commands, or a sensor for detecting gestures. The rearview mirror 420 can also automatically switch to the second mode when it detects an obstacle or part of an obstacle.

[0053] When in the second mode, the rearview mirror 420 can display objects behind the vehicle more clearly and in more detail than a conventional mirror or rearview mirror 420 operating in the first mode. For example... Figure 4B As shown, with Figure 4A Compared to the reflection 404, the display 408 shows more of the vehicle. When the camera used by the rearview mirror 420 is pointed rearward and located behind the last row of seats, objects or occupants inside the vehicle may not be visible in the display 408.

[0054] Figure 5A The illustration shows a rearview mirror 520 displaying the rear view of the area surrounding the vehicle. The rearview mirror 520 may be an electronic mirror. Depending on various aspects, the rearview mirror 520 may be similar to... Figure 1 and Figure 3 The rearview mirror is 120c. (Combination reference) Figure 1 , Figure 2 and Figure 5A The ECU 106 can send real-time video data, including a rear view field 208, received from an onboard camera (e.g., a rear-facing camera 116a) to the rearview mirror 520. In this way, the rearview mirror 520 can display the rear view field of the area surrounding the vehicle. The rear view field includes lane markings 511 (e.g., solid or dashed lines) that define the boundaries of one or more traffic lanes. The ECU 106 can generate lane identifiers (e.g., hash symbols, patterns, and / or colors) indicating the vehicle's current traffic lane. The ECU 106 can send the lane identifiers to a display so that the lane identifiers overlaid on the rear view field are displayed via the display, for example, as shown in the image. Figure 5B , Figure 5C , Figure 5Dand / or Figure 5E As shown in any of them.

[0055] Figure 5B A rearview mirror 520 is illustrated, displaying a lane identifier 521 that shows a rear field of view covering the area surrounding the vehicle. The lane identifier 521 may include one or more numeric markings highlighting the boundaries of traffic lanes. In the illustrated embodiment, the lane identifier 521 includes multiple numeric markings in the form of lines and / or colors highlighting lane markings 511 on the road. The lane identifier 521 may highlight one or more lane markings 511 on the road to allow the driver to quickly determine their own or current traffic lane 522. In this way, the lane identifier 521 may include numeric markings of lane markings 511 highlighting the vehicle's current traffic lane 522.

[0056] Figure 5C A rearview mirror 520 is illustrated that displays lane identifiers in the rear field of view covering the area surrounding the vehicle. Lane identifiers may include one or more numerical markers highlighting the boundaries of traffic lanes. In the illustrated embodiment, the lane identifiers include a first zone 531 and a second zone 532, the first zone 531 including the vehicle's current lane 522. At least one of the pattern, color, and / or brightness of the second zone 532 is adjusted relative to the first zone 531. For example, the second zone 532 (which may include the remainder of the field of view outside the first zone 531) may be darkened, its color adjusted (e.g., by biasing the color towards a predetermined base color, such as, for example, light blue), and other adjustments may be made. In this way, the vehicle's current traffic lane 522 can be highlighted or emphasized relative to the rest of the field of view to allow the driver to quickly identify their own traffic lane separated from adjacent traffic lanes.

[0057] In the example, ECU 106 can dim the second zone 532 relative to the first zone 531 and / or increase the brightness of the first zone 531 relative to the second zone 532, so that the driver can quickly and clearly identify the vehicle's current traffic lane.

[0058] Figure 5DA rearview mirror 520 is illustrated that displays lane identifiers in the rear field of view covering the area surrounding the vehicle. Lane identifiers may include one or more numerical markers highlighting the boundaries of traffic lanes. In the illustrated embodiment, the lane identifiers include multiple zones, each covering a single lane and assigned a unique color. For example, a first zone 541, including the vehicle's current traffic lane 522, can be displayed without color adjustment. In this way, the vehicle's current traffic lane 522 can be displayed as it normally would. A second zone 542, including an adjacent traffic lane 523, can be assigned a first color (e.g., yellow or any other suitable color) that covers the first adjacent lane 523. In other words, the first adjacent traffic lane 523 can be highlighted with the first color. A third zone 543, including a second adjacent traffic lane 524, can be assigned a second color (e.g., blue or any other suitable color) that covers the second adjacent traffic lane 524. In other words, the second adjacent traffic lane 524 can be highlighted with the second color. Any additional traffic lanes detected by ECU 106 can be assigned a corresponding zone. For example, the fourth zone 544, which includes the third traffic lane 525, and the fifth zone 545, which includes the fourth traffic lane 526, can each be assigned to cover the color of the corresponding traffic lane.

[0059] In various aspects, ECU 106 can assign colors to each traffic lane 522, 523, 524, 525, 526, so that ECU 106 uses the same color to highlight each corresponding traffic lane 522, 523, 524, 525, 526, regardless of which traffic lane the vehicle is currently in. For example, combined reference. Figure 5D and Figure 5E The vehicle can perform lane change maneuvers to move into lane 523. Figure 5EThe rear view of the vehicle after it has moved into lane 523 is shown. In response to ECU 106 detecting that the vehicle has moved into lane 523, ECU 106 can disable the highlighting of the vehicle's current traffic lane 523 and activate the highlighting of the vehicle's previous traffic lane 522. The previous traffic lane 522 (which was not previously highlighted) can be highlighted in a predetermined color, such as, for example, orange. If the vehicle moves back into traffic lane 522, then ECU 106 can disable the highlighting of traffic lane 522 and reactivate the highlighting of traffic lane 523 to highlight traffic lane 523 in a first color, such as, for example, yellow. Accordingly, ECU 106 can store the traffic lane color in memory 108. In this way, ECU 106 can generate one or more lane identifiers by processing the lane identifier data stored in memory 108. ECU 106 can send the lane identifier data (or processed lane identifier data) to a display (e.g., rearview mirror 520) so that one or more lane identifiers are displayed on the display. Maintaining the color of traffic lanes can reduce or eliminate driver confusion.

[0060] In various aspects, ECU 106 can use feature detection, pattern matching, image detection, estimation using camera packing parameters, machine learning, and / or any other suitable techniques to identify traffic lanes, horizons, and any other suitable features in real-time video data. Example feature detection techniques include Harris corner detection, Shi-Tomasi corner detection, Canny edge detection, blob detection, and Scale Invariant Feature Transform (SIFT). ECU 106 can detect lane markings 511 in image data to define one or more traffic lanes. In this way, ECU 106 can overlay digital signs at the correct location and / or area within the rear field of view of the area surrounding the vehicle. As the vehicle moves along the road, changes lanes, and / or changes direction of travel, ECU 106 can iteratively adjust and / or update the digital signs.

[0061] In various aspects, ECU 106 can use GPS data and / or map information (e.g., HD MAP) to help identify lane markings. For example, HD MAP information can provide information about the number of lanes on the road for vehicle 102.

[0062] Combination Reference Figure 1 , Figure 3 and Figures 5A-5EECU 106 can be configured to determine and / or predict that vehicle 102 is performing a lane change maneuver. ECU 106 can be configured to determine and / or predict the lane change maneuver using sensor feedback indicative of one or more conditions of vehicle 102. One or more conditions may include the current shift position, current vehicle speed, current steering signal condition, current steering angle, and / or current camera condition. In this example, ECU 106 may determine or receive one or more conditions of vehicle 102 via one or more sensors 118.

[0063] In various aspects, ECU 106 determines and / or predicts lane change maneuvers by detecting the movement of lane sign 511 relative to vehicle 102 using real-time video data received from rear-facing camera 116a. For example, when lane sign 511 moves horizontally across the display, ECU 106 can detect that vehicle 102 is moving laterally or across the lane. Similarly, ECU 106 can use real-time video data received from front-facing camera (e.g., lane departure warning) to determine and / or predict lane change maneuvers.

[0064] In all aspects, the ECU 106 determines and / or predicts lane change maneuvers by detecting the activation of the steering signal from the vehicle 102. The ECU 106 can receive the steering signal status from the steering signal sensor 118c.

[0065] In all aspects, ECU 106 uses steering angle sensor 118d to determine and / or predict lane change maneuvers. ECU 106 can determine and / or predict lane change maneuvers in response to steering angle sensor 118d indicating that steering wheel 306 is at a steering angle indicating a lane change.

[0066] ECU 106 may use any (one or more) suitable methods to determine and / or predict lane change maneuvers, and this disclosure is not intended to be particularly limited in this respect.

[0067] Figure 6 This is a flowchart of an example process 600 for controlling a traffic lane recognition system. One or more computers or one or more data processing devices (e.g., Figure 1 The ECU 106 of the traffic lane recognition system can implement process 600 after proper programming. For ease of description, the following refers to... Figure 1-5E The process 600 is described. However, the process 600 of this disclosure is not limited to using... Figure 1-5E An example lane recognition system.

[0068] The traffic lane recognition system can receive real-time video data (602) from one or more cameras 116. The real-time video data may include a rear field of view 208.

[0069] The traffic lane recognition system can generate a lane identifier (604) indicating the current traffic lane of vehicle 102. The lane identifier may include a sign highlighting the lane markings of the vehicle's current traffic lane, such as... Figure 5B As described. Lane identifiers may include a first zone and a second zone containing the current traffic lane of a vehicle, wherein at least one of the colors or brightness of the second zone is adjusted relative to the first zone, for example, as described above. Figure 5C As described. Lane identifiers can include multiple colors, each color being assigned to a corresponding traffic lane among multiple traffic lanes identified in the field of view, such as those described above. Figure 5D and Figure 5E As described.

[0070] The traffic lane recognition system can send lane identifiers to a display (e.g., rearview mirror 120c) so that the lane identifiers (606) covering the field of view can be displayed on the display. For example, ECU 106 can send lane identifiers to rearview mirror 120c so that, Figure 5B-5E The example illustrations (including combinations thereof) show lane identifiers overlaid within the field of view.

[0071] In various aspects, the traffic lane recognition system can determine that vehicle 102 is performing a lane change maneuver. In response to detecting a lane change maneuver, the traffic lane recognition system can temporarily disable the display of lane identifiers when vehicle 102 performs a lane change (e.g., see...). Figure 5A To avoid confusing the driver about the lane position, in response to detecting that vehicle 102 has completed a lane change maneuver, the traffic lane recognition system can generate an updated lane identifier indicating the vehicle's new current traffic lane. The traffic lane recognition system can send the updated lane identifier to the rearview mirror 120c so that the updated lane identifier is displayed overlaid in the real-time field of view via the rearview mirror 120c.

[0072] Figure 7 This is a flowchart of an example process 700 for controlling a traffic lane recognition system. One or more computers or one or more data processing devices (e.g., Figure 1 The ECU 106 of the traffic lane recognition system can implement process 700 after proper programming. For ease of description, the following refers to... Figure 1-5E The process 700 is described. However, the process 700 of this disclosure is not limited to using... Figure 1-5E An exemplary lane recognition system. Various aspects of process 700 can be integrated with... Figure 6 The process is similar to 600.

[0073] The traffic lane recognition system can receive real-time video data (602) from one or more cameras 116. The real-time video data may include a rear field of view 208.

[0074] The traffic lane recognition system can assign a first color to the vehicle's first (current) traffic lane and a second color to the vehicle's second (adjacent) traffic lane (703). For example, temporary combination reference. Figure 1 and Figure 5D ECU 106 can assign a first color (e.g., orange) to traffic lane 522. ECU 106 can assign a second color (e.g., yellow) to adjacent traffic lane 523.

[0075] The traffic lane recognition system can generate a lane identifier (704a) indicating the current traffic lane of vehicle 102. The lane identifier can include multiple colors, including a second color assigned in step 703, each color being assigned to a corresponding traffic lane among multiple traffic lanes identified in the field of view, for example, regarding... Figure 5D and Figure 5E As described.

[0076] The traffic lane recognition system can send lane identifiers to a display (e.g., rearview mirror 120c) for display of the lane identifiers (706a) covering the field of view. The lane identifiers include highlighting traffic lane 523 with a second color. Highlighting of traffic lane 522 can be omitted. In other words, for the current traffic lane 522 of vehicle 102, the first color can be omitted (see...). Figure 5D ).

[0077] The traffic lane recognition system can determine that vehicle 102 has moved from the first traffic lane to the second traffic lane (708). For example, ECU 106 can detect that vehicle 102 has moved from traffic lane 522 (see...). Figure 5D Move to traffic lane 523 (see...) Figure 5E ).

[0078] The traffic lane recognition system can generate an updated lane identifier (704b) indicating the updated current traffic lane for vehicle 102. Step 704b can be executed in response to ECU 106 detecting a lane change. The lane identifier can include multiple colors, including the first color assigned in step 703, each color being assigned to a corresponding traffic lane among multiple traffic lanes identified in the field of view, for example, regarding... Figure 5D and Figure 5E As described.

[0079] The traffic lane recognition system can send updated lane identifiers to a display (e.g., rearview mirror 120c) for display of the updated lane identifiers (706b) covering the field of view. The updated lane identifiers include highlighting traffic lane 522 with a first color. Highlighting lane 523 may be omitted from the updated lane identifiers. In other words, for the current traffic lane 523 of vehicle 102, the second color may be omitted (see [link to relevant documentation]). Figure 5E ).

[0080] Exemplary embodiments of this disclosure have been disclosed in an illustrative manner. Accordingly, the terminology used throughout should be understood in a non-limiting manner. While those skilled in the art may make minor modifications to the teachings herein, it should be understood that all such embodiments that reasonably fall within the scope of the technical advancements contributed herein should be limited to the scope of this patent grant, and that scope should not be limited except in accordance with the appended claims and their equivalents.

Claims

1. A traffic lane recognition system for assisting a vehicle driver, comprising: A display located on or inside a vehicle; as well as Electronic control unit (ECU), the ECU being coupled to a display and configured to: Real-time video data is sent to a display so that the field of view of the area surrounding the vehicle can be displayed on the display. Generate a lane identifier that indicates the vehicle's current traffic lane; Send lane identifiers to the display so that the lane identifiers covered by the field of view can be displayed on the display.

2. The traffic lane recognition system of claim 1 further includes a camera coupled to the ECU and configured to capture real-time video data of the field of view of the area surrounding the vehicle.

3. The traffic lane recognition system of claim 2, wherein the field of view of the area surrounding the vehicle includes a field of view relative to the rear of the vehicle.

4. The traffic lane recognition system of claim 3, wherein the display includes a digital rearview mirror configured to display the rear view and / or lane identifier.

5. The traffic lane recognition system of claim 1, wherein the lane identifier includes a sign that highlights the current traffic lane of the vehicle.

6. The traffic lane recognition system of claim 1, wherein the lane identifier includes a first zone and a second zone, the first zone including the vehicle's current traffic lane.

7. The traffic lane recognition system of claim 6, wherein at least one of the color or brightness of the second zone is adjusted relative to the first zone.

8. The traffic lane recognition system of claim 1, wherein the lane identifier includes multiple colors, each of the multiple colors being assigned to a corresponding traffic lane among multiple traffic lanes identified in the field of view.

9. The traffic lane recognition system of claim 1, wherein the ECU is further configured to: Confirm that the vehicle is performing a lane change maneuver; Generate an updated lane identifier that indicates the vehicle's new current traffic lane; The updated lane identifier is sent to the display so that the updated lane identifier, which is overlaid on the field of view, can be displayed on the display.

10. The traffic lane recognition system of claim 9, wherein the ECU is further configured to turn off the display of the lane identifier during lane change maneuvers.

11. A traffic lane recognition system for assisting a driver of a vehicle, comprising: A camera configured to capture real-time video data of the field of view of the area surrounding the vehicle; A display located on or inside a vehicle; as well as Electronic control unit (ECU), which is coupled to the display and camera and configured to: Real-time video data is sent to a display so that the field of view of the area surrounding the vehicle can be displayed on the display. Generate a lane identifier that indicates the vehicle's current traffic lane; Send lane identifiers to the display so that the lane identifiers covered by the field of view can be displayed on the display.

12. The traffic lane recognition system as claimed in claim 11, wherein: The field of view around the vehicle includes the field of view relative to the rear of the vehicle; as well as The display includes a digital rearview mirror configured to display the rear view and / or lane identifiers.

13. The traffic lane recognition system of claim 11, wherein the lane identifier includes a sign that highlights the lane marking of the vehicle's current traffic lane.

14. The traffic lane recognition system as claimed in claim 11, wherein: Lane identifiers include a first zone and a second zone, where the first zone includes the vehicle's current traffic lane; and At least one of the colors or brightness in the second zone is adjusted relative to the first zone.

15. The traffic lane recognition system of claim 11, wherein the lane identifier comprises a plurality of colors, each of the plurality of colors being assigned to a corresponding traffic lane among a plurality of traffic lanes identified in the field of view.

16. The traffic lane recognition system of claim 11, wherein the ECU is further configured to: Confirm that the vehicle is performing a lane change maneuver; Turn off the display of lane identifiers during lane change maneuvers; Confirm that the vehicle has completed the lane change maneuver; Generate an updated lane identifier that indicates the vehicle's new current traffic lane; as well as The updated lane identifier is sent to the display so that the updated lane identifier, which is overlaid on the field of view, can be displayed on the display.

17. A method for assisting a driver of a vehicle, comprising: The vehicle's electronic control unit (ECU) sends real-time video data to the display so that the field of view of the area surrounding the vehicle can be displayed on the display. The ECU generates a lane identifier that indicates the vehicle's current traffic lane. as well as The lane identifier is sent to the display via the ECU so that the lane identifier covered by the field of view can be displayed on the display.

18. The method of claim 17, wherein: The field of view around the vehicle includes the field of view relative to the rear of the vehicle; as well as The display includes a digital rearview mirror configured to display the rear view and / or lane identifiers.

19. The method of claim 17, wherein the lane identifier comprises at least one of the following: A sign that highlights the lane markings of the vehicle's current traffic lane; A first zone and a second zone, wherein the first zone includes the vehicle's current traffic lane, and at least one of the colors or brightness of the second zone is adjusted relative to the first zone; or Multiple colors, each of which is assigned to a corresponding traffic lane among multiple traffic lanes identified in the field of view.

20. The method of claim 17, further comprising: The ECU determines that the vehicle is performing a lane change maneuver; The lane identifier display is turned off via the ECU during lane change maneuvers; The ECU confirms that the vehicle has completed the lane change maneuver. The ECU generates an updated lane identifier that indicates the vehicle's new current traffic lane. as well as The updated lane identifier is sent to the display via the ECU so that the updated lane identifier is displayed on the display and overlaid on the field of view.

Images