Visual auxiliary lane changing method and device and vehicle

Abstract

The invention discloses a visual auxiliary lane changing method and device and a vehicle, and belongs to the technical field of vehicles. The visual auxiliary lane changing method comprises the following steps: determining a lane changing direction in response to a received lane changing trigger of an auxiliary lane changing system, and obtaining a lane condition related to the lane changing direction; displaying the lane condition in a display area which is in front of the view of the driver and matched with the lane changing direction; and generating and setting a frame matched with the lane condition for the display area, and dynamically adjusting the frame according to the change of the lane condition. According to the method, the lane condition related to lane changing can be visually provided for a driver in real time, and the display area can be dynamically changed along with the change of the lane condition, so that the experience of the driver on auxiliary lane changing is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a visual assisted lane changing method, device and vehicle. Background Art

[0002] At present, during the process of assisted lane change, the vehicle either only provides the lane change path for the vehicle, or prompts through the central control display screen when there is a risk of collision between the vehicle and surrounding obstacles (such as surrounding vehicles, curbs, etc.). In either case, the driver is required to observe the lane conditions through the front windshield, rearview mirror, etc. at all times, and cannot provide the driver with reliable and clear lane conditions in time, resulting in a poor experience of assisted lane change for the driver. Summary of the invention

[0003] In view of this, the present invention provides a visual assisted lane change method, device and vehicle, which can provide the driver with real-time visualization of lane conditions related to lane change, and can dynamically change the display area as the lane conditions change, so as to effectively enhance the driver's experience of assisted lane change.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0005] In a first aspect, the present invention provides a visualization assisted lane change method, comprising:

[0006] In response to receiving a lane change trigger of the lane change assistance system, determining a lane change direction, and acquiring a lane condition related to the lane change direction;

[0007] Displaying the lane condition in a display area in front of the driver's field of vision and matching the lane change direction;

[0008] A frame matching the lane condition is generated and set for the display area, and the frame is dynamically adjusted according to changes in the lane condition.

[0009] Optionally, the acquiring a lane condition related to the lane change direction includes:

[0010] When the lane change direction indicates a left lane change, acquiring a first environment image within a first set range of a left lane associated with the left lane change and a second environment image within a second set range of a current lane where the vehicle is located;

[0011] When the lane change direction indicates a right lane change, a third environment image within a third setting range of a right lane associated with the right lane change and a second environment image within a second setting range of a current lane where the vehicle is located are acquired.

[0012] Optionally, acquiring a second environment image within a second set range of a current lane where the vehicle is located includes:

[0013] Acquire a front environment image in front of the vehicle and a rear environment image behind the vehicle in the current lane.

[0014] Optionally, the above-mentioned visualization assisted lane changing method further includes:

[0015] The vehicle's driving status is displayed in a specific display area in front of the driver's field of vision.

[0016] Optionally, the display area in front of the driver's field of vision and matching the lane change direction displays the lane condition, including:

[0017] When the lane change direction indicates a left lane change, splicing the first environment image and the second environment image, and displaying the spliced ​​image in a left area of ​​the specific display area;

[0018] or,

[0019] When the lane change direction indicates a right lane change, the third environment image and the second environment image are spliced ​​together, and the spliced ​​image is displayed in a right area of ​​the specific display area.

[0020] Optionally, generating a border matching the lane condition for the display area includes:

[0021] The lane change assistance system identifies the lane condition and generates a lane change indication signal matching the lane condition according to the identification result;

[0022] A frame matching the lane change indication signal is generated and set.

[0023] Optionally, generating a frame matching the lane change indication signal includes:

[0024] In response to the lane change indication signal indicating a normal lane change, generating and configuring a frame of a first color for the display area;

[0025] In response to the lane change indication signal indicating that traffic is complex in the lane change area, generating and configuring a border of a second color for the display area;

[0026] In response to the situation in which the lane change indication signal indicates that the lane change area is occupied, a border of a third color is generated and configured for the display area.

[0027] Optionally, the above-mentioned visualization assisted lane changing method further includes:

[0028] In response to receiving a shutdown signal or an unavailable signal of the lane change assistance system, the display of the lane condition is stopped.

[0029] In a second aspect, an embodiment of the present invention provides a visual lane-changing assistance device, including: a processing unit and a display unit, wherein:

[0030] The processing unit determines a lane change direction in response to receiving a lane change trigger of the lane change assistance system, and obtains a lane condition related to the lane change direction;

[0031] The display unit is used to display the lane condition in a display area in front of the driver's field of view and matching the lane change direction; generate and set a border matching the lane condition for the display area, and dynamically adjust the border according to changes in the lane condition.

[0032] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0033] one or more processors;

[0034] a storage device for storing one or more programs,

[0035] When the one or more programs are executed by the one or more processors, the one or more processors implement the visualization assisted lane changing method provided in the above-mentioned first aspect embodiment.

[0036] In a fourth aspect, an embodiment of the present invention provides a computer-readable medium having a computer program for implementing a visualization-assisted lane change method stored thereon.

[0037] When the computer program is executed by the on-board processor, the visualization assisted lane changing method provided in the above-mentioned first aspect embodiment is implemented.

[0038] In a fifth aspect, an embodiment of the present invention provides a vehicle, which implements the visualization assisted lane changing method provided by the embodiment of the first aspect or includes the visualization assisted lane changing device provided by the embodiment of the second aspect.

[0039] The technical solution of the above invention has the following advantages or beneficial effects:

[0040] The technical solution provided by the embodiment of the present invention determines the lane change direction after receiving the lane change trigger of the lane change assistance system, obtains the lane condition related to the lane change direction, and displays the lane condition in a display area in front of the driver's field of view and matching the lane change direction, so as to provide the driver with the lane condition of the lane change assistance in a timely manner, so that the driver can control the vehicle or adjust the lane change assistance system in a timely manner according to the lane condition. In addition, the technical solution generates and sets a frame matching the lane condition for the display area, and dynamically adjusts the frame according to the change of the lane condition, so that the driver can pay attention to the lane condition and the change of the lane condition in a timely manner according to the display area frame and the change of the frame, so as to further improve the experience of the lane change assistance and the driving experience of the driver. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0042] Figure 2 is a schematic diagram of the main process of the visual assisted lane changing method provided according to an embodiment of the present invention;

[0043] Figure 3 is a schematic diagram of a first display state of a front windshield of a vehicle changing lanes according to an embodiment of the present invention;

[0044] Figure 4 is a schematic diagram of a second display state of a front windshield of a vehicle changing lanes according to an embodiment of the present invention;

[0045] Figure 5 is a schematic diagram of a first display state of a front windshield for a vehicle changing lanes on the left side provided by an embodiment of the present invention;

[0046] Figure 6 is a schematic diagram of a second display state of a front windshield for a vehicle changing lanes on the left side provided in an embodiment of the present invention;

[0047] Figure 7 is a schematic diagram of a third display state of a front windshield for a vehicle changing lanes on the left side provided in an embodiment of the present invention;

[0048] Figure 8 is a schematic diagram of a first display state of a front windshield for a vehicle changing lanes on the right side provided according to an embodiment of the present invention;

[0049] Fig. 9 is a schematic diagram of a second display state of a front windshield for a vehicle changing lanes on the right side provided according to an embodiment of the present invention;

[0050] Fig.10is a schematic diagram of a third display state of a front windshield for a right lane change of a vehicle provided according to an embodiment of the present invention;

[0051] Fig.11 is a schematic diagram of a fourth display state of a front windshield for a vehicle changing lanes on the left side provided according to an embodiment of the present invention;

[0052] Fig.12 is a schematic diagram of a fifth display state of a front windshield for a vehicle changing lanes on the left side provided according to an embodiment of the present invention;

[0053] Fig.13 is a schematic diagram of a fourth display state of a front windshield for a right lane change of a vehicle provided according to an embodiment of the present invention;

[0054] Fig.14 is a schematic diagram of a fifth display state of a front windshield for a right lane change of a vehicle provided according to an embodiment of the present invention;

[0055] Fig.15 is a schematic diagram of a third display state of a front windshield of a vehicle changing lanes according to an embodiment of the present invention;

[0056] Fig.16 is a schematic diagram of the main process of a visualization-assisted lane-changing method provided according to another embodiment of the present invention;

[0057] Fig.17 is a partial structural schematic diagram of a visual lane-changing assistance device provided according to an embodiment of the present invention;

[0058] Fig.18 is a partial structural schematic diagram of a visual assisted lane change system provided according to an embodiment of the present invention;

[0059] Fig.19 It is a structural diagram of a computer system suitable for implementing a visual assisted lane change according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0061] It should be pointed out that the embodiments of the present invention and the technical features therein may be combined with each other without conflict.

[0062] In addition, the "first", "second" and "third" involved in the embodiments of the present invention are to distinguish the features of different ranges or purposes, and they are not a restriction on the number or order of features. Exemplarily, the first environment image, the second environment image and the third environment image are to distinguish the environment images corresponding to different areas, such as the first environment image corresponds to the area related to the left lane change, the second environment image corresponds to the current lane where the vehicle is located, and the third environment image corresponds to the area related to the right lane change, etc. For another example, the first set range corresponds to the left lane, the second set range corresponds to the current lane where the vehicle is located, and the third set range corresponds to the right lane, etc. For another example, the first color, the second color and the third color are to distinguish different colors, and they are not a restriction on the number of colors or the order of colors. The vehicle involved in the embodiments of the present invention can be an internal combustion engine vehicle using an engine as a power source, a hybrid vehicle using an engine and an electric motor as a power source, an electric vehicle using an electric motor as a power source, etc.

[0063] The following describes a technical scenario to which the technical solution provided by the embodiment of the present invention is applicable based on the system architecture on which the technical solution provided by the embodiment of the present invention relies.

[0064] Figure 1 An exemplary system architecture 100 is shown, on which a visualization-assisted lane-changing method or a visualization-assisted lane-changing device according to an embodiment of the present invention may be applied.

[0065] like Figure 1 As shown, the vehicle system architecture 100 may include various systems, such as a driving control system 101, a power system 102, a sensor system 103, a control system 104, an auxiliary lane change system 105, one or more peripheral devices 106, a power supply 107, a computer system 108 and a user interface 109, wherein the visual assisted lane change method provided in the embodiment of the present invention may be implemented by interacting with the above-mentioned various systems. Optionally, the vehicle system architecture 100 may include more or fewer systems, and each system may include multiple components. In addition, each system and component of the vehicle system architecture 100 may be interconnected by wire or wirelessly.

[0066] The vehicle system architecture 100 includes a driving control system 101, which can be in a full or partial automatic driving mode. For example, the driving control system 101 can automatically control the vehicle to change lanes according to a lane change signal or lane change instruction provided by the lane change assistance system 105 without human interaction.

[0067] The power system 102 may include components that provide power movement for the vehicle. For example, the power system 102 may include an engine, an energy source, a transmission, wheels, tires, etc. Among them, the engine may be an internal combustion engine, an electric motor, an air compression engine, or a combination of other types of engines, such as a hybrid engine consisting of a gas-oil engine and an electric motor, and a hybrid engine consisting of an internal combustion engine and an air compression engine. The engine converts the energy source into mechanical energy and provides it to the transmission. Examples of energy sources may include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. The energy source may also provide energy for other systems of the vehicle. In addition, the transmission may include a gearbox, a differential, a drive shaft, a clutch, etc.

[0068] The sensor system 103 may include sensors for sensing the surrounding environment of the vehicle (such as sensors for sensing whether there are obstacles around) and pressure sensors for sensing whether there are passengers in the seats. For example, a positioning system (the positioning system may be a global positioning system (GPS) system, or a Beidou system or other positioning systems), a radar, a laser rangefinder, an inertial measurement unit (IMU), a camera, and a capacitive sensor. The positioning system may be used to locate the geographic location of the vehicle. The IMU is used to sense the position and orientation changes of the vehicle based on inertial acceleration. In one embodiment, the IMU may be a combination of an accelerometer and a gyroscope. The radar may use radio signals to sense objects in the surrounding environment of the vehicle. The capacitive sensor is used to locate the metal wheel stopper on the parking space. In some embodiments, in addition to sensing objects, the radar may also be used to sense the speed and / or direction of travel of the object.

[0069] In order to detect environmental information, objects, etc. outside the vehicle, a camera or the like may be configured at an appropriate location outside the vehicle. For example, in order to obtain an environmental image of the side of the vehicle, the camera may be on a rearview mirror on the side of the vehicle. The camera may be a static or video camera.

[0070] The control system 104 may include a software system for implementing vehicle driving control, such as a system for analyzing the vehicle's surrounding environment, a system for pre-tensioning seat belts, a system for route planning, a system for avoiding obstacles, a visual system for image analysis, etc. The control system 104 may also include hardware systems such as a throttle, a steering wheel system, a seat belt system, an airbag system, and peripheral devices (such as a projection device, a display, etc.). In addition, the control system 104 may include components other than those shown and described in addition or in substitution. Alternatively, some of the components shown above may be reduced.

[0071] Furthermore, as described above, the control system 1204 can further implement a portion of the above-mentioned visual assisted lane change method, determine the lane change direction, and obtain the lane condition related to the lane change direction, display the lane condition in a display area in front of the driver's field of view and matching the lane change direction, generate and set a border for the display area that matches the lane condition, and dynamically adjust the border according to changes in the lane condition.

[0072] In addition, the control system 104 may also interact with external sensors, other autonomous driving devices, other computer systems, or users through the peripheral devices 106. The peripheral devices 106 may include wireless communication systems, onboard computers, microphones and / or speakers, cameras, and projectors.

[0073] After receiving the trigger, the lane change assistance system 105 assists the power system 102 and the control system 104 to control the vehicle to change lanes. In addition, the lane change assistance system 105 interacts with the control system 104 to provide the control system 104 with a lane change trigger, so that the control system 104 starts executing a program related to a visual lane change assistance method based on the lane change trigger. Through the program of the visual lane change assistance method, the following are executed: determining the lane change direction and obtaining the lane condition related to the lane change direction; displaying the lane condition in a display area in front of the driver's field of view and matching the lane change direction; generating and setting a border matching the lane condition for the display area, and dynamically adjusting the border according to changes in the lane condition.

[0074] In some embodiments, the peripheral device 106 provides a means for a user of the control system 104 to interact with the user interface. For example, an onboard computer can provide information to a user of the vehicle. The user interface can also operate the onboard computer to receive input from the user. The onboard computer can be operated via a touch screen. In other cases, the peripheral device can provide a means for communicating with other devices located in the vehicle. For example, a microphone can receive audio (e.g., voice commands or other audio input) from a user of the control system 104. Similarly, a speaker can output audio to the user of the control system 104.

[0075] The wireless communication system can communicate with one or more devices wirelessly directly or via a communication network. For example, the wireless communication system can communicate with a network such as a cellular network, WiFi, and a wireless local area network (WLAN), or can communicate directly with a device using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various autonomous driving communication systems, etc.

[0076] The power source 107 can provide power to various components of the vehicle. The power source 107 can be a rechargeable lithium-ion or lead-acid battery.

[0077] Part or all of the functions of implementing the visual assisted lane change are controlled by the computer system 108. The computer system 108 may include at least one processor that executes instructions stored in a non-transitory computer-readable medium such as a memory. The computer system 108 provides the above control system with an execution code for implementing vehicle safety protection.

[0078] The processor can be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor can be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Those of ordinary skill in the art will appreciate that the processor, computer, or memory can actually include multiple processors, computers, or memories that may or may not be stored in the same physical housing. For example, the memory can be a hard disk drive or other storage media that is located in a housing different from the computer. Therefore, references to processors or computers will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Different from using a single processor to perform the steps described herein, some components such as steering components and deceleration components can each have their own processors that only perform calculations related to the functions specific to the components.

[0079] The user interface 109 is used to provide information to or receive information from a user of the vehicle. Optionally, the user interface 109 may include one or more input / output devices within the set of peripheral devices 106, such as a wireless communication system, an onboard computer, a microphone, and a speaker.

[0080] It should be understood that the above components are only examples. In actual applications, the components in the above modules or systems may be added or deleted according to actual needs. Figure 1 It should not be understood as limiting the embodiments of the present application.

[0081] Figure 2 is a schematic diagram of the main steps of a visualization-assisted lane-changing method according to an embodiment of the present invention. Specifically, Figure 2 As shown, the visualization assisted lane changing method mainly includes the following steps:

[0082] Step S201: Acquire lane conditions: In response to receiving a lane change trigger from the lane change assistance system, determine the lane change direction, and acquire lane conditions related to the lane change direction.

[0083] The lane change assistance system may be a system currently existing on the vehicle or a system newly added to the vehicle based on the visual lane change assistance method provided by the embodiment of the present invention. The lane change assistance system can assist the driver in controlling the lane change of the vehicle, wherein the lane change may be a lane change scenario during the vehicle's straight-ahead process, or a lane change scenario during the vehicle's entry into a ramp or exiting a ramp, etc.

[0084] Among them, receiving a lane change trigger of the assisted lane change system means that after the vehicle's assisted lane change system is started or activated, it automatically enters the lane change program or receives the driver's control to enter the lane change program, and generates a signal indicating lane change. After receiving the signal indicating lane change, the software program that implements the technical solution provided by the embodiment of the present invention activates or starts the execution logic of the software program that implements the technical solution provided by the embodiment of the present invention.

[0085] Among them, the lane change direction can be obtained from the vehicle's control system or the vehicle's power system (such as wheel steering, steering wheel steering) or the vehicle's peripheral equipment (such as the situation where both turn signals are off, etc.). The lane change direction can be from the current lane where the vehicle is located to the left lane to the left of the current lane, or from the current lane where the vehicle is located to the right lane to the right of the current lane.

[0086] The lane condition may be an image or picture related to the lane changing direction, or a relative position relationship change diagram generated for the vehicle based on the position of obstacles related to the lane changing direction identified by the peripheral equipment or sensor system of the vehicle. Preferably, the lane condition is an image or picture related to the lane changing direction. Specifically, for the case where the lane changing direction is from the current lane to the left lane, the image or picture related to the lane changing direction may be a complete image or picture of the left lane (the complete image or picture refers to the image or picture of the left area corresponding to the vehicle, the image or picture of the left rear side of the vehicle, and the image or picture of the left front side of the vehicle), or may be an image or picture that matches the range that can be observed by the left rearview mirror of the vehicle. In addition, the displayed range of the image or picture related to the lane changing direction may also be adjusted according to the driver's control, such as only the image or picture within the set range of the left rear side of the vehicle.

[0087] Step S202: Displaying lane conditions: Displaying lane conditions in a display area in front of the driver's field of vision and matching the lane change direction.

[0088] Among them, the display area in front of the driver's field of vision can be on the front windshield, or it can be located outside the front windshield. The display method can be achieved through projection, a vehicle-mounted head-up display system (HUD), a transparent display screen embedded in the front windshield, etc.

[0089] In addition, in this step, a better prompt effect can be achieved by controlling the display area that matches the lane changing direction, that is, the driver can intuitively obtain the lane changing direction of the vehicle with respect to the display area, and by displaying the lane condition in the display area in front of the driver's field of view and matching the lane changing direction, the driver can know the lane condition in the lane changing direction by looking straight ahead, so as to avoid blind spots, and also avoid observing the lane condition by looking sideways, so that the driver can take into account both the condition in front of the vehicle and the lane condition in the lane changing direction at the same time, thereby improving driving and lane changing safety.

[0090] The following takes the display area located on the front windshield as an example. Before the vehicle changes lanes, Figure 3 As shown, the front windshield W does not have any display information, and can also be Figure 4 As shown, there is a display area C at the front windshield W, which is used to display the vehicle's driving information (such as vehicle speed, driving power, vehicle control status, remaining power, etc.). Figure 3 or Figure 4 Based on the above, if the vehicle changes lanes to the left, the display area L showing the lane status may be as shown in FIG. Figures 5 to 7 as well as Fig.11 and Fig.12 As shown, the display area L is located on the left side of the driver or close to the left side of the driver. It can be understood that before changing lanes, the front windshield W is Figure 4 In the state shown (i.e., with display area C), when the vehicle changes lanes to the left or is about to change lanes to the left, the display area on the front windshield W may change to Figures 5 to 7 as well as Fig.11 and Fig.12 Any of the above, before changing lanes, the front windshield W is Figure 3 In the state shown in FIG. 1 (i.e., without any display area), when the vehicle changes lanes to the left or is about to change lanes to the left, the display area on the front windshield W may also change to Figures 5 to 7 as well as Fig.11 and Fig.12 Any of .

[0091] Accordingly, in Figure 3 or Figure 4 Based on the above, if the vehicle changes lanes to the right, the display area R showing the lane status may be as shown in FIG. Figures 8 to 10 as well as Fig.13 and Fig.14 As shown, the display area R is located on the right side of the driver or close to the right side of the driver. It can be understood that before changing lanes, the front windshield W is Figure 4 If the vehicle is in the state shown (i.e., has the display area C), when the vehicle changes lanes to the right or is preparing to change lanes to the right, the display area on the front windshield W may change to Figures 8 to 10 as well as Fig.13 and Fig.14 Any of the above, before changing lanes, the front windshield W is Figure 3 In the state shown in FIG. 1 (i.e., without any display area), when the vehicle changes lanes to the right or is about to change lanes to the right, the display area on the front windshield W may also change to Figures 8 to 10 as well as Fig.13 and Fig.14 Any of .

[0092] Step S203: Display area regulation: Generate and set a border for the display area that matches the lane condition, and dynamically adjust the border according to changes in the lane condition.

[0093] Among them, the border adjustment can be as follows Figures 5 to 14 As shown, the color of the border can be adjusted according to the change of lane conditions, and other forms can also be changed, such as flashing, light and dark, lace shape, etc. For example, when the lane condition indicates that the traffic condition in the lane change area is good and the lane change assistance can automatically complete the lane change, the border can be as follows Figure 5 or Figure 8 or Fig.11 The border is shown in green. If the lane condition indicates that the traffic in the lane change area is complex and requires driver assistance, the border can be Figure 6 or Fig. 9 or Fig.14 If the lane condition indicates that there is a risk of collision when changing lanes, the border may be Figure 7 or Fig.10 or Fig.13 Shown is displayed in red.

[0094] In addition, when the lane condition indicates that the environment is dark or lane change is not possible or visibility is poor, the border can be Fig.15 As shown, or the front windshield W always keeps Figure 3 or Figure 4 or Fig.15 status.

[0095] against Figure 2 The provided technical solution determines the lane change direction after receiving the lane change trigger of the lane change assistance system, obtains the lane condition related to the lane change direction, and displays the lane condition in a display area in front of the driver's field of view and matching the lane change direction, so as to provide the driver with the lane condition of the lane change assistance in a timely manner, so that the driver can control the vehicle or adjust the lane change assistance system in a timely manner according to the lane condition. In addition, the technical solution generates and sets a frame matching the lane condition for the display area, and dynamically adjusts the frame according to the change of the lane condition, so that the driver can pay attention to the lane condition and the change of the lane condition in a timely manner according to the display area frame and the change of the frame, so as to further improve the experience of the lane change assistance and the driving experience of the driver.

[0096] In the embodiment of the present invention, there may be two specific implementation plans for obtaining the lane condition related to the lane change direction.

[0097] Specifically, the first specific implementation scheme may include: when the lane change direction indicates a left lane change, obtaining a first environmental image within a first setting range of a left lane associated with the left lane change and a second environmental image within a second setting range of a current lane in which the vehicle is located; when the lane change direction indicates a right lane change, obtaining a third environmental image within a third setting range of a right lane associated with the right lane change and a second environmental image within the second setting range of the current lane in which the vehicle is located.

[0098] Among them, the first setting range can be set according to actual needs, for example, the first setting range can be one or more of the field of view of the left rearview mirror of the vehicle, the left rear area of ​​the vehicle, the left projection area of ​​the vehicle, and the left front area of ​​the vehicle. The second setting range can be the front area of ​​the vehicle and / or the rear area of ​​the vehicle. The third setting range can be set according to actual needs, for example, the third setting range can be one or more of the field of view of the right rearview mirror of the vehicle, the right rear area of ​​the vehicle, the right projection area of ​​the vehicle, and the right front area of ​​the vehicle.

[0099] Among them, a specific method of obtaining the second environment image within the second set range of the current lane where the vehicle is located may include: obtaining a front environment image in front of the vehicle and a rear environment image behind the vehicle in the current lane.

[0100] During the vehicle's lane changing process, in addition to displaying the environmental image related to the lane changing direction (the first environmental image within the first setting range of the left lane related to the left lane change or the third environmental image within the third setting range of the right lane related to the right lane change), a second environmental image within the second setting range of the vehicle's current lane is also obtained to better present the environment around the lane change to the driver, taking into account the lane conditions behind and in front of the vehicle, avoiding the driver only paying attention to the display area and ignoring the road conditions in front and behind the vehicle, so as to improve the safety of lane changing.

[0101] In addition, by simultaneously displaying an environmental image related to the lane change direction and a second environmental image within a second set range of the vehicle's current lane, the driver can take into account the environment around the vehicle through the display area, making it more convenient for the driver to change lanes.

[0102] The above-mentioned environmental images may be obtained by a vehicle-mounted camera installed on the vehicle, or the vehicle may communicate with roadside equipment to obtain the above-mentioned environmental images from images taken by roadside cameras.

[0103] A second specific implementation scheme may include: monitoring obstacles related to the lane change direction, obstacles in front of the vehicle and obstacles behind the vehicle through a vehicle-mounted radar, and then generating a relative position relationship diagram based on the relative position relationship between the obstacles and the vehicle monitored by the radar, and displaying the relative position relationship diagram in the above-mentioned display area.

[0104] Furthermore, in an embodiment of the present invention, the above-mentioned visual assisted lane change method may further include: displaying the vehicle driving status in a specific display area in front of the driver's field of vision. The specific display area can be set as required. Figure 4 , Figures 11 to 15 The specific display area C shown is where the vehicle driving status is displayed.

[0105] Wherein, in the case where the vehicle driving status is displayed in a specific display area in front of the driver's field of vision, displaying the lane status in the display area in front of the driver's field of vision and matching the lane change direction may also include: when the lane change direction indicates a left lane change, splicing the first environment image and the second environment image, and displaying the spliced ​​image in the left area of ​​the specific display area; when the lane change direction indicates a right lane change, splicing the third environment image and the second environment image, and displaying the spliced ​​image in the right area of ​​the specific display area. Wherein, the splicing of the first environment image and the second environment image or the splicing of the third environment image and the second environment image can be completed by using existing image splicing technology, which will not be repeated here.

[0106] For example, for Figure 4 As shown in FIG. 1 , a specific display area C is provided on the front windshield to indicate a left lane change direction. Fig.11 and Fig.12 As shown, the image spliced ​​by the first environment image and the second environment image is displayed in the left area of ​​the specific display area (ie Fig.11 and Fig.12 The display area L) shown in FIG. Fig.13 and Fig.14 As shown, the image spliced ​​by the third environment image and the second environment image is displayed in the right area of ​​the specific display area (ie Fig.13 and Fig.14 Display area R) shown.

[0107] Furthermore, in the above step S203, a specific implementation method of generating a border matching the lane condition for the display area may include: the auxiliary lane change system identifies the lane condition, and based on the identification result, generates a lane change indication signal matching the lane condition; and generates and sets a border matching the lane change indication signal.

[0108] The lane change indication signal can enable the existing lane change assistance system to be conveniently connected to the program for implementing the technical solution provided by the embodiment of the present invention, thereby avoiding modification of the lane change assistance system, thereby effectively improving the feasibility of the technical solution provided by the embodiment of the present invention and controlling costs.

[0109] Specifically, generating a border that matches the lane change indication signal may include: generating and configuring a border of a first color for the display area when the lane change indication signal indicates a normal lane change; generating and configuring a border of a second color for the display area when the lane change indication signal indicates that traffic in the lane change area is complex; generating and configuring a border of a third color for the display area when the lane change indication signal indicates that the lane change area is occupied. Figures 5 to 14 As shown, for the case where the lane change indication signal indicates a normal lane change, Figure 5 , Figure 8 and Fig.11 As shown in FIG. 1 , a green border is generated and configured for the display area. For the case where the lane change indicator signal indicates that the traffic in the lane change area is complex, such as Figure 6 , Fig. 9 and Fig.14 As shown in FIG. 1 , a yellow border is generated and configured for the display area. In the case where the lane change indication signal indicates that the lane change area is occupied, as shown in FIG. Figure 7 , Fig.10 , Fig.12 and Fig.13 As shown, a red border is generated and configured for the display area. The color of the border can change in real time according to the lane condition. For example, if the lane condition changes from occupied to normal lane change, the color of the border changes from yellow to green.

[0110] In addition, based on any of the above embodiments, the visual assisted lane change method may further include: in response to receiving a shutdown signal or an unavailable signal of the assisted lane change system, stopping the display of the lane condition. For example, in the case where the lane condition is displayed on the front windshield of the vehicle, after receiving a shutdown signal or an unavailable signal of the assisted lane change system, stopping the display of the lane condition, that is, the front windshield of the vehicle becomes Figure 3 or Figure 4 The status shown or remain Figure 3 or Figure 4 Status shown.

[0111] Below Figure 4 Based on the state shown in FIG. 1 (before the vehicle changes lanes, during normal driving, the vehicle driving state (such as vehicle speed, vehicle power, remaining battery power, etc.) is displayed in a specific area of ​​the front windshield (i.e., display area C)), the technical solution of visual assisted lane change provided by the embodiment of the present invention is described in detail. Fig.16 As shown, the visualization assisted lane changing method may include the following steps:

[0112] Step S1601: Determine the lane change direction: In response to receiving a lane change trigger of the lane change assistance system, determine the lane change direction, and if the lane change direction indicates a left lane change N, execute step S1602; if the lane change direction indicates a right lane change Y, execute step S1603;

[0113] Step S1602: Acquire images related to the left lane change: acquire the lane image of the left lane and the front image and rear image of the vehicle, stitch the images together, and execute step S1604;

[0114] Step S1603: Acquire images related to the right lane change: acquire a lane image of the right lane and a front image and a rear image of the vehicle, and stitch the images together;

[0115] Step S1604: Generate a border of the display area: The lane change auxiliary system identifies the lane condition, and generates a lane change indication signal matching the lane condition according to the identification result, generates and sets a border matching the lane change indication signal, and executes step S1605 for the lane change direction indication of left lane change N; and executes step S1606 for the lane change direction indication of right lane change Y;

[0116] Step S1605: displaying an image related to the left lane change and a frame of the display area L: displaying an image related to the left lane change and a frame of the display area L in the display area L on the left side of the display area C, and executing step S1607;

[0117] like Fig.11 As shown, if the lane condition is that the lane can be changed normally or the auxiliary lane change system can change lanes automatically, the display area L on the left side of the display area C displays an image related to the left lane change, and the border of the display area L is green.

[0118] like Fig.12 As shown, if the lane condition is that the lane is occupied, the display area L on the left side of the display area C displays an image related to the left lane change, and the border of the display area L is red.

[0119] Step S1606: displaying an image related to the right lane change and a frame of the display area R: displaying an image related to the right lane change in the display area R to the right of the display area C;

[0120] like Fig.13 As shown, if the lane condition is that the lane is occupied, the display area R on the right side of the display area C displays an image related to the right lane change, and the border of the display area R is red.

[0121] like Fig.14As shown, the lane condition is complex and the lane change assistance system cannot complete the lane change independently, then the display area R on the left side of the display area C displays an image related to the right lane change, and the border of the display area R is yellow.

[0122] Step S1607: Dynamically adjust the border of the display area.

[0123] For example, when the vehicle is changing lanes to the right, the lane condition changes from complicated to occupied, and the state of the display area R changes from Fig.14 becomes Fig.13 .

[0124] It is worth noting that any variation based on the above embodiment is within the scope of protection of this application. Fig.15 The status display shown is that the image on the left side of the vehicle or the left rearview mirror is displayed in the display area L, the image on the right side of the vehicle or the right rearview mirror is displayed in the display area R, and the vehicle driving status is displayed in the display area C. Fig.15 The status changes to Figures 5 to 7 , Fig.11 and Fig.12 One of the following; When the vehicle changes lanes to the right, Fig.15 The status changes to Figures 8 to 10 , Fig.13 and Fig.14 One of them.

[0125] Further, Fig.17 FIG. 2 is a schematic diagram showing a partial structure of a visual lane-changing assistance device provided by an embodiment of the present invention. Fig.17 As shown, the visual lane-changing assistance device 1700 may include: a processing unit 1701 and a display unit 1702, wherein:

[0126] The processing unit 1701 determines a lane change direction in response to receiving a lane change trigger of the lane change assistance system, and obtains a lane condition related to the lane change direction;

[0127] The display unit 1702 is used to display the lane condition in a display area in front of the driver's field of vision and matching the lane change direction; generate and set a border matching the lane condition for the display area, and dynamically adjust the border according to changes in the lane condition.

[0128] Furthermore, the processing unit 1701 is further used to obtain a first environmental image within a first setting range of a left lane associated with the left lane change and a second environmental image within a second setting range of a current lane in which the vehicle is located when the lane change direction indicates a left lane change; and to obtain a third environmental image within a third setting range of a right lane associated with the right lane change and a second environmental image within a second setting range of a current lane in which the vehicle is located when the lane change direction indicates a right lane change.

[0129] Furthermore, the processing unit 1701 is further used to obtain a front environment image in front of the vehicle and a rear environment image behind the vehicle in the current lane.

[0130] Furthermore, the processing unit 1701 is further configured to display the vehicle driving status in a specific display area in front of the driver's field of vision.

[0131] Furthermore, the display unit 1702 is further used to splice the first environment image and the second environment image, and display the spliced ​​image in the left area of ​​the specific display area when the lane change direction indicates a left lane change; or, when the lane change direction indicates a right lane change, splice the third environment image and the second environment image, and display the spliced ​​image in the right area of ​​the specific display area.

[0132] Furthermore, the display unit 1702 is further used to generate and set a frame that matches the lane change indication signal, wherein the lane change indication signal is generated by the auxiliary lane change system to identify the lane condition and based on the identification result, it matches the lane condition.

[0133] Furthermore, the display unit 1702 is further used to generate and configure a border of a first color for the display area when the lane change indication signal indicates a normal lane change; to generate and configure a border of a second color for the display area when the lane change indication signal indicates that traffic in the lane change area is complicated; and to generate and configure a border of a third color for the display area when the lane change indication signal indicates that the lane change area is occupied.

[0134] Furthermore, the display unit 1702 is further configured to stop displaying the lane condition in response to receiving a shutdown signal or an unavailable signal of the lane change assistance system.

[0135] In addition, an embodiment of the present invention further provides a vehicle, which can implement the visualization assisted lane changing method provided by any of the above embodiments or includes the visualization assisted lane changing device provided by any of the above embodiments.

[0136] Furthermore, an embodiment of the present invention also provides a visual assisted lane change system for a vehicle. Fig.18As shown, the visual assisted lane change system 1800 may include: an assisted lane change system 1801 and a visual assisted lane change device 1700 provided in any of the above embodiments, wherein:

[0137] The lane change assistance system 1801 is used to send a lane change signal to the visual lane change assistance device 1701;

[0138] The visual assisted lane changing device 1700 is used to execute the steps of the visual assisted lane changing method provided in any of the above embodiments after receiving a lane changing signal, so as to visually display the lane conditions and changes in the lane conditions related to the lane changing.

[0139] Furthermore, an embodiment of the present invention also provides an electronic device. The electronic device may include:

[0140] one or more processors;

[0141] a storage device for storing one or more programs,

[0142] When the one or more programs are executed by the one or more processors, the one or more processors implement the visualization-assisted lane change method as described in the above embodiment.

[0143] Furthermore, an embodiment of the present invention also provides a computer-readable medium, on which a computer program for implementing a visualization-assisted lane-changing method is stored.

[0144] When the computer program is executed by the on-board processor, the visualization assisted lane changing method provided in the above-mentioned first aspect embodiment is implemented.

[0145] Reference below Fig.19 , which shows a structural schematic diagram of a computer system 1900 suitable for implementing the method for preventing items from being left behind in a vehicle according to an embodiment of the present invention. Fig.19 The computer system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0146] like Fig.19 As shown, the computer system 1900 includes a central processing unit (CPU) 1901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1902 or a program loaded from a storage part 1908 into a random access memory (RAM) 1903. In the RAM 1903, various programs and data required for the operation of the system 1900 are also stored. The CPU 1901, the ROM 1902, and the RAM 1903 are connected to each other via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0147] The following components are connected to the I / O interface 1905: an input section 1906 including an output section 1907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1908 including a hard disk, etc.; and a communication section 1909 including a network interface card such as a LAN card, a modem, etc. The communication section 1909 performs communication processing via a network such as the Internet. A drive 1910 is also connected to the I / O interface 1205 as needed. A removable medium 1911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1910 as needed, so that a computer program read therefrom is installed into the storage section 1908 as needed.

[0148] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1909, and / or installed from the removable medium 1911. When the computer program is executed by the central processing unit (CPU) 1901, the above-mentioned functions defined in the system of the present invention are executed.

[0149] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0150] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0151] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes the above-mentioned processing unit and a display unit. The names of these modules or units do not constitute limitations on the modules or units themselves in some cases. For example, the processing unit may also be described as "a module or unit for determining a lane change direction and obtaining a lane condition related to the lane change direction".

[0152] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: in response to receiving a lane change trigger of an auxiliary lane change system, determining the lane change direction and obtaining lane conditions related to the lane change direction; displaying the lane conditions in a display area in front of the driver's field of view and matching the lane change direction; generating and setting a border matching the lane conditions for the display area, and dynamically adjusting the border according to changes in the lane conditions.

[0153] According to the technical solution of the embodiment of the present invention, after receiving the lane change trigger of the lane change assistance system, the lane change direction is determined, and the lane condition related to the lane change direction is obtained, and the lane condition is displayed in the display area in front of the driver's field of view and matching the lane change direction, so as to provide the driver with the lane condition of the lane change assistance in time, so that the driver can control the vehicle or adjust the lane change assistance system in time according to the lane condition. In addition, the technical solution generates and sets a frame matching the lane condition for the display area, and dynamically adjusts the frame according to the change of the lane condition, so that the driver can pay attention to the lane condition and the change of the lane condition in time according to the display area frame and the change of the frame, so as to further improve the experience of the lane change assistance and the driving experience of the driver.

[0154] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A visual assisted lane change method, characterized in that: include: In response to receiving a lane change trigger of the lane change assistance system, determining a lane change direction, and acquiring a lane condition related to the lane change direction; Displaying the lane condition in a display area in front of the driver's field of vision and matching the lane change direction; A frame matching the lane condition is generated and set for the display area, and the frame is dynamically adjusted according to changes in the lane condition.

2. The visualization-assisted lane-changing method according to claim 1, characterized in that: The acquiring of a lane condition related to the lane changing direction includes: When the lane change direction indicates a left lane change, acquiring a first environment image within a first set range of a left lane associated with the left lane change and a second environment image within a second set range of a current lane where the vehicle is located; When the lane change direction indicates a right lane change, a third environment image within a third setting range of a right lane associated with the right lane change and a second environment image within a second setting range of a current lane where the vehicle is located are acquired.

3. The visualization-assisted lane-changing method according to claim 2, characterized in that: Acquiring a second environment image within a second set range of the current lane where the vehicle is located, including: Acquire a front environment image in front of the vehicle and a rear environment image behind the vehicle in the current lane.

4. The visualization-assisted lane-changing method according to claim 2 or 3, characterized in that: Also includes: The vehicle's driving status is displayed in a specific display area in front of the driver's field of vision.

5. The visualization-assisted lane-changing method according to claim 4, characterized in that: The display area in front of the driver's field of vision and matching the lane change direction displays the lane condition, including: When the lane change direction indicates a left lane change, splicing the first environment image and the second environment image, and displaying the spliced ​​image in a left area of ​​the specific display area; or, When the lane change direction indicates a right lane change, the third environment image and the second environment image are spliced ​​together, and the spliced ​​image is displayed in a right area of ​​the specific display area.

6. The visualization-assisted lane-changing method according to claim 1, characterized in that: The step of generating a frame for the display area that matches the lane condition includes: The lane change assistance system identifies the lane condition and generates a lane change indication signal matching the lane condition according to the identification result; A frame matching the lane change indication signal is generated and set.

7. The visualization-assisted lane-changing method according to claim 6, characterized in that: Generating a frame matching the lane change indication signal includes: In response to the lane change indication signal indicating a normal lane change, generating and configuring a frame of a first color for the display area; In response to the lane change indication signal indicating that traffic is complex in the lane change area, generating and configuring a border of a second color for the display area; In response to the situation in which the lane change indication signal indicates that the lane change area is occupied, a border of a third color is generated and configured for the display area.

8. The visualization-assisted lane-changing method according to any one of claims 1 to 7, characterized in that: Also includes: In response to receiving a shutdown signal or an unavailable signal of the lane change assistance system, the display of the lane condition is stopped.

9. A visual lane-changing assistance device, characterized in that: include: A processing unit and a display unit, wherein: The processing unit determines a lane change direction in response to receiving a lane change trigger of the lane change assistance system, and obtains a lane condition related to the lane change direction; The display unit is used to display the lane condition in a display area in front of the driver's field of view and matching the lane change direction; generate and set a border matching the lane condition for the display area, and dynamically adjust the border according to changes in the lane condition.

10. A vehicle, characterized in that: Implement the visualization assisted lane changing method described in any one of claims 1 to 8 or include the visualization assisted lane changing device described in claim 9.

Images