Vehicle lane changing control method and device, electronic equipment and medium

Abstract

The embodiments of the present disclosure disclose a vehicle lane changing control method and device, electronic equipment and medium. Based on the obstacle information of the front obstacle in the current lane of the vehicle, when the vehicle needs to change lanes, the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane and the driving state of the driver of the vehicle are obtained. Then, based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides and the driving state, when the preset lane changing condition is met, the voice prompt message of whether to change lanes is output. Furthermore, in response to detecting the voice instruction message of confirming lane changing sent by the driver within a preset time, the vehicle is controlled to perform the lane changing operation, so that the lane changing of the vehicle under the L2 automatic driving level is realized without the need for the driver to manually take over the driving action of the current vehicle.

Description

Technical Field

[0001] This disclosure relates to artificial intelligence technology, and in particular to a vehicle lane-changing control method, apparatus, electronic device, and medium. Background Technology

[0002] With the rapid development of autonomous driving technology, autonomous vehicles have gradually become more widely used. Autonomous driving is classified into different levels based on the degree of automation. The autonomous driving level classification standards developed by the National Transportation Safety Administration (NHTSA) and the Society of Automotive Engineers (SAE) are currently the recognized standards for autonomous driving classification. Based on the NHTSA's standards, autonomous driving levels include five levels from L0 to L4; based on the SAE's standards, autonomous driving levels include six levels from L0 to L5.

[0003] According to the autonomous driving level classification standards established by NHTSA and SAE, Level 2 autonomous driving is partial autonomous driving, also known as Advanced Driver Assistance Systems (ADAS). The vehicle provides driving actions through various operations such as steering wheel operation and acceleration / deceleration, while the driver is responsible for the remaining driving actions. Summary of the Invention

[0004] When the vehicle is in Level 2 autonomous driving mode, it cannot change lanes autonomously. When it needs to avoid or overtake a slow-moving vehicle directly in front, the driver needs to manually take over the driving.

[0005] To address the aforementioned technical problems, this disclosure is proposed. Embodiments of this disclosure provide a vehicle lane-changing control method, apparatus, electronic device, and medium that enables lane changing at Level 2 autonomous driving without requiring manual driver intervention.

[0006] According to one aspect of the present disclosure, a vehicle lane-changing control method is provided, comprising:

[0007] Based on obstacle information of obstacles in front of the vehicle in its current lane, when it is determined that the vehicle needs to change lanes, the lane line information in front of the vehicle, obstacle information in the adjacent lanes on both sides of the current lane, and the driving status of the vehicle's driver are obtained.

[0008] Based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides, and the driving status, when it is determined that the preset lane-changing conditions are met, a voice prompt message asking whether to change lanes is output.

[0009] In response to detecting a voice instruction message sent by the driver to confirm the lane changing within a preset time, the vehicle is controlled to perform the lane changing operation.

[0010] According to another aspect of the embodiments of the present disclosure, a vehicle lane changing control device is provided, comprising:

[0011] A determination module is configured to, when it is determined that the vehicle needs to change lanes based on obstacle information of a front obstacle in a current lane of the vehicle, acquire lane line information in front of the vehicle, obstacle information in adjacent lanes on both sides of the current lane, and a driving state of a driver of the vehicle.

[0012] A voice interaction module is configured to, when it is determined that a preset lane changing condition is met based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides, and the driving state, output a voice prompt message for whether to change lanes.

[0013] A control module is configured to, in response to detecting a voice instruction message sent by the driver to confirm the lane changing within a preset time, control the vehicle to perform the lane changing operation.

[0014] According to still another aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, the storage medium storing a computer program, the computer program being configured to execute the vehicle lane changing control method according to any one of the embodiments of the present disclosure.

[0015] According to yet another aspect of the embodiments of the present disclosure, an electronic device is provided, comprising:

[0016] A processor;

[0017] A memory for storing instructions executable by the processor;

[0018] The processor is configured to execute the vehicle lane changing control method according to any one of the embodiments of the present disclosure.

[0019] The vehicle lane changing control method, device, electronic device and storage medium provided by the above embodiments of the present disclosure can determine that the vehicle needs to change lanes based on the obstacle information of the front obstacle in the lane where the vehicle is currently located, obtain the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the lane where the vehicle is currently located and the driving state of the driver of the vehicle, then determine that the preset lane changing condition is met based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the lane where the vehicle is currently located and the driving state of the driver, output a voice prompt message of whether to change lanes, and further, in response to detecting the voice instruction message of confirming lane changing sent by the driver within a preset time, control the vehicle to perform the lane changing operation. Thus, the embodiments of the present disclosure can realize lane changing of the vehicle at the L2 automatic driving level without manual intervention of the driver in driving actions of the current vehicle, effectively avoid or overtake the vehicle traveling at low speed in front or the dangerous vehicle, reduce the safety risk caused thereby, thereby improving the safety of vehicle driving, and help to reduce the takeover rate of ADAS, thereby improving the driving experience of the user. In addition, the embodiments of the present disclosure confirm whether to change lanes and trigger lane changing through voice interaction with the driver. Since the recognition accuracy and efficiency of voice are relatively high, it is helpful to strive for the lane changing opportunity and correctly understand the indication of whether to change lanes by the driver, thereby ensuring the safety of vehicle lane changing, avoiding traffic accidents caused by lane changing of the vehicle in the case of missing the lane changing opportunity or the driver not indicating lane changing, and improving the safety of vehicle driving.

[0020] The technical solutions of the present disclosure will be described in further detail below by means of the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. The drawings provided below are used to provide further understanding of the embodiments of the present disclosure, and constitute a part of the specification, and are used to explain the present disclosure together with the embodiments of the present disclosure, and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0022] Figure 1 is a scenario to which the present disclosure is applicable.

[0023] Figure 2 is a system diagram to which the present disclosure is applicable.

[0024] Figure 3 is a flowchart of a vehicle lane changing control method provided by an exemplary embodiment of the present disclosure.

[0025] Figure 4 is a flowchart of a vehicle lane changing control method provided by another exemplary embodiment of the present disclosure.

[0026] Figure 5 FIG. 1 is a flowchart of a vehicle lane changing control method according to an example embodiment of the present disclosure.

[0027] Figure 6 FIG. 2 is a flowchart of a vehicle lane changing control method according to another example embodiment of the present disclosure.

[0028] Figure 7 FIG. 3 is a flowchart of a vehicle lane changing control method according to yet another example embodiment of the present disclosure.

[0029] Figure 8 FIG. 4 is a flowchart of an application example of a vehicle lane changing control method according to an example embodiment of the present disclosure.

[0030] Figure 9 FIG. 5 is a structural diagram of a vehicle lane changing control device according to an example embodiment of the present disclosure.

[0031] Figure 10 FIG. 6 is a structural diagram of a vehicle lane changing control device according to another example embodiment of the present disclosure.

[0032] Figure 11 FIG. 7 is a structural diagram of an electronic device according to an example embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, and do not represent all the embodiments of the present disclosure. It should be understood that the present disclosure is not limited by the example embodiments described herein.

[0034] It should be noted that: unless otherwise specified, the relative arrangement, numerical expressions and values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0035] Those skilled in the art can understand that the terms "first", "second" and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, and do not represent any specific technical meaning, nor do they represent the inevitable logical sequence between them.

[0036] It should also be understood that in the embodiments of the present disclosure, "a plurality of" can mean two or more, and "at least one" can mean one, two or more.

[0037] It should also be understood that for any component, data or structure mentioned in the embodiments of the present disclosure, unless specifically limited or given the opposite implication by the context, it can be understood as one or more in general.

[0038] In addition, the term "and / or" in the present disclosure is merely a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the front and rear associated objects.

[0039] It should also be understood that the description of the various embodiments of the present disclosure focuses on the differences between the various embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0040] At the same time, it should be understood that, for the convenience of description, the size of each part shown in the drawings is not drawn according to the actual proportional relationship.

[0041] The following description of at least one example embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0042] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered part of the specification where appropriate.

[0043] It should be noted that similar reference numbers and letters refer to similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0044] Embodiments of the present disclosure can be applied to terminal devices, computer systems, servers, and other electronic devices, which can operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with terminal devices, computer systems, servers, and other electronic devices include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, mainframe computer systems, and distributed cloud computing technology environments including any of the above systems.

[0045] Electronic devices such as terminal devices, computer systems, servers, and the like can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, and the like that perform particular tasks or implement particular abstract data types. Computer systems / server can be practiced in distributed cloud computing environments with other computer systems coupled via communication networks. The program modules can be stored in the local or remote computer system memory devices.

[0046] SUMMARY

[0047] According to the different degrees of vehicle driving automation, automatic driving is divided into different levels. The automatic driving level division standard formulated by NHTSA and SAE is the currently recognized automatic driving grading standard. Among them, based on the automatic driving level division standard formulated by NHTSA, the automatic driving level includes L0-L4, a total of five levels; based on the automatic driving level division standard formulated by SAE, the automatic driving level includes L0-L5, a total of six levels. As shown in Table 1 below, it is the automatic driving level division standard table formulated by NHTSA and SAE. It should be noted that the embodiments of the present disclosure only take the automatic driving level division standard shown in Table 1 as an example for description, but are not limited to the automatic driving level division standard shown in Table 1.

[0048] Table 1

[0049]

[0050] The inventor of the present disclosure found through research that when the vehicle is in the L2 automatic driving level, the vehicle cannot autonomously change lanes. When a low-speed vehicle appears directly in front of the current vehicle, the current vehicle can only follow the low-speed vehicle moving directly in front and cannot avoid or overtake the low-speed vehicle moving directly in front by autonomously changing lanes.

[0051] When it is necessary to avoid or overtake the low-speed vehicle moving directly in front, the driver needs to manually take over the driving action of the current vehicle. If the vehicle moving directly in front moves at low speed and the driver does not manually take over the driving action of the current vehicle, the current vehicle can only follow the low-speed vehicle moving directly in front, which has a high safety risk and seriously affects the user's driving experience.

[0052] Exemplary System

[0053] The embodiments of the present disclosure can be used in the automatic driving scene of the vehicle. Figure 1 is a scenario to which the present disclosure is applicable. Figure 2 is a system diagram to which the present disclosure is applicable. Please refer toFigure 1 and Figure 2 The driving scene image and lane line image of the vehicle are collected by the first perception acquisition module 101 (such as a camera, etc.) deployed outside the vehicle (the current vehicle, i.e., the ego vehicle), and input into the vehicle lane changing control device 102 of the embodiment of the present disclosure; the driver image of the vehicle is collected by the second perception acquisition module 103 (such as a camera, etc.) deployed inside the vehicle, and input into the vehicle lane changing control device 102 of the embodiment of the present disclosure. The vehicle lane changing control device 102 obtains the obstacle information in the adjacent lanes on both sides of the lane in which the vehicle is currently located based on the driving scene image obtained by the first perception acquisition module 101, such as Figure 1 the obstacle information of the obstacles 1 and 2 in the left adjacent lane and the obstacle information of the obstacles 3 and 4 in the right adjacent lane, and determines that the vehicle needs to change lanes based on the obstacle information of the front obstacle in the lane in which the vehicle is currently located. When it is determined that the vehicle needs to change lanes, the lane line information of the preset lane line (such as Figure 1 the left-left lane line, the left lane line, the right lane line, and the right-right lane line) in front of the vehicle is determined based on the lane line image obtained by the first perception acquisition module 101, the driving state of the driver is determined based on the driver image obtained by the second perception acquisition module 103, and then when the preset lane changing condition is met based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the lane in which the vehicle is currently located, and the driving state of the driver, the vehicle outputs the voice prompt message of whether to change lanes through the audio playing module 104 (such as a loudspeaker), collects the voice instruction message sent by the driver through the audio acquisition module 105 (such as a microphone, etc.), and performs voice recognition on the collected voice instruction message. In response to collecting the voice instruction message sent by the driver to confirm the lane changing within a preset time, the vehicle performs the lane changing operation.

[0054] Therefore, the embodiment of the present disclosure realizes the lane changing of the vehicle at the L2 automatic driving level without the manual intervention of the driver in the driving action of the current vehicle, can effectively avoid or overtake the vehicle or dangerous vehicle driving at low speed in front, reduce the safety risk caused thereby, thereby improving the safety of vehicle driving, and helps to reduce the takeover rate of ADAS, and further improves the driving experience of the user. In addition, the embodiment of the present disclosure confirms whether to change lanes with the driver through voice interaction, and triggers the lane changing. Since the recognition accuracy and efficiency of the voice are relatively high, it is helpful to strive for the lane changing opportunity and correctly understand the indication of whether to change lanes by the driver, thereby ensuring the safety of the vehicle lane changing, avoiding the traffic accidents caused by the lane changing of the vehicle in the case of missing the lane changing opportunity or the driver not indicating the lane changing, and improving the safety of vehicle driving.

[0055] Exemplary Method

[0056] Figure 3 is a flowchart of a vehicle lane changing control method provided by an example embodiment of the present disclosure. The embodiment can be applied to electronic devices, vehicles, and the like, and as shown in the figure, the vehicle lane changing control method of the embodiment includes the following steps: Figure 3

[0057] In step 201, when it is determined that the vehicle needs to change lanes based on the obstacle information of the front obstacle in the current lane of the vehicle, the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the driver of the vehicle are obtained.

[0058] The vehicle herein can be a vehicle that can be controlled to change lanes by the embodiment of the present disclosure, and can also be referred to as a current vehicle, a self vehicle, or a host vehicle.

[0059] In step 202, when it is determined that the preset lane changing condition is met based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the driver of the vehicle, a voice prompt message of whether to change lanes is output.

[0060] In step 203, in response to detecting a voice instruction message of confirming lane changing sent by the driver within a preset time, the vehicle is controlled to perform a lane changing operation.

[0061] Based on the embodiment, the lane changing of the vehicle at the L2 automatic driving level is realized without the need for the driver to manually take over the driving action of the current vehicle, which can effectively avoid or overtake the vehicle traveling at low speed in front or the dangerous vehicle, reduce the safety risk caused thereby, thereby improving the safety of vehicle driving, and helping to reduce the takeover rate of ADAS, and further improving the driving experience of the user. In addition, the embodiment confirms whether to change lanes and triggers lane changing through voice interaction with the driver. Since the recognition accuracy and efficiency of voice are relatively high, it is helpful to strive for the lane changing opportunity and correctly understand the indication of whether the driver changes lanes, thereby ensuring the safety of vehicle lane changing, avoiding traffic accidents caused by lane changing when the lane changing opportunity is missed or the driver does not indicate lane changing, and improving the safety of vehicle driving.

[0062] Optionally, in some implementations, the obstacle information in the embodiment of the present disclosure, such as the obstacle information of the front obstacle in the current lane, the obstacle information in the adjacent lanes on both sides of the current lane, can include but is not limited to any one or more of the following: obstacle type, position, moving speed, and the like.

[0063] The obstacle type may, for example, include but is not limited to any one or more of the following: a small car, a truck, a bus, a tricycle, a motorcycle, a pedestrian, an animal, and the like.​

[0064] The position, i.e., the position of the obstacle relative to the current vehicle, may, for example, include but is not limited to any one or more of the following: front, rear, left front, left rear, right front, right rear, and the like.

[0065] The moving speed, i.e., the moving speed of the obstacle, may be the actual moving speed of the obstacle (i.e., absolute speed), or may be the relative speed of the obstacle relative to the current vehicle in the moving direction of the current vehicle, and the like, which is not limited in the embodiments of the present disclosure. When the moving speed of the obstacle is the actual moving speed of the obstacle, the relative speed of the obstacle relative to the current vehicle in the moving direction of the current vehicle can be obtained based on the actual moving speed of the obstacle and the moving speed of the current vehicle.

[0066] Based on the embodiments, based on the obstacle type, position, moving speed, and the like of the front obstacle in the current lane of the vehicle, it is determined whether the vehicle needs to change lanes to avoid or overtake the vehicle traveling at low speed in front or the dangerous vehicle, to reduce the safety risk caused thereby, and to improve the safety of vehicle driving.

[0067] Figure 4 FIG. 1 is a flowchart of a vehicle lane changing control method according to another example embodiment of the present disclosure. As shown in FIG. 1, based on the obstacle information of the front obstacle in the current lane of the vehicle, it is determined whether the vehicle needs to change lanes, which can include the following steps: Figure 4 Figure 3 Based on the embodiments shown in FIG. 1, in step 201, based on the obstacle information of the front obstacle in the current lane of the vehicle, it is determined that the vehicle needs to change lanes, which can include the following steps:

[0068] In step 2011, it is identified whether the obstacle information of the front obstacle in the current lane of the vehicle satisfies a preset lane changing trigger condition.

[0069] The preset lane changing trigger condition may, for example, include but is not limited to any one or more of the following: the obstacle type of the front obstacle in the current lane of the vehicle is a preset dangerous type, the moving speed of the front obstacle is lower than a preset speed requirement, and the like.

[0070] The preset dangerous type, i.e., the type that may affect the driving safety of the current vehicle, may, for example, include but is not limited to any one or more of the following: truck, truck, oil tanker, bus, tricycle, motorcycle, pedestrian, animal, and the like.

[0071] ​The preset speed requirement may include, but is not limited to, any one or more of the following: the moving speed of the front obstacle in the current lane of the vehicle is lower than the moving speed of the vehicle, the moving speed of the front obstacle in the current lane of the vehicle is lower than a preset speed, and the like. The preset speed may be determined based on the maximum speed limit of the current lane of the vehicle, for example, the preset speed may be a certain percentage (for example, 50%) of the maximum speed limit of the current lane, and the like. The specific requirements of the preset speed in the embodiments of the present disclosure are not limited.

[0072] Optionally, in some implementations, the first perception acquisition module may be deployed outside the vehicle, including a front camera and side-view cameras on both sides. The front camera is used to continuously capture images of a certain field of view range (which may include the front, left front, and right front) in front of the vehicle. The left side-view camera is used to continuously capture images of a certain field of view range (which may include the left front, left side, and left rear) on the left side of the vehicle. The right side-view camera is used to continuously capture images of a certain field of view range (which may include the right front, right side, and right rear) on the right side of the vehicle. The driving scene image captured by the front camera is used to obtain the obstacle information of the front obstacle in the current lane of the vehicle.

[0073] Optionally, in some implementations, the first neural network model may be trained in advance using driving scene image samples labeled with obstacle types. The trained first neural network model can identify the positions and types of obstacles in the driving scene image. In step 2011, the driving scene image captured by the front camera is input into the trained first neural network model, and the first neural network model outputs the obstacle detection result, including the positions and types of obstacles. Based on the positions of the obstacles in the driving scene image and the previous frame of the driving scene image captured by the front camera, and the time difference between the capture time of the driving scene image and the previous frame of the driving scene image, the moving speed of each obstacle can be determined. Thus, the obstacle information of the front obstacle in the current lane of the vehicle can be obtained.

[0074] In step 2012, in response to the obstacle information of the front obstacle in the current lane of the vehicle satisfying the preset lane-changing trigger condition, it is determined that the vehicle needs to change lanes.

[0075] Otherwise, if the obstacle information of the front obstacle in the current lane of the vehicle does not satisfy the preset lane-changing trigger condition, it is determined that the vehicle does not need to change lanes.

[0076] Based on the embodiment, whether the obstacle information of the front obstacle in the lane where the vehicle is currently located meets the specific preset lane-changing trigger condition determines whether the vehicle needs to change lanes. When the type of the front obstacle is a preset dangerous type and / or the moving speed of the front obstacle is lower than a preset speed requirement, it is determined that the vehicle needs to change lanes to avoid or overtake the vehicle that is driving at a low speed or the dangerous vehicle in front, reduce the safety risk caused thereby, and improve the safety of vehicle driving.

[0077] Figure 5 is a flowchart of a vehicle lane-changing control method provided by another exemplary embodiment of the present disclosure. As shown in the figure, Figure 5 on the basis of the embodiments shown in the above Figure 3 or Figure 4 , in step 201, the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the lane where the vehicle is currently located, and the driving state of the driver of the vehicle are obtained, which can include the following steps:

[0078] In step 2013, the lane line information of the preset lane line in front of the vehicle is determined based on the lane line image obtained by the first perception acquisition module outside the vehicle.

[0079] The preset lane line may, for example, include but is not limited to the left lane line, the right lane line, the left second lane line, and the right second lane line of the lane where the vehicle is currently located, etc.

[0080] The lane line information may, for example, include but is not limited to whether the lane line is clear, the solid-void type of the lane line, and the length of the lane line, etc.

[0081] Optionally, in some implementations, the first perception acquisition module can include a front camera and side-view cameras disposed on both sides outside the vehicle. The front camera is used to collect images in a certain field of view range (which can include the front, left front, and right front) in front of the vehicle, and the lane line image can be collected by the front camera.

[0082] Optionally, in some implementations, the first neural network model can be trained in advance using lane line image samples labeled with preset lane lines and lane line types (dashed or solid). The trained first neural network model can identify the lane line in the lane line image and the type of the lane line.

[0083] In this step 2013, the lane line image acquired by the first perception acquisition module can be input into the trained second neural network model, and the lane line image is detected by the second neural network model to output the lane line detection result, including the pixel points belonging to each preset lane line in the lane line image and the probability value belonging to the lane line, and the probability value of the lane line type being a dashed line and a solid line. Whether each preset lane line is clear is determined based on whether the probability value belonging to each preset lane line in the lane line detection result is greater than a preset confidence threshold (for example, 60%). If the probability value belonging to the preset lane line is greater than the preset confidence threshold, it can be considered that the corresponding preset lane line is clear; otherwise, if the probability value belonging to the preset lane line is not greater than the preset confidence threshold, it can be considered that the corresponding preset lane line is not clear. The solid and dashed type of each preset lane line is determined based on the probability value of the lane line type being a dashed line or a solid line corresponding to each preset lane line in the lane line detection result. For example, for each preset lane line, the lane line type with a greater probability value between the probability values of the dashed line and the solid line can be selected as the solid and dashed type of the corresponding preset lane line. Based on the pixel points belonging to each preset lane line determined in the lane line detection result, the actual length of each preset lane line can be determined as the lane line length based on the conversion relationship between the image coordinate system and the world coordinate system. Thus, the lane line information of the preset lane line in front of the vehicle can be obtained.

[0084] In step 2014, based on the driving scene image acquired by the first perception acquisition module outside the vehicle, the obstacle information in the adjacent lanes on both sides of the lane where the vehicle is currently located is acquired.

[0085] Optionally, in some implementations, the first perception acquisition module can include a front camera and side-view cameras disposed on both sides of the vehicle outside the vehicle, wherein the left side-view camera is used to continuously acquire images within a certain field of view range (which can include the left front, left side, and left rear) on the left side of the vehicle, and the right side-view camera is used to continuously acquire images within a certain field of view range (which can include the right front, right side, and right rear) on the right side of the vehicle. The driving scene image can be acquired by the side-view cameras on both sides.

[0086] Optionally, in some implementations, based on the driving scene image acquired by the first perception acquisition module outside the vehicle, the obstacle type, position, and moving speed of the obstacles located in front of, behind, and parallel to the current vehicle in the adjacent lanes on both sides of the lane where the current vehicle is located can be acquired.

[0087] Optionally, in some implementations, the first neural network model can be trained in advance using driving scene image samples labeled with obstacle types, and the trained first neural network model can identify the positions and types of obstacles in the driving scene images. In this step 2014, the driving scene images captured by the two side-view cameras can be respectively input into the trained first neural network model, and the first neural network model can output obstacle detection results including the positions and types of obstacles. Based on the positions of the obstacles in the driving scene images captured by the two side-view cameras and the front frame driving scene image, and the time difference between the capture time of the driving scene image and the front frame driving scene image, the moving speed of each obstacle can be determined. Thus, the obstacle information in the adjacent lanes on both sides of the lane where the vehicle is currently located can be obtained.

[0088] In step 2015, the driving state of the driver of the vehicle is determined based on the driver image obtained by the second perception acquisition module inside the vehicle.

[0089] Optionally, in some implementations, the second perception acquisition module can be a camera deployed inside the vehicle, in front of the driver's seat, in the left front, upper, front upper, left front upper, right front upper, etc. The camera can continuously capture images of the driver to obtain the driver image.

[0090] Optionally, in some implementations, the driving state of the driver can include, but is not limited to, whether it is in a fatigue state, whether it is in a distraction state, etc.

[0091] In some specific examples, the third neural network model can be trained in advance using first driver sample images labeled with fatigue state labels indicating whether the driver is in a fatigue state, and the trained third neural network model can identify whether the driver in the driver image is in a fatigue state.

[0092] In some specific examples, whether it is in a distraction state can be determined by whether the driver's line of sight area meets the preset line of sight requirements and whether the driver makes a preset distraction action. If the driver's line of sight area meets the preset line of sight requirements and the driver does not make a preset distraction action, it can be considered that the driving state of the driver is not in a distraction state; otherwise, if the driver's line of sight area does not meet the preset line of sight requirements and / or the driver makes a preset distraction action, it can be considered that the driving state of the driver is in a distraction state.

[0093] The line-of-sight region of the driver meets the preset line-of-sight requirement, for example, can include any one of the following: the angle of the line-of-sight direction of the driver deviating from the driving direction of the vehicle is less than a first preset angle (for example, 5°), at which time it can be considered that the driver's line of sight is looking forward; the angle of the line-of-sight direction of the driver deviating from the driving direction of the vehicle is not less than the first preset angle and less than a second preset angle (for example, 15°), at which time it can be considered that the driver's line of sight is directed to the left rearview mirror or by the rearview mirror, wherein the first preset angle is less than the second preset angle. Otherwise, if the angle of the line-of-sight direction of the driver deviating from the driving direction of the vehicle is greater than the second preset angle, for example, the line-of-sight direction of the driver is directed to the rear, left rear, right rear, etc., it is considered that the line-of-sight region of the driver does not meet the preset line-of-sight requirement.

[0094] In a specific implementation, the fourth neural network model can be trained in advance using a second driver sample image, wherein the second driver sample image is labeled with a deviation angle label for indicating the angle of the line-of-sight direction of the driver deviating from the driving direction of the vehicle. The trained fourth neural network model can predict the angle of the line-of-sight direction of the driver in the driver image deviating from the driving direction of the vehicle. Based on the relationship between the angle predicted by the fourth neural network model and the first preset angle and the second preset angle, it can be determined that the line-of-sight region of the driver meets the preset line-of-sight requirement.

[0095] The preset distraction action may, for example, include but is not limited to any one or more of the following: looking down, smoking, drinking water, eating, making a phone call, entertainment, makeup, etc. In a specific implementation, the fifth neural network model can be trained in advance using a third driver sample image, wherein the third driver sample image is labeled with a label indicating whether the driver has a preset distraction action and a preset distraction action type. The trained fifth neural network model can predict the distraction action detection result of the driver image, including whether the driver has a preset distraction action and a preset distraction action type. Based on the distraction action detection result predicted by the fifth neural network model, it can be determined whether the driver makes a preset distraction action.

[0096] There is no execution timing restriction between the above steps 2013-2015, and they can be executed simultaneously or in any sequence or time difference.

[0097] Based on the embodiment, the lane line information of the preset lane line in front of the vehicle can be determined based on the lane line image acquired by the first perception acquisition module outside the vehicle, the obstacle information in the adjacent lanes on both sides of the lane where the vehicle is currently located can be acquired based on the driving scene image acquired by the first perception acquisition module, and the driving state of the driver of the vehicle can be determined based on the driver image acquired by the second perception acquisition module inside the vehicle, so as to objectively and accurately determine that the preset lane changing condition is met based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides, and the driving state of the driver, thereby improving the safety of lane changing of the vehicle.

[0098] Optionally, in some implementations, the preset lane changing condition may, for example, include but is not limited to: a preset lane line condition, a preset obstacle condition, and a preset driving state condition. Accordingly, in some implementations, Figure 3 In step 202 of the embodiment shown, whether the preset lane line condition, the preset obstacle condition, and the preset driving state condition are met can be determined based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the lane where the vehicle is currently located, and the driving state of the vehicle. In response to the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the lane where the vehicle is currently located, and the driving state of the driver of the vehicle, the preset lane changing condition is met, i.e., the vehicle can change lanes.

[0099] Based on the embodiment, whether the preset lane changing condition is met by the vehicle can be determined based on whether the lane line information in front of the vehicle meets the preset lane line condition, whether the obstacle information in the adjacent lanes on both sides of the lane where the vehicle is currently located meets the preset obstacle condition, and whether the driving state of the driver meets the preset driving state condition, thereby improving the safety of lane changing of the vehicle.

[0100] Optionally, in some implementations, the preset lane changing condition may, for example, include but is not limited to: a preset lane line condition, a preset obstacle condition, and a preset driving state condition. Accordingly, in some implementations, Figure 3 In step 202 of the embodiment shown, in response to the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the lane where the vehicle is currently located, and the driving state of the vehicle, the preset lane changing condition is met, i.e., the vehicle can change lanes.

[0101] In a case where the lateral control function of the vehicle is in an activated state, the vehicle can be controlled in a lateral direction, and a lane centering control can be performed on the vehicle based on a position of the vehicle in a current lane and a centerline position of the current lane, so as to avoid a sudden lane change of the vehicle, and thus avoid a collision between the vehicle and an obstacle in another lane, or a collision between the vehicle and a road-side barrier or a road edge, thereby improving driving safety of the vehicle. In a case where the longitudinal control function of the vehicle is in an activated state, the vehicle can be controlled in a longitudinal direction, and a driving speed of the vehicle can be controlled based on a distance and a moving speed between the vehicle and an obstacle in front of or behind the vehicle, so as to avoid a sudden acceleration or deceleration of the vehicle, and thus avoid a collision between the vehicle and the obstacle in front of or behind the vehicle, thereby improving driving safety of the vehicle. In a specific application, whether the lateral and longitudinal control functions of the vehicle are in the activated state can be obtained from state information of the vehicle in an automatic driving system, or can be obtained from state information of the vehicle in a central control system of the vehicle, and the like. The specific manner of obtaining whether the lateral and longitudinal control functions of the vehicle are in the activated state is not limited in the embodiments of the present disclosure.

[0102] According to the embodiments, whether the lateral and longitudinal control functions of the vehicle are in the activated state is further determined. Only in a case where the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the vehicle satisfy the corresponding preset lane line condition, preset obstacle condition and preset driving state condition, and the lateral and longitudinal control functions of the vehicle are in the activated state, it is determined that the preset lane changing condition is satisfied, and it is determined that the vehicle can change lanes. The vehicle can avoid a collision with other obstacles during or after the lane changing, thereby improving safety of the lane changing of the vehicle.

[0103] Optionally, in some implementations, the adjacent lanes on both sides of the current lane of the vehicle can include a left adjacent lane of the current lane of the vehicle and a right adjacent lane of the current lane of the vehicle.

[0104] Correspondingly, the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the vehicle satisfying the preset lane line condition, preset obstacle condition and preset driving state condition can be that, based on the lane line information in front of the vehicle, the left lane line and the left second lane line of the current lane of the vehicle are clear respectively and have a length greater than a preset length requirement, and a solid-void type of the left lane line is a dashed line; based on the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, an obstacle type of the obstacle in the left adjacent lane is not a preset dangerous type, and a moving speed of a front obstacle (i.e., an obstacle in front of the vehicle, for example, obstacle 1 in FIG. 1) in the left adjacent lane is not less than a preset speed requirement, and further can include that a rear obstacle (i.e., an obstacle behind the vehicle, for example, obstacle 2 in FIG. 1) in the left adjacent lane has a moving speed not less than the preset speed requirement. Figure 1 Correspondingly, the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the vehicle satisfying the preset lane line condition, preset obstacle condition and preset driving state condition can be that, based on the lane line information in front of the vehicle, the left lane line and the left second lane line of the current lane of the vehicle are clear respectively and have a length greater than a preset length requirement, and a solid-void type of the left lane line is a dashed line; based on the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, an obstacle type of the obstacle in the left adjacent lane is not a preset dangerous type, and a moving speed of a front obstacle (i.e., an obstacle in front of the vehicle, for example, obstacle 1 in FIG. 1) in the left adjacent lane is not less than a preset speed requirement, and further can include that a rear obstacle (i.e., an obstacle behind the vehicle, for example, obstacle 2 in FIG. 1) in the left adjacent lane has a moving speed not less than the preset speed requirement.Figure 1 If the obstacle 2) in the lane moves at a speed lower than the vehicle's speed, and / or the distance between the vehicle and the obstacle behind it in the adjacent lane on the left is greater than a preset distance; and the driver of the vehicle is neither fatigued nor distracted, then the conditions for the vehicle to switch lanes from its current lane to the adjacent lane on the left are met.

[0105] Among these criteria, lane line clarity is defined as a probability value greater than a preset confidence threshold for each pixel in the lane line image belonging to a lane line. For details regarding lane line clarity, lane line length, lane line solid / phasic type determination, obstacle type determination, and driver's driving state determination, please refer to the above. Figure 5 The steps 2013-2015 described in the illustrated embodiment will not be repeated here.

[0106] Optionally, in a specific example, the lane length is greater than a preset length requirement. This may include, but is not limited to, lane lengths greater than a preset length, or lane lengths greater than the product of the vehicle speed and a preset coefficient. The preset length can be pre-set according to actual needs, for example, a preset length of 100 meters, and can be adjusted according to actual needs. The preset coefficient can also be pre-set according to actual needs, for example, a preset coefficient of 1.5, and can be adjusted according to actual needs.

[0107] Alternatively, the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the vehicle's current lane, and the vehicle's driving status can satisfy preset lane line conditions, preset obstacle conditions, and preset driving status conditions. This can also be achieved by: based on the lane line information in front of the vehicle, the right lane line and the second lane line on the right of the vehicle's current lane are clearly defined and their lengths exceed a preset length requirement; the right lane line is a dashed line. Based on the obstacle information in the adjacent lanes on both sides of the vehicle's current lane, the obstacle type in the right adjacent lane is not a preset hazard type, and the obstacle in front of the right adjacent lane (i.e., the obstacle located in front of the vehicle, for example...) Figure 1 The moving speed of obstacle 3) in the vehicle is not lower than a preset speed requirement. Furthermore, it may also include: obstacles behind the vehicle in the adjacent lane on the right (i.e., obstacles located behind the vehicle, such as...). Figure 1 If the obstacle 4) in the lane moves at a speed lower than the vehicle's speed, and / or the distance between the vehicle and the obstacle behind it in the adjacent lane on the right is greater than a preset distance; and the driver of the vehicle is neither fatigued nor distracted, then the conditions for the vehicle to switch lanes from its current lane to the adjacent lane on the right are met.

[0108] Based on the embodiment, whether the lane changing condition of the vehicle from the current lane to the left or right adjacent lane is met can be determined based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the vehicle, so as to determine the adjacent lane that can be switched by the vehicle.

[0109] Figure 6 is a flowchart of a vehicle lane changing control method provided by another exemplary embodiment of the present disclosure. As shown in the figure, Figure 6 on the basis of any of the above Figures 3-5 embodiments, step 202 can include the following steps:

[0110] Step 2021, based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the driver of the vehicle, when the preset lane changing condition is met, determining the target lane in the adjacent lanes on both sides that meets the preset lane changing condition.

[0111] Among the target lanes in the adjacent lanes on both sides that meet the preset lane changing condition, any one or more of the following can be included: the left adjacent lane of the current lane of the vehicle, the right adjacent lane of the current lane of the vehicle.

[0112] Step 2022, outputting a voice prompt message whether to change lanes to the target lane.

[0113] Based on the embodiment, when the preset lane changing condition is met, the target lane in the adjacent lanes on both sides that meets the preset lane changing condition is determined, and a voice prompt message whether to change lanes to the target lane is outputted, so that the driver confirms the situation of the target lane, and thus determines whether to change lanes to the target lane for driving, ensuring the safety of vehicle lane changing.

[0114] Figure 7 is a flowchart of a vehicle lane changing control method provided by another exemplary embodiment of the present disclosure. As shown in the figure, Figure 6 on the basis of any of the above Figures 3-6 embodiments, step 203 can include the following steps:

[0115] Step 2031, in response to detecting the voice instruction message sent by the driver to confirm lane changing within a preset time, the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the driver of the vehicle are acquired again.

[0116] The preset time can be set according to actual driving environment, road type setting of the current lane, and other actual factors, for example, can be set to 3 seconds. When the driving environment is simple (for example, there are fewer obstacles), the corresponding maximum speed limit of the vehicle driving of the road type setting of the current lane is low, and the preset time can be longer. When the driving environment is complex (for example, there are more obstacles), the corresponding maximum speed limit of the vehicle driving of the road type setting of the current lane is high, and the preset time can be shorter. The specific size of the preset time can also be set considering other factors, and the specific size and factors considered by the disclosed embodiments are not limited.

[0117] In step 2032, whether the preset lane changing condition is met is determined based on the lane line information acquired again in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the driver of the vehicle.

[0118] In step 2033, in response to the lane line information acquired again in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the driver of the vehicle meeting the preset lane changing condition, the vehicle is controlled to perform a lane changing operation, that is, the vehicle is controlled to change lanes from the current lane to the target lane that meets the preset lane changing condition.

[0119] Based on the present embodiment, since the traffic conditions change dynamically, when the driver confirms to change lanes for the vehicle, the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the driver of the vehicle are acquired again and it is confirmed whether the preset lane changing condition is met. Only when it is confirmed again that the preset lane changing condition is met, the vehicle is controlled to perform a lane changing operation, further improving the safety of vehicle lane changing, which can avoid traffic accidents caused by lane changing when the lane changing opportunity is missed or the driver does not indicate lane changing, and improve the safety of vehicle driving.

[0120] Figure 8 is a flowchart of an application embodiment of the vehicle lane changing control method provided based on the present embodiment. The vehicle lane changing control method of the present embodiment will be further described in detail through a specific application embodiment. As shown in Figure 8 The application embodiment includes the following steps:

[0121] In step 301, the obstacle information of the front obstacle in the current lane of the vehicle is acquired based on a first perception acquisition module outside the vehicle.

[0122] In step 302, whether the obstacle information of the front obstacle in the current lane of the vehicle meets a preset lane changing trigger condition is identified, for example, the obstacle type of the front obstacle is a preset dangerous type, and the moving speed of the front obstacle is lower than a preset speed requirement.

[0123] If the obstacle type of the front obstacle is a preset dangerous type, and / or the moving speed of the front obstacle is lower than a preset speed requirement, it is determined that the vehicle needs to change lanes, and steps 303, 305 and 307 are respectively performed; otherwise, if any of the preset lane changing triggering conditions is not met, it is determined that the vehicle does not need to change lanes, and the subsequent process of the embodiment is not performed, and the current assisted driving mode is continued.

[0124] In step 303, lane line information of a preset lane line in front of the vehicle is determined based on a lane line image obtained by the first perception acquisition module outside the vehicle.

[0125] In step 304, it is determined whether the lane line information of the preset lane line in front of the vehicle meets a preset lane line condition, to obtain a first determination result.

[0126] If the lane line information of the preset lane line in front of the vehicle meets the preset lane line condition, step 309 is performed; otherwise, if the lane line information of the preset lane line in front of the vehicle does not meet the preset lane line condition, the subsequent process of the embodiment is not performed, and the current assisted driving mode is continued.

[0127] In step 305, obstacle information in adjacent lanes on both sides of the lane in which the vehicle is currently located is obtained based on a driving scene image obtained by the first perception acquisition module outside the vehicle.

[0128] In step 306, it is determined whether the obstacle information in the adjacent lanes on both sides of the lane in which the vehicle is currently located meets a preset obstacle condition, to obtain a second determination result.

[0129] If the obstacle information in the adjacent lanes on both sides of the lane in which the vehicle is currently located meets the preset obstacle condition, step 309 is performed; otherwise, if the obstacle information in the adjacent lanes on both sides of the lane in which the vehicle is currently located does not meet the preset obstacle condition, the subsequent process of the embodiment is not performed, and the current assisted driving mode is continued.

[0130] In step 307, a driving state of the driver of the vehicle is determined based on a driver image obtained by the second perception acquisition module inside the vehicle.

[0131] In step 308, it is determined whether the driving state of the driver meets a preset driving state condition, to obtain a third determination result.

[0132] If the driving state of the driver meets the preset driving state condition, step 309 is performed; otherwise, if the driving state of the driver does not meet the preset driving state condition, the subsequent process of the embodiment is not performed, and the current assisted driving mode is continued.

[0133] There is no execution time limit between the above steps 303, 305 and 307, and the three can be executed simultaneously or in any sequence or time difference.

[0134] Step 309: Based on the first determination result, the second determination result and the third determination result, it is determined whether the preset lane line condition, the preset obstacle condition and the preset driving state condition are met simultaneously.

[0135] If the preset lane line condition, the preset obstacle condition and the preset driving state condition are met simultaneously, step 310 is executed; otherwise, if any of the preset lane line condition, the preset obstacle condition and the preset driving state condition is not met, the subsequent process of the embodiment is not executed, and the current assisted driving mode is continued.

[0136] Step 310: It is confirmed that the lateral and longitudinal control function of the vehicle is in an activated state.

[0137] If the lateral and longitudinal control function of the vehicle is in an activated state, step 311 is executed; otherwise, if the lateral and longitudinal control function of the vehicle is not in an activated state, the subsequent process of the embodiment is not executed, and the current assisted driving mode is continued.

[0138] Step 311: Based on the first determination result and the second determination result, a target lane meeting the preset lane changing condition in the two adjacent lanes on both sides of the current lane of the vehicle is determined. The target lane can be the left adjacent lane of the current lane of the vehicle, the right adjacent lane of the current lane of the vehicle, or both the left adjacent lane and the right adjacent lane.

[0139] If the target lane meeting the preset lane changing condition in the two adjacent lanes on both sides of the current lane of the vehicle includes both the left adjacent lane and the right adjacent lane, one of the left adjacent lane and the right adjacent lane can be determined as the final target lane according to the preset priority, or the final target lane can be confirmed by the driver through voice interaction.

[0140] Step 312: A voice prompt message of whether to change lanes to the target lane is output, and timing is started according to a preset time after the voice prompt message is output.

[0141] For example, it can be "whether to overtake"; or, when the target lane meeting the preset lane changing condition is the left adjacent lane, a voice prompt message "whether to switch to the left adjacent lane" can be output; or, when the target lane meeting the preset lane changing condition is the right adjacent lane, a voice prompt message "whether to switch to the right adjacent lane" can be output; or, when the target lane meeting the preset lane changing condition includes both the left adjacent lane and the right adjacent lane, a voice prompt message "whether to switch to the left or right adjacent lane" can be output, and the like.

[0142] Step 313: Confirm whether a voice instruction message confirming lane change sent by the driver was detected within a preset time.

[0143] If a voice instruction message confirming lane change is detected from the driver within a preset time, step 314 is executed; otherwise, if no voice instruction message confirming lane change is detected from the driver within the preset time, the subsequent process of this embodiment is not executed, and the current assisted driving mode is maintained. Alternatively, a voice message indicating abandonment of lane change may be played, such as "Response timeout, abandon overtaking".

[0144] Step 314: Determine again whether the preset lane-changing conditions are met.

[0145] For example, steps 303-311 can be performed again to determine the target lanes that meet the preset lane-changing conditions in the adjacent lanes on both sides of the lane where the vehicle is currently located.

[0146] If the preset lane-changing conditions are again met, proceed to step 315; or, a voice response message, such as "Okay, overtaking immediately," can be played first, followed by step 315; or, a voice response message, such as "Okay, overtaking immediately," can be played simultaneously with step 315. Otherwise, if the preset lane-changing conditions are again not met, the subsequent process of this embodiment is not executed, and the current assisted driving mode is maintained; or, a voice message indicating abandonment of lane changing, such as "The intersection is complex, abandon overtaking," can be played.

[0147] Step 315: Control the vehicle to perform a lane change operation, that is, control the vehicle to change lanes from its current lane to a target lane that meets the preset lane change conditions.

[0148] Any of the vehicle lane-changing control methods provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any of the vehicle lane-changing control methods provided in this disclosure can be executed by a processor, such as by a processor executing any of the vehicle lane-changing control methods mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0149] Exemplary Apparatus

[0150] The vehicle lane change control device of this disclosure can be used to implement the vehicle lane change control methods of the above embodiments of this disclosure. Figure 9 This is a schematic diagram of the structure of a vehicle lane-changing control device provided in an exemplary embodiment of this disclosure. Figure 9 As shown, the vehicle lane-changing control device of this embodiment includes: a determination module 401, a voice interaction module 402, and a control module 403. Wherein:

[0151] The determining module 401 is configured to, based on the obstacle information of the front obstacle in the current lane of the vehicle, determine that the vehicle needs to change lanes, and acquire lane line information in front of the vehicle, obstacle information in adjacent lanes on both sides of the current lane of the vehicle, and a driving state of a driver of the vehicle.

[0152] The voice interaction module 402 is configured to, based on the lane line information in front of the vehicle, the obstacle information in adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the driver of the vehicle acquired by the determining module 401, determine that a preset lane changing condition is met, and output a voice prompt message of whether to change lanes.

[0153] The control module 403 is configured to, in response to detecting a voice instruction message of confirming lane changing sent by the driver within a preset time, control the vehicle to perform a lane changing operation.

[0154] Based on the embodiment, the lane changing of the vehicle at the L2 automatic driving level is realized without the need for the driver to manually take over the driving action of the current vehicle, the vehicle can effectively avoid or overtake the vehicle or dangerous vehicle driving at low speed in front, and reduce the safety risk caused thereby, thereby improving the safety of vehicle driving, and helping to reduce the takeover rate of ADAS, and further improving the driving experience of the user. In addition, the embodiment confirms whether to change lanes and triggers lane changing through voice interaction with the driver, since the recognition accuracy and efficiency of the voice are relatively high, it is helpful to strive for the lane changing opportunity and correctly understand the indication of whether the driver changes lanes, thereby ensuring the safety of the vehicle lane changing, avoiding traffic accidents caused by lane changing when the lane changing opportunity is missed or the driver does not indicate lane changing, and improving the safety of vehicle driving.

[0155] Optionally, in some implementations, the obstacle information in the embodiment of the present disclosure, for example, the obstacle information of the front obstacle in the current lane, the obstacle information in the adjacent lanes on both sides of the current lane, can include but is not limited to any one or more of the following: obstacle type, position, moving speed, etc.

[0156] Figure 10 is a structural schematic diagram of a vehicle lane changing control device provided by another exemplary embodiment of the present disclosure. As shown in Figure 10 , in Figure 9On the basis of the illustrated embodiment, in the vehicle lane changing control of the embodiment, the determining module 401 can include a first determining unit 4011 and a first obtaining unit 4012. The first determining unit 4011 is configured to identify whether the obstacle information of the front obstacle meets a preset lane changing trigger condition. The preset lane changing trigger condition can include, but is not limited to, any one or more of the following: the obstacle type of the front obstacle is a preset dangerous type, and the moving speed of the front obstacle is lower than a preset speed requirement. In response to the obstacle information of the front obstacle meeting the preset lane changing trigger condition, it is determined that the vehicle needs to change lanes. The first obtaining unit 4012 is configured to obtain lane line information in front of the vehicle, obstacle information in adjacent lanes on both sides of the lane in which the vehicle is currently located, and a driving state of the driver of the vehicle when the first determining unit 4011 determines that the vehicle needs to change lanes.

[0157] Optionally, in some implementations, the first obtaining unit 4012 can include a first determining subunit, an obtaining subunit, and a second determining subunit. The first determining subunit is configured to determine lane line information of a preset lane line in front of the vehicle based on a lane line image obtained by the first perception obtaining module outside the vehicle. The preset lane line can include, but is not limited to, for example, a left lane line of the lane in which the vehicle is currently located, a right lane line of the lane in which the vehicle is currently located, a left second lane line of the lane in which the vehicle is currently located, and a right second lane line of the lane in which the vehicle is currently located. The lane line information can include, but is not limited to, for example, any one or more of the following: whether the lane line is clear, the solid and dashed type of the lane line, and the length of the lane line. The obtaining subunit is configured to obtain obstacle information in adjacent lanes on both sides of the lane in which the vehicle is currently located based on a driving scene image obtained by the first perception obtaining module outside the vehicle. The second determining subunit is configured to determine a driving state of the driver of the vehicle based on a driver image obtained by the second perception obtaining module inside the vehicle.

[0158] Optionally, in some implementations, the above-mentioned preset lane changing condition can include, but is not limited to, for example, a preset lane line condition, a preset obstacle condition, and a preset driving state condition. Accordingly, the voice interaction module 402 is configured to output a voice prompt message of whether to change lanes based on the lane line information in front of the vehicle, the obstacle information in adjacent lanes on both sides of the lane in which the vehicle is currently located, and the driving state of the driver of the vehicle when it is determined that the preset lane changing condition is met.

[0159] The voice interaction module 402 can include a second determination unit 4021 and an output unit 4022. The second determination unit 4021 is configured to determine whether a preset lane changing condition is met based on lane line information in front of the vehicle, obstacle information in adjacent lanes on both sides of a lane in which the vehicle is currently located, and a driving state of a driver of the vehicle; and determine that the preset lane changing condition is met in response to the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the lane in which the vehicle is currently located, and the driving state of the driver of the vehicle meeting preset lane line conditions, preset obstacle conditions, and preset driving state conditions. The output unit 4022 is configured to output a voice prompt message of whether to change lanes when the second determination unit 4021 determines that the preset lane changing condition is met.

[0160] Optionally, in the above implementation manners, the preset lane changing condition can further include that a lateral and longitudinal control function of the vehicle is in an activated state. Accordingly, in this embodiment, the second determination unit 4021 is specifically configured to determine that the preset lane changing condition is met in response to the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the lane in which the vehicle is currently located, and the driving state of the driver meeting the preset lane line conditions, the preset obstacle conditions, and the preset driving state conditions, and the lateral and longitudinal control function of the vehicle being in the activated state.

[0161] Optionally, in some of the implementation manners, the adjacent lanes on both sides of the lane in which the vehicle is currently located can include a left adjacent lane and a right adjacent lane. Accordingly, in this embodiment, the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the lane in which the vehicle is currently located, and the driving state of the driver meeting the preset lane line conditions, the preset obstacle conditions, and the preset driving state conditions can include that:

[0162] Based on the lane line information in front of the vehicle, the left lane line and the left second lane line of the current lane of the vehicle are clear respectively, and the lane line length is greater than the preset length requirement, the virtual-real type of the left lane line is a dashed line; based on the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, the obstacle type of the obstacle in the left adjacent lane is not a preset dangerous type, and the moving speed of the front obstacle in the left adjacent lane is not less than the preset speed requirement; the driving state of the driver is not in the fatigue state and the distraction state. And / or, based on the lane line information in front of the vehicle, the right lane line and the right second lane line of the current lane of the vehicle are clear respectively, and the lane line length is greater than the preset length requirement, the virtual-real type of the right lane line is a dashed line; based on the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, the obstacle type of the obstacle in the right adjacent lane is not a preset dangerous type, and the moving speed of the front obstacle in the right adjacent lane is not less than the preset speed requirement; the driving state of the driver is not in the fatigue state and the distraction state.

[0163] Optionally, in some implementations, the second determination unit 4021 is further configured to determine, based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the driver, whether the preset lane changing condition is met, and determine a target lane in the two adjacent lanes on both sides of the current lane of the vehicle that meets the preset lane changing condition when the preset lane changing condition is met. The output unit 4022 is configured to output a voice prompt message indicating whether to change lanes to the target lane based on the target lane determined by the second determination unit 4021.

[0164] Optionally, in some implementations, the control module 403 is configured to instruct the determination module 401 to reacquire the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the driver of the vehicle in response to detecting a voice instruction message indicating to confirm lane changing sent by the driver within a preset time. The voice interaction module 402 is configured to determine whether the preset lane changing condition is met based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the driver of the vehicle reacquired by the determination module 401. The control module 403 is configured to control the vehicle to perform a lane changing operation based on the determination result of the voice interaction module 402 and in response to the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane of the vehicle, and the driving state of the driver of the vehicle reacquired by the determination module 401 meeting the preset lane changing condition.

[0165] Exemplary Electronic Device

[0166] In the following, reference will be made to Figure 11An electronic device according to embodiments of the present disclosure will be described. The electronic device can be either one or both of the first and second devices, or a stand-alone device independent of them, which can communicate with the first and second devices to receive the acquired input signals therefrom.

[0167] Figure 11 A block diagram of an electronic device according to embodiments of the present disclosure is illustrated. As shown, the electronic device includes one or more processors 501 and a memory 502. Figure 11

[0168] The processor 501 can be a central processing unit (CPU) or other form of processing unit having data processing and / or instruction executing capabilities, and can control other components in the electronic device to perform desired functions.

[0169] The memory 502 can include one or more computer program products that can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache memory, and / or the like. The non-volatile memory, for example, can include read-only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program instructions can be stored on the computer-readable storage media, which the processor 501 can execute to implement the vehicle lane-changing control method of the present disclosure and / or other desired functions as described above. Various contents such as input signals, signal components, noise components, and the like can also be stored in the computer-readable storage media.

[0170] In one example, the electronic device can further include an input device 503 and an output device 504, which are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0171] For example, when the electronic device is the first or second device, the input device 503 can be the microphone or microphone array described above, for capturing the input signals of the sound source. When the electronic device is a stand-alone device, the input device 503 can be a communication network connector, for receiving the acquired input signals from the first and second devices.

[0172] In addition, the input device 503 can further include, for example, a keyboard, a mouse, and the like.

[0173] The output device 504 can output various information including the determined distance information, direction information, and the like, to the outside. The output device 504 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like. ​

[0174] Of course, in order to simplify, Figure 11 Only some of the components of the electronic device related to the present disclosure are shown in FIG. 1, and components such as a bus, an input / output interface, and the like are omitted. In addition, the electronic device can further include any other appropriate components according to a specific application.

[0175] Exemplary Computer Program Product and Computer-Readable Storage Medium

[0176] In addition to the above-mentioned method and device, an embodiment of the present disclosure can be a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the vehicle lane change control method according to various embodiments of the present disclosure described in the above "Exemplary Method" section of the specification.

[0177] The computer program product can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, etc., and conventional procedural programming languages, such as the "C" programming language, or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

[0178] In addition, an embodiment of the present disclosure can also be a computer readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to perform the steps of the vehicle lane change control method according to various embodiments of the present disclosure described in the above "Exemplary Method" section of the specification.

[0179] The computer readable storage medium can take any combination of one or more of the following: a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination of the above.

[0180] The above describes the basic principles of the present disclosure in conjunction with specific embodiments, but it should be noted that the advantages, benefits, effects and the like mentioned in the present disclosure are merely examples and are not limiting, and these advantages, benefits, effects and the like cannot be considered as necessary for each embodiment of the present disclosure. In addition, the above specific details are merely for the purpose of example and understanding, and are not limiting, and the above details do not limit the present disclosure to be necessarily implemented with the above specific details.

[0181] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between each embodiment can be mutually referred to. For system embodiments, since they basically correspond to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0182] The block diagrams of the devices, apparatuses, equipment, systems involved in the present disclosure are merely exemplary examples and are not intended to require or imply the connection, arrangement, configuration shown in the block diagram. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have" and the like are open-ended words, which mean "including but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0183] The methods and apparatuses of the present disclosure can be implemented in many ways. For example, the methods and apparatuses of the present disclosure can be implemented by software, hardware, firmware, or any combination of software, hardware, firmware. The above order of steps for the method is merely for illustration, and the steps of the method of the present disclosure are not limited to the above specific description, unless otherwise specifically described. In addition, in some embodiments, the present disclosure can also be implemented as programs recorded in recording media, which include machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers the recording media storing the programs for executing the method according to the present disclosure.

[0184] It should also be noted that in the apparatuses, equipment and methods of the present disclosure, each component or step can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of the present disclosure.

[0185] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0186] The above description has been presented to enable any person skilled in the art to make or use the disclosure. Furthermore, the purpose of the above description is not intended to limit the embodiments of the present disclosure to the form disclosed herein. Although various example aspects and embodiments have been discussed above, those of ordinary skill in the art will appreciate a variety of modifications, alternatives, permutations, additions, and sub-combinations of the described aspects and embodiments.

Claims

1. A vehicle lane changing control method, comprising: in response to determining that the vehicle needs to change lanes based on obstacle information of a front obstacle in a current lane of the vehicle, acquiring lane line information in front of the vehicle, obstacle information in adjacent lanes on both sides of the current lane, and a driving state of a driver of the vehicle; in response to determining that a preset lane changing condition is met based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane, and the driving state, outputting a voice prompt message of whether to change lanes; in response to detecting a voice instruction message of confirming lane changing sent by the driver within a preset time, controlling the vehicle to perform a lane changing operation; wherein the response to detecting the voice instruction message of confirming lane changing sent by the driver within the preset time, the vehicle is controlled to perform the lane changing operation, comprising: in response to detecting the voice instruction message of confirming lane changing sent by the driver within the preset time, acquiring again the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane, and the driving state of the driver of the vehicle, the preset time being set based on a driving environment and a road type of the current lane; determining whether the preset lane changing condition is met based on the acquired again lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane, and the driving state; in response to the acquired again lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane, and the driving state meeting the preset lane changing condition, controlling the vehicle to perform the lane changing operation.

2. The method of claim 1, wherein, The obstacle information comprises: obstacle type, position and moving speed.

3. The method of any of claims 1-2, wherein, The response to determining that the vehicle needs to change lanes based on the obstacle information of the front obstacle in the current lane of the vehicle, comprising: identifying whether the obstacle information of the front obstacle meets a preset lane changing trigger condition; wherein the preset lane changing trigger condition comprises any one or more of the following: the obstacle type of the front obstacle is a preset dangerous type, and the moving speed of the front obstacle is lower than a preset speed requirement; in response to the obstacle information of the front obstacle meeting the preset lane changing trigger condition, determining that the vehicle needs to change lanes.

4. The method of any one of claims 1-3, wherein, The acquiring of the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane, and the driving state of the driver of the vehicle, comprising: determining lane line information of a preset lane line in front of the vehicle based on a lane line image acquired by a first perception acquisition module outside the vehicle; wherein the preset lane line comprises: a left lane line of the current lane, a right lane line of the current lane, a left second lane line of the current lane, and a right second lane line of the current lane; and the lane line information comprises: whether the lane line is clear, a solid-void type of the lane line, and a length of the lane line; acquiring obstacle information in the adjacent lanes on both sides of the current lane based on a driving scene image acquired by the first perception acquisition module outside the vehicle. Determine a driving state of a driver of the vehicle based on a driver image acquired by a second perception acquisition module of the vehicle interior.

5. The method of claim 4, wherein, The preset lane-changing condition comprises: a preset lane line condition, a preset obstacle condition and a preset driving state condition. The determination of the preset lane-changing condition based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides and the driving state comprises: Determine whether the preset lane line condition, the preset obstacle condition and the preset driving state condition are met based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides and the driving state. Determine that the preset lane-changing condition is met in response to the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides and the driving state meeting the preset lane line condition, the preset obstacle condition and the preset driving state condition.

6. The method of claim 5, wherein, The preset lane-changing condition further comprises: the lateral and longitudinal control function of the vehicle being in an activated state. The determination of the preset lane-changing condition based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides and the driving state meeting the preset lane line condition, the preset obstacle condition and the preset driving state condition comprises: Determine that the preset lane-changing condition is met in response to the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides and the driving state meeting the preset lane line condition, the preset obstacle condition and the preset driving state condition, and the lateral and longitudinal control function of the vehicle being in an activated state.

7. The method of any of claims 5-6, wherein, The adjacent lanes on both sides comprise a left adjacent lane and a right adjacent lane. The lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides and the driving state meeting the preset lane line condition, the preset obstacle condition and the preset driving state condition comprises: Based on the lane line information in front of the vehicle, the left lane line and the left second lane line are clear respectively, and the lane line length is greater than a preset length requirement, and the solid-void type of the left lane line is a dashed line; based on the obstacle information in the adjacent lanes on both sides, the obstacle type of the obstacle in the left adjacent lane is not a preset dangerous type, and the moving speed of the front obstacle in the left adjacent lane is not less than a preset speed requirement; the driving state is not a fatigue state and a distraction state; and / or, Based on the lane line information in front of the vehicle, the right lane line and the right second lane line are clear respectively, and the lane line length is greater than a preset length requirement, and the solid-void type of the right lane line is a dashed line; based on the obstacle information in the adjacent lanes on both sides, the obstacle type of the obstacle in the right adjacent lane is not a preset dangerous type, and the moving speed of the front obstacle in the right adjacent lane is not less than a preset speed requirement; the driving state is not a fatigue state and a distraction state.

8. The method of any one of claims 5-7, wherein, The voice prompt message of whether to change lanes is outputted when it is determined that the preset lane changing condition is met based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides, and the driving state. The target lane in the adjacent lanes on both sides of the current lane of the vehicle is determined when it is determined that the preset lane changing condition is met based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides, and the driving state. The voice prompt message of whether to change lanes to the target lane is outputted. 9.A vehicle lane changing control apparatus, comprising: a determination module configured to, when it is determined that the vehicle needs to change lanes based on obstacle information of an obstacle in front of a current lane of the vehicle, acquire lane line information in front of the vehicle, obstacle information in adjacent lanes on both sides of the current lane, and a driving state of a driver of the vehicle; a voice interaction module configured to, when it is determined that a preset lane changing condition is met based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides, and the driving state, output a voice prompt message of whether to change lanes; a control module configured to, in response to detecting a voice instruction message of confirming lane changing sent by the driver within a preset time, control the vehicle to perform a lane changing operation; wherein the control module is further configured to, in response to detecting the voice instruction message of confirming lane changing sent by the driver within the preset time, acquire again the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides of the current lane, and the driving state of the driver of the vehicle, the preset time being set based on a driving environment and a road type of the current lane, the voice interaction module is further configured to, based on the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides, and the driving state acquired again, determine whether the preset lane changing condition is met, and the control module is further configured to, in response to the lane line information in front of the vehicle, the obstacle information in the adjacent lanes on both sides, and the driving state acquired again, meeting the preset lane changing condition, control the vehicle to perform the lane changing operation. 10.A computer readable storage medium, the storage medium storing a computer program, the computer program being configured to execute the vehicle lane changing control method of any one of claims 1-8. 11.An electronic device, comprising: a processor; a memory configured to store executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the vehicle lane changing control method of any one of claims 1-8.

Images