HMI Display Method for Obstacle, Lane Line, and Planned Path Information Based on Ethernet Communication

Abstract

This invention discloses an HMI display method for obstacle, lane line, and planned path information based on Ethernet communication. The method includes: pre-defining Ethernet signals for lane line information, obstacle information, and intelligent navigation assistance system (ADAS) status; sending lane line information detected and identified by the ADAS's forward-looking camera to the ICM using pre-defined signals; sending surrounding obstacle information detected and identified by the ADAS's radar camera to the ICM using pre-defined signals; and sending curve-related attribute information from the intelligent navigation assistance system's path planning to the ICM using pre-defined signals. When the vehicle's infotainment system receives these signals from the ADAS, it displays the obstacles, lane lines, and planned path on the HMI based on the information contained in these signals. This display method can meet the display requirements of automated driving assistance systems.

Description

Technical Field

[0001] This invention relates to the field of automotive electronics technology, and in particular to an HMI display method for obstacle, lane line and planned route information based on Ethernet communication, which can display lane lines, obstacles, route specifications and other information on the instrument interface (HMI) during vehicle operation. Background Technology

[0002] With the continuous development of automotive electronics technology, Navigate on Autopilot (NOA) is being used more and more in mid-to-high-end vehicle models. As a new technology leading the trend of automotive intelligence, autonomous driving is changing the way people travel.

[0003] As a feature of fully autonomous driving, the vehicle's automatic assisted navigation driving system not only brings drivers a brand-new driving experience, but also combines "navigation" and "assisted driving". On the basis of the original L2 assisted driving, such as lane keeping and automatic following, it adds map navigation information from the vehicle's infotainment system to achieve automatic lane changing and autonomous driving from point X to point Y.

[0004] Automatic navigation-assisted driving systems make the driving experience easier and more convenient for users. More importantly, automatic navigation-assisted driving systems can help drivers drive the vehicle more safely and efficiently, fully meeting users' needs for a safe and convenient experience. Summary of the Invention

[0005] The purpose of this invention is to provide an HMI display method for obstacle, lane line and planned path information based on Ethernet communication, so as to meet the display requirements of automatic assisted navigation driving system.

[0006] To address this, the present invention provides an HMI display method for obstacle, lane, and planned path information based on Ethernet communication, comprising: pre-defining Ethernet signals for lane information, obstacle information, and intelligent navigation assistance system status; sending lane information detected and identified by the ADAS's onboard forward-looking camera to the ICM using pre-defined signals, sending surrounding obstacle information detected and identified by the ADAS's radar camera to the ICM using pre-defined signals, and sending curve-related attribute information of the intelligent navigation assistance system's path planning to the ICM using pre-defined signals; and displaying obstacles, lanes, and planned paths on the HMI based on the information contained in these signals when the vehicle's infotainment system receives the lane information signals, obstacle information signals, and curve-related attribute information signals from the ADAS.

[0007] This invention predefines Ethernet signals for lane line information, obstacle information, and intelligent navigation assistance system status. It then collects lane line information, surrounding obstacle information, and curve-related attribute information of the intelligent navigation assistance system's path planning and sends them to the vehicle's infotainment system. The ICM then displays the obstacles, lane lines, and planned path in real time on the HMI, thereby meeting the display requirements of the automatic assisted navigation driving system for the external driving environment.

[0008] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0009] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0010] Figure 1 This is a flowchart of an HMI display method for obstacle, lane line and planned path information based on Ethernet communication proposed in an embodiment of the present invention;

[0011] Figure 2 This is a system architecture for an HMI display method for obstacle, lane line and planned path information proposed in an embodiment of the present invention. Detailed Implementation

[0012] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0013] This invention provides an HMI display method for obstacle, lane line, and planned route information. The flow of an embodiment of this display method is as follows: Figure 1 Ethernet network topology communication logic is as follows: Figure 2 .

[0014] like Figure 1 As shown, the display method includes the following steps S1-S3.

[0015] S1, Ethernet signals for predefined lane line information, Ethernet signals for obstacle information, and Ethernet signals for the status of the intelligent navigation assistance system.

[0016] In some embodiments, the lane line information contained in the predefined lane line information signal includes the following member variables and assignments: lane line curvature equation parameters, type, lane line quality, lane line color, lane line width, lane near end point, lane far end point, left and right lane differentiation information, and lane line detection delay.

[0017] In some embodiments, the obstacle information contained in the predefined obstacle information signal includes the following member variables and their values: the target's position coordinates and its lateral / longitudinal direction, the target vehicle type, the target's current lane, the target's relative speed (lateral / longitudinal direction), the target's tail width, the target's tail height, the target's validity, and the target's detection delay.

[0018] In some embodiments, the predefined intelligent navigation assistance system state Ethernet signal contains curve-related attribute information including the following member variables and assignments: planned path confidence, planned path length, and planned path curvature equation parameters.

[0019] S2. The lane line information detected and identified by the ADAS vehicle-mounted forward-looking camera is sent to the ICM in a predefined signal. The surrounding obstacle information detected and identified by the ADAS radar camera is sent to the ICM in a predefined signal. The curve-related attribute information of the intelligent navigation assistance system path planning is sent to the ICM in a predefined signal.

[0020] In some embodiments, lane line information is sent via the Event_ObjectInfo API signal in the form of ArrFix_LaneInfo / Struct_LaneInfo messages; obstacle information is sent via the Event_LaneInfo API signal in the form of ArrFix_ObjectInfo / Struct_ObjectInfo messages; and curve-related attribute information is sent via the Event_PlanningPath API signal in the form of Struct_PlanningPath messages.

[0021] S3. When the vehicle infotainment system receives lane line information signals, obstacle information signals, and curve-related attribute information signals from ADAS, it displays the obstacles, lane lines, and planned path on the HMI based on the information contained in these signals.

[0022] Lane detection and recognition are prerequisites for the intelligent navigation assistance system to enable path planning. When lane lines are not detected, the intelligent navigation assistance system is prohibited from enabling path planning, path planning information is not sent out, and the planned path is not displayed on the HMI.

[0023] In some embodiments, the lane lines displayed on the HMI are generated in real time by the lane line curvature equation built into the ICM based on the variable members and value values ​​contained in the lane line information signal it receives.

[0024] In some embodiments, the planned path displayed on the HMI is generated in real time by the planning path curvature equation built into the ICM based on the variable members and value values ​​contained in the received curve-related attribute information.

[0025] In some embodiments, the lane lines displayed on the HMI can provide information about a vehicle deviating from its lane to the lane departure warning system and / or lane departure mitigation system.

[0026] In some embodiments, obstacles displayed on the HMI are used to provide the adaptive cruise control system and / or forward collision warning system with accurate location information of surrounding vehicles.

[0027] This invention predefines Ethernet signals for lane line information, obstacle information, and intelligent navigation assistance system status. It then collects lane line information, surrounding obstacle information, and curve-related attribute information of the intelligent navigation assistance system's path planning and sends them to the vehicle's infotainment system. The ICM then displays the obstacles, lane lines, and planned path in real time on the HMI, thereby meeting the display requirements of the automatic assisted navigation driving system for the external driving environment.

[0028] The following is combined Figure 2 This paper provides an example to illustrate the display method and its system architecture.

[0029] This display method first involves: the design of lane line information Ethernet signals, the design of obstacle information Ethernet signals, and the design of intelligent navigation assistance system (NOA system, Navigate on Autopilot) status Ethernet signals.

[0030] The Ethernet service name ADASDisplay includes three API signals: Event_ObjectInfo, Event_LaneInfo, and Event_PlanningPath.

[0031] Lane information is detected and identified by the front camera module (FCM) of the vehicle. The information includes the curvature equation, type, quality, color, width, near and far ends of the lane, left and right lane distinction information, and lane detection delay. The information is sent to the vehicle's infotainment system and displayed in the instrument cluster module (ICM).

[0032] This lane information system, combined with Lane Departure Warning (LDW) and Lane Departure Prohibitory (LDP) systems, allows the LDW system to alert the driver (through warnings such as steering wheel vibration or audible alerts from the microphone) and instrument panel displays when the vehicle is likely to deviate from its lane. The LDP system assists with steering maneuvers to help the driver return the vehicle to its lane and avoid danger. When the vehicle deviates significantly from its lane, the system applies the brakes to prevent further lane departure. This significantly reduces collisions caused by lane departure.

[0033] The lane marking information is displayed on the dashboard as follows:

[0034] 1. The forward-facing camera (FCM) monitors the lane line status, and the ADAS periodically sends lane line information signals through the Event_LaneInfo API signal in the form of ArrFix_LaneInfo messages;

[0035] 2. The vehicle's infotainment system receives lane information signals from the ADAS and displays the real-time status of the lanes on the instrument panel.

[0036] Lane information (Event_LaneInfo) is transmitted via Ethernet using an Array data type. For example, ADAS hardware supports a maximum of X lanes, while highways typically support a maximum of 4 lanes (X≥4). The content includes the lane line's curvature equation parameters, type, lane line quality, color, distance between near and far endpoints, width, position, and lane line detection delay.

[0037] The equation for lane curvature is as follows:

[0038] f(x) = ax 3 +bx 2 +cx+d

[0039] The parameters of the lane curvature equation for Ethernet transmission are as follows:

[0040] a is the lane curvature parameter CurvatureRateParameterC3, b is the lane curvature parameter CurvatureRateParameterC2, c is the lane angle parameter HeadingParameterC1, and d is the lane lateral displacement PositionParameterC0.

[0041] The longitudinal distance from the near endpoint (ClosePointX), the longitudinal distance from the far endpoint (FarPointX), and the lane marking width (LaneMarkingWidth) are used to depict the position and width of the lane lines.

[0042] The API signals are defined in detail in the table below.

[0043] Table 1 Lane Line Information Ethernet Signal

[0044]

[0045]

[0046] Obstacle information is information about surrounding obstacles detected and identified by radar cameras. The content includes the target's position coordinates (lateral / longitudinal), target vehicle type, target's current lane, target's relative speed (lateral / longitudinal), target's rear width, target's rear height, target validity, and target detection delay. This information is sent to the vehicle's infotainment system for reception and displayed on the instrument panel.

[0047] This obstacle information system, combined with the Forward Collision Warning (FCW) and Adaptive Cruise Control (ACC) systems, uses radar cameras to obtain the accurate positions of surrounding vehicles. When there is a potential collision hazard, the FCW system will warn the driver, and the ACC system will send an execution signal to the engine or braking system to reduce the vehicle speed when it detects that the vehicle in front is slowing down. This not only allows the vehicle to maintain a certain speed, but also automatically adjusts the speed according to the distance to the vehicle in front to ensure the optimal safe distance, and avoids the need to frequently cancel and set cruise control.

[0048] The obstacle information is displayed on the dashboard as follows:

[0049] 1. The radar camera monitors and identifies obstacles around the vehicle. The ADAS periodically sends obstacle signals through the Event_ObjectInfo API signal in the form of ArrFix_ObjectInfo messages.

[0050] 2. The vehicle's infotainment system receives obstacle information signals from the ADAS and displays the real-time status of obstacles on the instrument panel.

[0051] Obstacle information (Event_ObjectInfo) is transmitted via Ethernet in Array format. The ADAS hardware supports detecting a maximum of Y obstacles, while the instrument uses only Z obstacle information (Y≥Z). The content includes the target obstacle's position (longitudinal / lateral) coordinates, type, current lane, relative speed (lateral / longitudinal), rear width, rear height, validity, and target detection delay.

[0052] The API signals are defined in detail in the table below.

[0053] Table 2 Obstacle Information Ethernet Signal

[0054]

[0055]

[0056] ADAS sends the curve-related attribute information of the trajectory planning to the vehicle's infotainment system, which then displays it on the instrument panel. The information includes the confidence level of the planned path, the length of the planned path, and the curvature equation parameters of the planned path.

[0057] The route planning information is displayed on the dashboard as follows:

[0058] 1. ADAS periodically sends out curve-related attribute information for path planning via the Event_ObjectInfo API signal in the form of a Struct_PlanningPath message;

[0059] 2. The vehicle's infotainment system receives the path planning curve-related attribute signals from the ADAS and displays the real-time status of the path planning on the instrument panel.

[0060] The path planning information (Event_PlanningPath) is transmitted in Struct format via Ethernet. It includes the planned path confidence, planned path length, and parameters of the planned path curvature equation.

[0061] For example, when NOA is activated, the lane for this vehicle is highlighted in green. The trajectory planning curve displays the guide line in front of the vehicle. When NOA is activated and there is no lane change, the guide line is forward in the current lane. When there is a lane change, the guide line will change to the corresponding lane.

[0062] The equation for the curvature of the planned path is the same as the equation for the lane lines:

[0063] f(x) = ax 3 +bx 2 +cx+d

[0064] Where a is the curvature parameter CurvatureRateParatemerC3 of the planned path, b is the curvature parameter CurvatureRateParatemerC2 of the planned path, c is the angle parameter HeadingParmeterC1 between the planned path and the x-axis, and d is the lateral displacement PositionParameterC0 of the planned path.

[0065] When lane lines are detected, the DisplayLength of the planned path is the same as the length of the lane lines. When lane lines are not detected, NOA cannot be enabled, there is no path planning function, path planning information is not sent out, and the planned path is not displayed.

[0066] The API signals are defined in detail in the table below.

[0067] Table 3. Status of Intelligent Navigation Assist System (Ethernet Signal)

[0068]

[0069]

[0070] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0071] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0072] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0073] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0074] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for displaying obstacle, lane line, and planned path information via Ethernet communication in an HMI (Hardware Management Interface), characterized by... It lies in, include: Ethernet signals for predefined lane line information, obstacle information, and intelligent navigation assistance system status; The lane line information detected and identified by the ADAS's onboard forward-looking camera is sent to the ICM in a predefined signal. The surrounding obstacle information detected and identified by the ADAS's radar camera is sent to the ICM in a predefined signal. The curve-related attribute information of the intelligent navigation assistance system's path planning is sent to the ICM in a predefined signal. When the vehicle infotainment system receives lane line information signals, obstacle information signals, and curve-related attribute information signals from ADAS, it displays obstacles, lane lines, and planned paths on the HMI based on the information contained in these signals. The lane line information signal predefined in the predefined lane line information signal includes the following member variables and their values: The curvature equation parameters, type, quality, color, width, near and far endpoints, left and right lane distinction information, and lane detection delay of lane lines; The lane lines displayed on the HMI are generated in real time by the lane line curvature equation built into the ICM based on the variable members and values ​​contained in the lane line information signal it receives. The obstacle information contained in the predefined obstacle information signal includes the following member variables and their values: The target's location coordinates and its lateral / longitudinal coordinates, target vehicle type, target's current lane, target's lateral / longitudinal relative speed, target's tail width, target's tail height, target validity, and target detection delay; The predefined intelligent navigation assistance system state Ethernet signal contains curve-related attribute information, including the following member variables and their values: Path confidence, path length, and path curvature equation parameters; The planned path displayed on the HMI is generated in real time by the curvature equation of the planned path built into the ICM, based on the variable members and values ​​contained in the received curve-related attribute information.

2. The HMI display method for obstacle, lane line, and planned path information based on Ethernet communication according to claim 1, characterized in that, The lane line information is sent via the Event_ObjectInfo API signal in the form of ArrFix_LaneInfo / Struct_LaneInfo messages. The obstacle information is sent via the Event_LaneInfo API signal in the form of ArrFix_ObjectInfo / Struct_ObjectInfo messages; The curve-related attribute information is sent via the Event_PlanningPath API signal in the form of a Struct_PlanningPath message.

3. The HMI display method for obstacle, lane line, and planned path information based on Ethernet communication according to claim 1, characterized in that, The lane lines displayed on the HMI are used to provide information about the vehicle deviating from its lane to the lane departure warning system and / or lane departure mitigation system.

4. The HMI display method for obstacle, lane line, and planned path information based on Ethernet communication according to claim 1, characterized in that, The obstacles displayed on the HMI are used to provide the adaptive cruise control system and / or forward collision warning system with accurate location information of surrounding vehicles.

5. The HMI display method for obstacle, lane line, and planned path information based on Ethernet communication according to claim 1, characterized in that, The detection and recognition of lane lines is a prerequisite for the intelligent navigation assistance system to start path planning. When lane lines are not detected, the intelligent navigation assistance system is prohibited from starting path planning, path planning information is not sent out, and the planned path is not displayed on the HMI.

Images