Commercial vehicle mountainous road driving assistance system, method and equipment and medium

Abstract

The invention provides a commercial vehicle mountainous road driving assistance system, method and device and a medium, and belongs to the technical field of commercial vehicle intelligent driving assistance. The system comprises a data acquisition sensing module used for acquiring vehicle state, environment and driver information; the data processing module is used for performing fusion analysis, environmental risk assessment, vehicle stability calculation and intelligent vehicle speed prediction based on the multi-source data; the AR-HUD dynamic display module is used for fusing and displaying the generated early warning and guiding information and a road scene in an augmented reality mode; and the data interaction module is used for realizing real-time data transmission among the modules. According to the method, specific modeling is carried out on the special road condition of the mountainous area, and key safety information is visually presented in combination with AR-HUD, so that the driving safety of the commercial vehicle under the complex terrain is effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent driving assistance technology for commercial vehicles, and more specifically relates to a driving assistance system, method, device and medium for commercial vehicles on mountain roads. Background Technology

[0002] Mountain roads pose a severe challenge to vehicle safety due to their unique terrain and climate conditions. Continuous sharp bends, steep slopes, large elevation changes, and frequent low-visibility weather make the driving environment particularly complex, demanding higher levels of judgment and vehicle control from drivers. Against this backdrop, commercial vehicles, due to their large load variations and high inertia, face even more prominent safety risks, and traditional driver assistance technologies are no longer adequate to meet the safety requirements of mountainous road conditions.

[0003] Currently widely used driver assistance systems are mainly designed based on conventional parameters for flat roads, lacking targeted optimization for typical mountainous conditions. For example, in scenarios such as continuous curves and long steep downhill slopes, existing systems often struggle to accurately establish the dynamic coupling relationship between slope and load, leading to significant deviations or delays in functions such as speed guidance and braking warnings. This not only fails to provide timely and effective decision support for drivers but may also increase operational risks due to misleading prompts.

[0004] Furthermore, traditional head-up display systems typically only provide static, basic information, such as vehicle speed and simple navigation instructions, and cannot integrate critical warnings and guidance information into the actual road view. Drivers still need to frequently shift their gaze to obtain information, and in complex mountainous road sections, this visual interruption can significantly distract attention, prolong reaction time, and thus weaken the safety enhancement effect that the assistance system should provide.

[0005] While existing technologies offer some improvements to head-up display (HUD) content optimization and latency compensation, these solutions primarily address general issues under typical road conditions and do not fully consider the dynamic complexity of road conditions in mountainous environments, the real-time nature of information display, and the unique dynamic characteristics of heavy-duty vehicles. Therefore, developing a dedicated auxiliary system capable of adapting to the characteristics of mountainous roads and achieving scenario-based integrated display of critical information is of significant practical importance for improving the safety of commercial vehicles driving in mountainous areas. Summary of the Invention

[0006] To address the above problems, the present invention aims to provide a driving assistance system, method, device, and medium for commercial vehicles in mountainous areas. By integrating multi-sensor data fusion and vehicle dynamics modeling, and combining augmented reality head-up display technology, the system achieves quantitative risk assessment of complex road conditions in mountainous areas and real-time scenario-based guidance of key safety information, thereby significantly improving the safety and human-machine interaction experience of commercial vehicles when driving on mountainous roads.

[0007] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, embodiments of this application provide a commercial vehicle mountain road driving assistance system, including: a data acquisition and perception module, a data processing module, an AR-HUD dynamic display module, and a data interaction module; The data acquisition and sensing module is used to collect vehicle status information, environmental information, and driver status information using the vehicle's built-in sensors and functional components as basic data. The data processing module is used to perform multi-data fusion, AR-HUD image layout and location determination, environmental hazard level judgment, vehicle stability estimation, intelligent target vehicle speed prediction, and generate instructions and parameters for graphical display based on the basic data and through preset algorithms. The AR-HUD dynamic display module is used to receive all output instructions and parameters from the data processing module and perform augmented reality visualization rendering on a specified area on the windshield. The data interaction module is used to transmit data in real time between the data acquisition and sensing module, the data processing module and the AR-HUD dynamic display module using the controller area network bus protocol and the vehicle network protocol.

[0008] In one optional implementation, the data acquisition and perception module includes: a vehicle body sensor, a camera vision sensor, radar, a driving assistance map, and an intelligent driving assistance controller; Vehicle body sensors are used to collect real-time status data of the vehicle, including vehicle speed, longitudinal acceleration, lateral acceleration, current gear, and brake pedal status data. The camera vision sensor is used to collect visual perception data through the built-in camera of the vehicle. The visual perception data includes: images of the road ahead for extracting lane line curvature and quality, image information of objects in the front and side blind spots, and image information of the driver's eyes. Radar is used to collect point cloud data and motion data of targets using a preset elevation angle scanning mode, including the distance, relative speed, and azimuth of targets in the blind spots in front and to the side relative to the vehicle. Driving assistance maps are used to provide prior information about the road ahead, including road curvature, slope information, and curve topology. The intelligent driving assistance controller is used to provide vehicle driving assistance status information, including warning signals from the blind spot monitoring system, speed limits from traffic sign recognition, and set speeds and distances from the adaptive cruise control system.

[0009] In an optional implementation, the data processing module includes: an AR-HUD image layout and location determination unit, a multi-data fusion unit, an environmental hazard level judgment unit, a vehicle stability estimation unit, and an intelligent target vehicle speed prediction unit; The AR-HUD image layout and position determination unit is used to calculate the position and direction of the driver's eyes in the vehicle coordinate system in real time based on the driver's eye image information and through computer vision algorithms, and then dynamically determine the coordinates of the AR display area on the windshield that is adapted to the current driver's sitting posture and field of vision. The multi-data fusion unit is used to generate comprehensive target information based on the target image information and target point cloud data through timestamp alignment, coordinate system unification, and feature-level fusion algorithms, and to clearly distinguish the static and dynamic attributes of the target; the comprehensive target information includes position, velocity, and acceleration; The environmental hazard level judgment unit is used to comprehensively consider road curvature, slope information, distance and relative speed of blind spot objects relative to the vehicle, and images of the road ahead. Through a preset weighted evaluation model, it calculates the real-time comprehensive hazard coefficient of the road ahead and classifies the environmental hazard level. The vehicle stability estimation unit is used to calculate the safe speed for passing through the curve, the estimated braking distance, and the vehicle's lateral stability margin in real time based on the vehicle's real-time status data, road curvature and slope information, combined with the vehicle dynamics model and load parameters, and to determine the corresponding stability status indicator. The intelligent target speed prediction unit is used to correct the safe speed for passing through the curve based on environmental information, generate a comprehensive safe speed for the curve, and further determine the target speed by comparing it with the speed set by the current adaptive cruise control.

[0010] In an optional implementation, the AR-HUD dynamic display module is specifically used for: Project the AR display area onto the windshield based on the coordinates of the display area. Based on the target vehicle speed and compared with the current actual vehicle speed collected by the vehicle's sensors, the color of the vehicle speed display in the projection is dynamically changed to indicate the degree of speeding; based on the environmental hazard level, risk warning icons of different colors and flashing frequencies are displayed in the projection. Based on the current safe speed for passing through the curve, the estimated braking distance, the road curvature, the slope information, and the comprehensive target information provided by the multi-data fusion unit, virtual lane boundary extension lines, curve curvature indication arcs, and key braking distance indicators are superimposed and drawn on the real road view in the projection. Based on the stability status indicator, the corresponding vehicle attitude or stability status icon is displayed in the projection.

[0011] In an optional implementation, the environmental hazard level determination unit is specifically used for: Using the formula P=(KK) 安 ) / K 安 Calculate the deviation rate P; where K is the road curvature, K 安 The safety curvature threshold for commercial vehicles is defined; the road curvature hazard value W is determined based on the deviation rate P; if P≤0, W=0; if 0<P≤50%, the value of W ranges from 0.1 to 0.5; if 50%<P≤100%, the value of W ranges from 0.5 to 0.8; if P>100%, the value of W ranges from 0.8 to 1. Through formula Calculate the target proximity rate M in the blind zone; where, Where is the relative speed, and D is the distance between the target object in the blind spot and the vehicle. Through formula Calculate the slope change rate ;in The current road gradient is... The slope of the road ahead. This represents the current road segment length. Based on the image of the road ahead, the integrity score A, contrast score B, and occlusion score C of the road are determined by the image recognition algorithm; the lane quality score is calculated by the formula X=A×40%+B×30%+C×30%. The formula HW = W × 30% + M × 40% + The comprehensive risk factor HW is calculated by multiplying X by 20% and X by 10%. When 0 ≤ HW ≤ 0.2, the environmental hazard level is no risk; when 0.2 < HW ≤ 0.4, the environmental hazard level is low risk; when 0.4 < HW ≤ 0.6, the environmental hazard level is medium risk; when 0.6 < HW ≤ 0.8, the environmental hazard level is high risk; and when 0.8 < HW ≤ 1.0, the environmental hazard level is extremely high risk.

[0012] In an optional implementation, the vehicle stability estimation unit is specifically used for: The safe speed for passing the curve can be calculated using the following formula. :

[0013] Wherein, the curve radius R=1 / K, K is the road curvature, θ is the longitudinal slope angle, L is the wheelbase, h is the vehicle center of gravity height, μ is the tire-road friction coefficient, and g is the gravitational acceleration; The braking distance is calculated using the following formula. :

[0014] Where V is the vehicle's current speed. PB represents the slope compensation term, where PB is the reaction time. Component force along the road surface Correction for deceleration; Using the formula MC=(V mc -V c ) / V mc Calculate the vehicle's lateral stability margin MC; where the maximum lateral acceleration V mc = Current lateral acceleration V c = ; The stability status indicator is determined based on the range of the vehicle's lateral stability margin MC. When MC ≥ 0.4, the stability status indicator is safe; when 0.2 ≤ MC < 0.4, the stability status indicator is warning; when 0.1 ≤ MC < 0.2, the stability status indicator is danger; and when MC < 0.1, the stability status indicator is out of control.

[0015] In an optional implementation, the intelligent target vehicle speed prediction unit is specifically used for: Through formula Calculate the overall safe speed for cornering; in, The safe speed for passing through the curve at the current time is... As an environmental correction factor, For vehicle state factors. For driver state factors;

[0016]

[0017]

[0018] This is a visibility correction factor. d represents visibility; For road surface condition correction factors, , This refers to the actual tire-road friction coefficient on the road surface. Tire-road friction coefficient on dry surfaces; As a correction factor for precipitation intensity, Precipitation intensity ; For actual quality; To indicate the degree of tire wear, Driver attention score ; Through formula Calculate the overspeed difference ; The vehicle speed set for the current adaptive cruise control; If 0 < ≤ 5 km / h is considered a slight speeding violation; if 5 < ≤ 15 km / h is considered moderate speeding; if A speed of >15 km / h is considered a serious speeding violation; The target vehicle speed is calculated using the following formula:

[0019] in, For safety correction factor, When it is determined to be a minor speeding, When it is determined to be moderate speeding, When it is determined to be serious speeding, .

[0020] Secondly, embodiments of this application also provide a driving assistance method for commercial vehicles on mountain roads, including: The vehicle uses built-in sensors and functional components to collect vehicle status information, environmental information, and driver status information as basic data. Based on the basic data, the system performs multi-data fusion, AR-HUD image layout and location determination, environmental hazard level assessment, vehicle stability estimation, and intelligent target vehicle speed prediction using a preset algorithm, and generates instructions and parameters for graphical display. It receives instructions and parameters for graphical display and performs augmented reality visualization rendering on a specified area on the windshield.

[0021] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the commercial vehicle mountain road driving assistance method described in any of the above descriptions.

[0022] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the commercial vehicle mountain road driving assistance method as described in any of the above claims.

[0023] As can be seen from the above technical solutions, the present invention has the following advantages: The commercial vehicle mountain road driving assistance system provided in this application addresses the unique driving risks of commercial vehicles in complex mountain road environments. By integrating multi-source sensor information, vehicle dynamics models, and high-precision map data, it can calculate and evaluate key safety parameters such as safe cornering speed, braking distance, and vehicle lateral stability in real time and with high accuracy. This system not only achieves precise calculation and prediction of key safety parameters such as safe cornering speed, braking distance, and lateral stability, but also uses augmented reality technology to directly overlay guide lines and warning information onto the real road view, significantly reducing driver eye shifts and cognitive load. This effectively improves the intuitiveness of human-machine interaction and decision-making efficiency, ultimately providing a reliable technical solution for reducing the risk of accidents on mountain roads and enhancing the driving safety of commercial vehicles.

[0024] This application achieves accurate perception of complex driving environments in mountainous areas by integrating vehicle sensors, visual cameras, radar with a specific elevation angle scanning mode, and high-precision maps. The system not only acquires vehicle status and standard road information but also effectively captures multi-dimensional data such as steep terrain, dynamic targets in blind spots, and lane line quality changes. This provides a comprehensive and reliable data foundation for subsequent intelligent decision-making, overcoming the limitations of traditional driver assistance systems in terms of insufficient perception capabilities under non-standard road conditions.

[0025] This application establishes a dynamic risk assessment model based on multi-dimensional parameters, capable of quantitatively classifying environmental hazards. Its core algorithm integrates key factors such as road curvature deviation, blind spot target proximity rate, slope change rate, and lane marking quality, and outputs a comprehensive hazard coefficient and corresponding level in real time through weighted calculation. This refined dynamic assessment method, compared to traditional single-threshold alarms, can identify complex high-risk road sections such as continuous curves and blind spots at the crest of slopes earlier and more accurately, providing drivers with proactive decision support.

[0026] This application achieves a deep coupling analysis of vehicle dynamics characteristics and real-time road conditions. By integrating key parameters such as load, gradient, and tire-road adhesion coefficient, the system calculates safe cornering speed, braking distance, and lateral stability margin, which more closely match the actual dynamic response of commercial vehicles under different operating conditions, such as fully loaded and unloaded, in mountainous areas. This personalized stability estimation effectively avoids warning biases caused by model simplification or parameter rigidity, making safety guidance information more targeted and reliable.

[0027] This application utilizes augmented reality (AR) technology to present key safety information in an intuitive and contextualized human-computer interaction. Through the AR-HUD dynamic display module, the system accurately overlays and integrates virtual lane boundary extensions, curve guidance trajectories, and braking distance indicators into the driver's real road view. This allows the driver to directly understand the road geometry and potential risks without switching eyes or processing information internally, significantly reducing cognitive load and enhancing the intuitiveness and safety of the interaction.

[0028] This application not only provides risk warnings but also integrates vehicle status, environmental risks, and driver intent to generate and display positive guidance information such as intelligently recommended target speeds. This design elevates the driver assistance function from passive alarms to proactive collaboration, fundamentally enhancing the driver's ability to cope with complex mountain road conditions and providing an effective technical solution for improving the driving safety of commercial vehicles. Attached Figure Description

[0029] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A schematic diagram of the structure of the commercial vehicle mountain road driving assistance system provided in this application.

[0031] Figure 2 A flowchart illustrating the driving assistance method for commercial vehicles on mountain roads provided in this application.

[0032] Figure 3 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0033] The various embodiments of this disclosure will be described more fully in the following detailed description of the specific architecture and functions of the commercial vehicle mountain road driving assistance system. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0034] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a particular feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of the foregoing.

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Please see Figure 1 The diagram shown is a structural schematic of a commercial vehicle driving assistance system for mountainous roads in a specific embodiment. The system includes: a data acquisition and perception module, a data processing module, an AR-HUD dynamic display module, and a data interaction module.

[0037] The data acquisition and sensing module is used to collect vehicle status information, environmental information, and driver status information using the vehicle's built-in sensors and functional components as basic data.

[0038] In a specific implementation, the data acquisition and perception module includes: a vehicle body sensor, a camera vision sensor, radar, a driving assistance map, and an intelligent driving assistance controller.

[0039] Vehicle body sensors are used to collect real-time status data of the vehicle, including vehicle speed, longitudinal acceleration, lateral acceleration, current gear, and brake pedal status data.

[0040] The camera vision sensor is used to collect visual perception data through the built-in camera of the vehicle. The visual perception data includes: images of the road ahead for extracting lane line curvature and quality, image information of objects in the front and side blind spots, and image information of the driver's eyes.

[0041] Radar is used to collect point cloud data and motion data of targets using a preset elevation angle scanning mode, including the distance, relative speed, and azimuth angle of targets in the blind spots in front and to the side relative to the vehicle.

[0042] Driving assistance maps provide prior information about the road ahead, including road curvature, gradient information, and curve topology.

[0043] The intelligent driving assistance controller is used to provide vehicle driving assistance status information, including warning signals from the blind spot monitoring system, speed limits from traffic sign recognition, and set speeds and distances from the adaptive cruise control system.

[0044] For example, the data acquisition and perception module consists of vehicle body sensors, camera vision sensors, radar (LiDAR), a high-precision driving assistance map, and an intelligent driving assistance controller. The radar is configured with a 15° elevation scanning mode to capture steep slope terrain features. The vehicle body sensors acquire the current vehicle speed, acceleration, gear, and braking status. The camera vision sensors acquire information on changes in road curvature and lane quality, objects in front, on the roadside, and in the driver's blind spots, as well as driver eye information. The radar (LiDAR) acquires the relative distance and speed between the vehicle and objects in the front and side blind spots, as well as the azimuth and elevation angles of the objects relative to the vehicle. The high-precision driving assistance map acquires information on the road ahead. The intelligent driving assistance controller issues blind spot monitoring warnings, speed limit information, adaptive cruise control speed settings, and time-distance settings.

[0045] The data processing module is used to perform multi-source data fusion, AR-HUD image layout and location determination, environmental hazard level judgment, vehicle stability estimation, intelligent target vehicle speed prediction, and generate instructions and parameters for graphical display based on the basic data and through preset algorithms.

[0046] In a specific implementation, the data processing module includes: an AR-HUD image layout and location determination unit, a multi-data fusion unit, an environmental hazard level judgment unit, a vehicle stability estimation unit, and an intelligent target vehicle speed prediction unit.

[0047] The AR-HUD image layout and position determination unit is used to calculate the position and line of sight of the driver's eyes in the vehicle coordinate system in real time based on the driver's eye image information and through computer vision algorithms, and then dynamically determine the coordinates of the AR display area on the windshield that is adapted to the current driver's sitting posture and field of vision.

[0048] For example, the AR-HUD image layout location determination unit is specifically used for: Record the head pitch angle and yaw angle (such as the angle of head tilt and head turn) to determine the driver's head posture; Capture the three-dimensional coordinates of the center of the driver's pupils (establish a coordinate system with the center console as the origin) and filter out interference from blinking and minor head shaking. Based on head posture and binocular data, the field of view is determined by geometric fitting and physiological parameter compensation. By using coordinate transformation and boundary calculation, the display area on the windshield is locked.

[0049] The multi-data fusion unit is used to generate comprehensive target information based on the target image information and target point cloud data through timestamp alignment, coordinate system unification, and feature-level fusion algorithms, and to clearly distinguish the static and dynamic attributes of the target; the comprehensive target information includes position, velocity, and acceleration.

[0050] For example, when the data processing module receives target information acquired by the camera vision and radar sensors in the aforementioned data acquisition and perception module, this unit fuses the data to obtain more accurate target position, velocity, and acceleration, distinguishing between static and dynamic targets, and tracking them separately. This unit is specifically used for: Data preprocessing involves synchronizing the acquisition times of the camera and radar using a time protocol, mapping the data of both to the same vehicle coordinate system through calibration, and simultaneously cleaning up image fog and radar noise.

[0051] Target feature matching uses the YOLOv8 algorithm to identify the target category and appearance of the camera, a clustering algorithm to obtain the position and velocity of the radar target, and the Hungarian algorithm to calculate the spatial distance and semantic matching degree between the two. If the semantic matching degree is ≥0.7, the target is determined to be the same target.

[0052] The state fusion output integrates position and velocity according to precision weights, while distinguishing between dynamic and static targets (velocity ≥ 0.5m / s is considered dynamic, otherwise it is judged as static based on semantic matching degree).

[0053] To address the differences between dynamic and static targets, a "prediction + correction" strategy is used for tracking to avoid loss. For dynamic targets, an adaptive Kalman filter is used. First, the position and speed of the next target are predicted based on vehicle dynamics, and then adjustments are made based on newly acquired data. For static targets (with a speed of 0), the initial position is determined first. When there is no obstruction, the position is updated and errors are corrected. After the obstruction disappears, the same target is confirmed by distance (≤0.5m). If they are inconsistent, they are identified as new targets.

[0054] The environmental hazard level assessment unit is used to comprehensively consider road curvature, slope information, distance and relative speed of objects in the blind spot relative to the vehicle, and images of the road ahead. Through a preset weighted evaluation model, it calculates the real-time comprehensive hazard coefficient of the road ahead and classifies the environmental hazard level.

[0055] For example, the environmental hazard level determination unit is specifically used for: Using the formula P=(KK) 安 ) / K 安 Calculate the deviation rate P; where K is the road curvature, K 安The safety curvature threshold for commercial vehicles is defined; the road curvature hazard value W is determined based on the deviation rate P; if P≤0, W=0; if 0<P≤50%, the value of W ranges from 0.1 to 0.5; if 50%<P≤100%, the value of W ranges from 0.5 to 0.8; if P>100%, the value of W ranges from 0.8 to 1. Through formula Calculate the target proximity rate M in the blind zone; where, Where is the relative speed, and D is the distance between the target object in the blind spot and the vehicle. Through formula Calculate the slope change rate ;in The current road gradient is... The slope of the road ahead. This represents the current road segment length. Based on the image of the road ahead, the integrity score A, contrast score B, and occlusion score C of the road are determined by the image recognition algorithm; the lane quality score is calculated by the formula X=A×40%+B×30%+C×30%. The formula HW = W × 30% + M × 40% + The comprehensive risk factor HW is calculated by multiplying X by 20% and X by 10%. When 0 ≤ HW ≤ 0.2, the environmental hazard level is no risk; when 0.2 < HW ≤ 0.4, the environmental hazard level is low risk; when 0.4 < HW ≤ 0.6, the environmental hazard level is medium risk; when 0.6 < HW ≤ 0.8, the environmental hazard level is high risk; and when 0.8 < HW ≤ 1.0, the environmental hazard level is extremely high risk.

[0056] The vehicle stability estimation unit is used to calculate the safe speed for passing through the current curve, the estimated braking distance, and the vehicle's lateral stability margin in real time, based on the vehicle's real-time status data, road curvature and slope information, combined with the vehicle dynamics model and load parameters, and to determine the corresponding stability status indicator.

[0057] For example, a vehicle stability estimation unit is specifically used for: The safe speed for passing the curve can be calculated using the following formula. :

[0058] Wherein, the curve radius R=1 / K, K is the road curvature, θ is the longitudinal slope angle, L is the wheelbase, h is the vehicle center of gravity height, μ is the tire-road friction coefficient, and g is the gravitational acceleration; The braking distance is calculated using the following formula. :

[0059] Where V is the vehicle's current speed. PB represents the slope compensation term, where PB is the reaction time. Component force along the road surface Correction for deceleration; Using the formula MC=(V mc -V c ) / V mc Calculate the vehicle's lateral stability margin MC; where the maximum lateral acceleration V mc = Current lateral acceleration V c = ; The stability status indicator is determined based on the range of the vehicle's lateral stability margin MC. When MC ≥ 0.4, the stability status indicator is safe; when 0.2 ≤ MC < 0.4, the stability status indicator is warning; when 0.1 ≤ MC < 0.2, the stability status indicator is danger; and when MC < 0.1, the stability status indicator is out of control.

[0060] The intelligent target speed prediction unit is used to correct the safe speed for passing through the curve based on environmental information, generate a comprehensive safe speed for the curve, and further determine the target speed by comparing it with the speed set by the current adaptive cruise control.

[0061] For example, the intelligent target vehicle speed prediction unit is specifically used for: Through formula Calculate the overall safe speed for cornering; in, The safe speed for passing through the curve at the current time is... As an environmental correction factor, For vehicle state factors. For driver state factors;

[0062]

[0063]

[0064] This is a visibility correction factor. d represents visibility; For road surface condition correction factors, , This refers to the actual tire-road friction coefficient on the road surface. Tire-road friction coefficient on dry surfaces; As a correction factor for precipitation intensity, Precipitation intensity ; For actual quality; To indicate the degree of tire wear, Driver attention score ; Through formula Calculate the overspeed difference ; The vehicle speed set for the current adaptive cruise control; If 0 < ≤ 5 km / h is considered a slight speeding violation; if 5 < ≤ 15 km / h is considered moderate speeding; if A speed of >15 km / h is considered a serious speeding violation; The target vehicle speed is calculated using the following formula:

[0065] in, For safety correction factor, When it is determined to be a minor speeding, When it is determined to be moderate speeding, When it is determined to be serious speeding, .

[0066] The AR-HUD dynamic display module is used to receive all output instructions and parameters from the data processing module and perform augmented reality visualization rendering on a specified area on the windshield.

[0067] In a specific implementation, the AR-HUD dynamic display module is used for: Project the AR display area onto the windshield based on the coordinates of the display area. Based on the target vehicle speed and compared with the current actual vehicle speed collected by the vehicle's sensors, the color of the vehicle speed display in the projection is dynamically changed to indicate the degree of speeding; based on the environmental hazard level, risk warning icons of different colors and flashing frequencies are displayed in the projection. Based on the current safe speed for passing through the curve, the estimated braking distance, the road curvature, the slope information, and the comprehensive target information provided by the multi-data fusion unit, virtual lane boundary extension lines, curve curvature indication arcs, and key braking distance indicators are superimposed and drawn on the real road view in the projection. Based on the stability status indicator, the corresponding vehicle attitude or stability status icon is displayed in the projection.

[0068] For example, after the data processing module sends the results to the AR-HUD dynamic display module via the data interaction module, it is displayed in a hierarchical, contextualized, and personalized manner. The presented image information includes: current vehicle speed, comprehensive safe vehicle speed, intelligent target vehicle speed, current gear, curve radius, vehicle stability status, and braking distance. The specific implementation process is as follows: Step 1: Obtain the image display position signal based on the driver's sitting posture and eye position, and display the image at a reasonable position on the windshield.

[0069] Step 2: Obtain the current vehicle speed, the degree of speeding difference, and the intelligent target vehicle speed.

[0070] (a) If the current speed is a safe speed, the current speed sign will be green; (b) If the current speed is slightly over the limit, the current speed sign will be displayed in yellow; (c) If the current speed is moderately excessive, the current speed sign will be displayed in orange; (d) If the current speed is seriously exceeding the speed limit, the current speed sign will be displayed as a flashing red sign.

[0071] Step 3: After obtaining the current curve radius, intelligent target speed, and braking distance, virtual lane lines are displayed, and the curve radius ahead, as well as intelligent target speed and braking distance prompts, are shown in real time.

[0072] Step 4: Once the vehicle stability status is obtained.

[0073] (a) If lateral stability is safe, the stability status indicator is green; (b) If lateral stability is in warning condition, the stability status indicator will be displayed in yellow; (c) If lateral stability is dangerous, the stability status indicator will be displayed in orange; (d) If lateral stability is out of control, the stability status indicator will flash red.

[0074] Step 4: Obtain different environmental hazard levels.

[0075] (a) If the environmental hazard level is no risk, the environmental hazard level sign will be displayed in green; (b) If the environmental hazard level is low risk, the environmental hazard level sign will be displayed in yellow; (c) If the environmental hazard level is medium risk, the environmental hazard level sign shall be displayed in orange. (d) If the environmental hazard level is high risk, the environmental hazard level sign will be displayed in orange and flash rapidly; (e) If the environmental hazard level is extremely high, the environmental hazard level sign will be displayed in red in the center of the image area to warn the driver.

[0076] The data interaction module is used to transmit data in real time between the data acquisition and sensing module, the data processing module and the AR-HUD dynamic display module using the controller area network bus protocol and the vehicle network protocol.

[0077] This module is responsible for data communication between various modules in the system. It adopts automotive bus protocols (such as CAN protocol) and hard-wired connections to ensure stable and real-time data transmission between sensors, controllers and display terminals.

[0078] In this embodiment, real-time quantitative assessment of the environmental risks of mountain roads is achieved through multi-source data fusion and dynamic modeling. By using AR-HUD fusion display technology, key information such as curve guide lines and braking distance prompts are accurately superimposed on the real road scene, which significantly reduces the driver's cognitive load and the frequency of eye shifts, thereby effectively improving the driving safety and operational reliability of vehicles in complex terrain.

[0079] like Figure 2 As shown, the following are embodiments of the commercial vehicle mountain road driving assistance method provided in this disclosure. This method and the commercial vehicle mountain road driving assistance system in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the commercial vehicle mountain road driving assistance method, please refer to the embodiments of the commercial vehicle mountain road driving assistance system described above.

[0080] A driving assistance method for commercial vehicles on mountain roads includes the following steps: S1: Use the vehicle's built-in sensors and functional components to collect vehicle status information, environmental information, and driver status information as basic data.

[0081] S2: Based on the basic data, the system performs multi-data fusion, AR-HUD image layout and location determination, environmental hazard level judgment, vehicle stability estimation, and intelligent target vehicle speed prediction using a preset algorithm, and generates instructions and parameters for graphical display.

[0082] S3: Receives instructions and parameters for graphical display and performs augmented reality visualization rendering on the specified area of ​​the windshield.

[0083] The commercial vehicle mountain road driving assistance method provided in this embodiment integrates multi-source sensor data and high-precision map information to construct a dedicated risk assessment model and vehicle dynamics analysis model for complex road conditions in mountainous areas. By utilizing augmented reality head-up display technology, the calculated key information such as safe speed, braking distance, curve guide lines, and risk level are fused and projected into the actual road field of view in real time and accurately. This significantly reduces the driver's cognitive load and the frequency of eye shifts, while achieving proactive early warning and guidance for driving risks in mountainous areas, effectively improving the driving safety and operational reliability of commercial vehicles in special terrain environments.

[0084] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0085] The commercial vehicle mountain road driving assistance method provided in this application embodiment can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0086] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0087] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0088] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0089] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0090] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.

[0091] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0092] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0093] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0094] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0095] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.

[0096] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0097] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0098] A display screen is used to display images, videos, etc. A display screen includes a display panel.

[0099] The aforementioned electronic device realizes the commercial vehicle mountain road driving assistance method of this application through multi-source perception and fusion technology, a dedicated risk and vehicle dynamics analysis model for mountain roads, and a contextual information fusion display method based on AR-HUD. It achieves the beneficial effect of upgrading traditional passive warning to real-time quantitative assessment and proactive guidance of complex driving risks in mountainous areas, thereby significantly improving driving safety and interactive experience.

[0100] The storage medium provided in this application stores a program product capable of implementing a driving assistance method for commercial vehicles on mountain roads.

[0101] Commercial vehicle driving assistance methods for mountain roads include: The vehicle uses built-in sensors and functional components to collect vehicle status information, environmental information, and driver status information as basic data. Based on the basic data, the system performs multi-data fusion, AR-HUD image layout and location determination, environmental hazard level assessment, vehicle stability estimation, and intelligent target vehicle speed prediction using a preset algorithm, and generates instructions and parameters for graphical display. It receives instructions and parameters for graphical display and performs augmented reality visualization rendering on a specified area on the windshield.

[0102] In some possible implementations, the commercial vehicle mountain road driving assistance method of this disclosure can be implemented as a program product that includes program code, which, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of this disclosure.

[0103] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0104] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A driving assistance system for commercial vehicles on mountain roads, characterized in that, include: The system comprises a data acquisition and sensing module, a data processing module, an AR-HUD dynamic display module, and a data interaction module. The data acquisition and sensing module is used to collect vehicle status information, environmental information, and driver status information using the vehicle's built-in sensors and functional components as basic data. The data processing module is used to perform multi-data fusion, AR-HUD image layout and location determination, environmental hazard level judgment, vehicle stability estimation, intelligent target vehicle speed prediction, and generate instructions and parameters for graphical display based on the basic data and through preset algorithms. The AR-HUD dynamic display module is used to receive all output instructions and parameters from the data processing module and perform augmented reality visualization rendering on a specified area on the windshield. The data interaction module is used to transmit data in real time between the data acquisition and sensing module, the data processing module and the AR-HUD dynamic display module using the controller area network bus protocol and the vehicle network protocol.

2. The commercial vehicle mountain road driving assistance system according to claim 1, characterized in that, The data acquisition and perception module includes: a vehicle body sensor, a camera vision sensor, radar, a driving assistance map, and an intelligent driving assistance controller; Vehicle body sensors are used to collect real-time status data of the vehicle, including vehicle speed, longitudinal acceleration, lateral acceleration, current gear, and brake pedal status data. The camera vision sensor is used to collect visual perception data through the built-in camera of the vehicle. The visual perception data includes: images of the road ahead for extracting lane line curvature and quality, image information of objects in the front and side blind spots, and image information of the driver's eyes. Radar is used to collect point cloud data and motion data of targets using a preset elevation angle scanning mode, including the distance, relative speed, and azimuth of targets in the blind spots in front and to the side relative to the vehicle. Driving assistance maps are used to provide prior information about the road ahead, including road curvature, slope information, and curve topology. The intelligent driving assistance controller is used to provide vehicle driving assistance status information, including warning signals from the blind spot monitoring system, speed limits from traffic sign recognition, and set speeds and distances from the adaptive cruise control system.

3. The commercial vehicle mountain road driving assistance system according to claim 2, characterized in that, The data processing module includes: an AR-HUD image layout and location determination unit, a multi-data fusion unit, an environmental hazard level judgment unit, a vehicle stability estimation unit, and an intelligent target vehicle speed prediction unit; The AR-HUD image layout and position determination unit is used to calculate the position and direction of the driver's eyes in the vehicle coordinate system in real time based on the driver's eye image information and through computer vision algorithms, and then dynamically determine the coordinates of the AR display area on the windshield that is adapted to the current driver's sitting posture and field of vision. The multi-data fusion unit is used to generate comprehensive target information based on the target image information and target point cloud data through timestamp alignment, coordinate system unification, and feature-level fusion algorithms, and to clearly distinguish the static and dynamic attributes of the target; the comprehensive target information includes position, velocity, and acceleration; The environmental hazard level judgment unit is used to comprehensively consider road curvature, slope information, distance and relative speed of blind spot objects relative to the vehicle, and images of the road ahead. Through a preset weighted evaluation model, it calculates the real-time comprehensive hazard coefficient of the road ahead and classifies the environmental hazard level. The vehicle stability estimation unit is used to calculate the safe speed for passing through the curve, the estimated braking distance, and the vehicle's lateral stability margin in real time based on the vehicle's real-time status data, road curvature and slope information, combined with the vehicle dynamics model and load parameters, and to determine the corresponding stability status indicator. The intelligent target speed prediction unit is used to correct the safe speed for passing through the curve based on environmental information, generate a comprehensive safe speed for the curve, and further determine the target speed by comparing it with the speed set by the current adaptive cruise control.

4. The commercial vehicle mountain road driving assistance system according to claim 3, characterized in that, The AR-HUD dynamic display module is specifically used for: Project the AR display area onto the windshield based on the coordinates of the display area. Based on the target vehicle speed, and compared with the current actual vehicle speed collected by the vehicle body sensors, the color of the vehicle speed display in the projection is dynamically changed to indicate the degree of speeding. Based on the environmental hazard level, risk warning icons of different colors and flashing frequencies are displayed on the projection. Based on the current safe speed for passing through the curve, the estimated braking distance, the road curvature, the slope information, and the comprehensive target information provided by the multi-data fusion unit, virtual lane boundary extension lines, curve curvature indication arcs, and key braking distance indicators are superimposed and drawn on the real road view in the projection. Based on the stability status indicator, the corresponding vehicle attitude or stability status icon is displayed in the projection.

5. The commercial vehicle mountain road driving assistance system according to claim 3, characterized in that, The environmental hazard level determination unit is specifically used for: Using the formula P=(KK) 安 ) / K 安 Calculate the deviation rate P; where K is the road curvature, K 安 The safety curvature threshold for commercial vehicles is defined; the road curvature hazard value W is determined based on the deviation rate P; if P≤0, W=0; if 0<P≤50%, the value of W ranges from 0.1 to 0.5; if 50%<P≤100%, the value of W ranges from 0.5 to 0.8; if P>100%, the value of W ranges from 0.8 to 1. Through formula Calculate the target proximity rate M in the blind zone; where, Where is the relative speed, and D is the distance between the target object in the blind spot and the vehicle. Through formula Calculate the slope change rate ;in The current road gradient is... The slope of the road ahead. This represents the current road segment length. Based on the image of the road ahead, the integrity score A, contrast score B, and occlusion score C of the road are determined by the image recognition algorithm; the lane quality score is calculated by the formula X=A×40%+B×30%+C×30%. The formula HW = W × 30% + M × 40% + The comprehensive risk factor HW is calculated by multiplying X by 20% and X by 10%. When 0 ≤ HW ≤ 0.2, the environmental hazard level is no risk; when 0.2 < HW ≤ 0.4, the environmental hazard level is low risk; when 0.4 < HW ≤ 0.6, the environmental hazard level is medium risk; when 0.6 < HW ≤ 0.8, the environmental hazard level is high risk; and when 0.8 < HW ≤ 1.0, the environmental hazard level is extremely high risk.

6. The commercial vehicle mountain road driving assistance system according to claim 3, characterized in that, The vehicle stability estimation unit is specifically used for: The safe speed for passing the curve can be calculated using the following formula. : Wherein, the curve radius R=1 / K, K is the road curvature, θ is the longitudinal slope angle, L is the wheelbase, h is the vehicle center of gravity height, μ is the tire-road friction coefficient, and g is the gravitational acceleration; The braking distance is calculated using the following formula. : Where V is the vehicle's current speed. PB represents the slope compensation term, where PB is the reaction time. Component force along the road surface Correction for deceleration; Using the formula MC=(V mc -V c ) / V mc Calculate the vehicle's lateral stability margin MC; where the maximum lateral acceleration V mc = Current lateral acceleration V c = ; The stability status indicator is determined based on the range of the vehicle's lateral stability margin MC. When MC ≥ 0.4, the stability status indicator is safe; when 0.2 ≤ MC < 0.4, the stability status indicator is warning; when 0.1 ≤ MC < 0.2, the stability status indicator is danger; and when MC < 0.1, the stability status indicator is out of control.

7. The commercial vehicle mountain road driving assistance system according to claim 3, characterized in that, The intelligent target vehicle speed prediction unit is specifically used for: Through formula Calculate the overall safe speed for cornering; in, The safe speed for passing through the curve at the current time is... As an environmental correction factor, For vehicle state factors. For driver state factors; This is a visibility correction factor. d represents visibility; For road surface condition correction factors, , This refers to the actual tire-road friction coefficient on the road surface. Tire-road friction coefficient on dry surfaces; As a correction factor for precipitation intensity, Precipitation intensity ; For actual quality; To indicate the degree of tire wear, Driver attention score ; Through formula Calculate the overspeed difference ; The vehicle speed set for the current adaptive cruise control; If 0 < If the speed is ≤ 5 km / h, it is considered a slight speeding; if 5 < ≤ 15 km / h is considered moderate speeding; if A speed of >15 km / h is considered a serious speeding violation; The target vehicle speed is calculated using the following formula: in, For safety correction factor, When it is determined to be a minor speeding, When it is determined to be moderate speeding, When it is determined to be serious speeding, .

8. A driving assistance method for commercial vehicles on mountain roads, characterized in that, The system employs the commercial vehicle mountain road driving assistance system as described in any one of claims 1 to 7; The method includes: The vehicle uses built-in sensors and functional components to collect vehicle status information, environmental information, and driver status information as basic data. Based on the basic data, the system performs multi-data fusion, AR-HUD image layout and location determination, environmental hazard level assessment, vehicle stability estimation, and intelligent target vehicle speed prediction using a preset algorithm, and generates instructions and parameters for graphical display. It receives instructions and parameters for graphical display and performs augmented reality visualization rendering on a specified area on the windshield.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the commercial vehicle mountain road driving assistance method as described in claim 8.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the commercial vehicle mountain road driving assistance method as described in claim 8.

Images