METHOD FOR ADAPTIVE DRIVER MONITORING FOR PATH-BASED AUTOMATIC DRIVING AID

Abstract

Various embodiments provide methods, systems, and vehicle devices. Methods for implementing phases of steering override control in a vehicle using a deep neural network (DNN) include: receiving a set of vehicle inputs, including lane data, vehicle sensor interpretation information, and driver inputs, through a steering assistance unit located in the vehicle; configuring a mission planning module to determine a desired vehicle path; and configuring a vehicle path prediction module to determine a set of predicted vehicle dynamics.Configuring a driver override determination module in communication with the DNN to override the automatic steering control torque based on a phase of the steering control operation and by information of a corresponding intervention phase classified in the DNN, and configuring a side control module to override the steering control torque for automatic steering assistance based on a specific phase of the steering control operation and driver input such as the driver steering torque and torque rate.

Description

INTRODUCTION

[0001] The technical field relates generally to methods for steering assistance and, in particular, to a method for intuitively overriding the steering of a vehicle by means of an interface which at least automatically sets a required effort for overriding a steering system according to the invention.

[0002] In recent years, there have been significant advances in autonomous and semi-autonomous driving features in land-driven vehicles, such as Super Cruise (a hands-free semi-autonomous driver assistance feature that uses high-resolution maps and road-monitoring sensors to assist a vehicle's acceleration and deceleration), Lane Keep Assist (Lane Keeping Assist, a semi-autonomous driving feature that assists the steering to keep a vehicle within lane boundaries or in the center of the lane), and others. Vehicles can still be improved in many ways.

[0003] With continuous automatic steering (e.g., SuperCruise), the driver's perception of safety changes depending on the vehicle position and road conditions. When the vehicle is in a safer operating state, the driver will perceive steering control as less necessary and prefer less effort to stop automatic steering intervention.

[0004] It is desirable to reduce the driver's workload when using automatic manual steering and to reduce the general annoyance of overriding manual steering when the vehicle is in a safe condition, and to interpret different areas of steering control based on environmental, path planning and control algorithm states.

[0005] It is desirable to provide methods to classify the intervention phase of steering assistance and to determine and adjust the reduction of the driver's steering override effort based on the range of steering control and the classified intervention phase.

[0006] Further desirable features of the present invention will become apparent from the following detailed description and the appended claims, taken in conjunction with the accompanying drawings and the foregoing technical field and background.

[0007] DE 11 2017 001 295 B4 describes a driving assistance apparatus comprising: a steering system for automatically lateral guidance of a vehicle, a switching control unit for terminating automatic lateral guidance and for switching to manual lateral guidance by the driver if, during automatic lateral guidance, a steering torque or an actuation quantity of the driver on the steering handle exceeds a threshold value, and a reliability detection unit for detecting a deterioration in the reliability of the automatic lateral guidance.

[0008] DE 10 2012 215 228 A1 describes how a vehicle's steering angle measurements can be measured and expected steering angle measurements calculated. Each calculated expected steering angle measurement corresponds to a measured steering angle measurement. The difference between a measured steering angle measurement and its corresponding calculated expected value can be calculated. Furthermore, lower and upper limits can be calculated for transition zones outside the control ranges. If differences lie within these ranges, the vehicle's steering can be gradually transferred from the automatic vehicle control system to the driver over a specified period of time.

[0009] DE 10 2012 112 442 A1 relates to the control of a vehicle with a driver assistance system that enables automated, partially automated, and manual driving. The transition occurs from automated driving with the highest degree of automation via various operating modes of the assistance system to manual driving with the lowest degree of automation. Here, the vehicle control elements are HMI interfaces between the driver and the assistance system that influence longitudinal and lateral dynamics. The invention provides that by actuating a vehicle control element to deactivate at least one assistance function of the assistance system, a detection threshold to be detected by this system is lowered with increasing degree of automation.

[0010] DE 10 2014 107 194 A1 describes a method for operating a steering system of a motor vehicle. A steering mechanism is connected to a steering gear via a superimposed steering system. During autonomous driving of the motor vehicle, a driver intervention in the steering mechanism is determined. The superimposed steering system is operated in such a way that a coupling between a steering mechanism angle and an angle introduced into the steering gear is adjusted depending on the determined driver intervention. DESCRIPTION

[0011] The object of the invention is to enable the vehicle to operate more autonomously, requiring less intervention from the driver in the steering. This not only reduces the driver's workload but also potentially disrupts unwanted steering maneuvers, resulting in an overall improved driving experience. This object is achieved by the subject matter according to claim 1.

[0012] A method, system, and apparatus for providing steering assistance for a vehicle by adapting vehicle trajectories based on road geometry and driving scenario, controlling intervention outputs taking trajectory tracking into account, and mitigating overcorrections and tracking anomalies are disclosed.

[0013] In an exemplary embodiment, a method for implementing steering override phases in a vehicle using a deep neural network (DNN) is provided. The method includes receiving, by a steering assistance unit located in the vehicle, a set of vehicle inputs, including lane data interpretation information and driver inputs; configuring a mission planning module located in the steering assistance unit to use the lane and vehicle sensor data interpretation information to determine at least one desired path of the vehicle; configuring a vehicle path prediction module located in the steering assistance unit to use the lane data interpretation information to determine at least one set of predicted dynamics of the vehicle;Configuring a driver override determination module, located in the steering assist unit and in communication with the DNN, to intervene with override steering control based on a phase of steering control operation determined in part by the desired path and predicted dynamics of the vehicle and by information of a corresponding intervention phase classified in the DNN; and in response to a determination to override the automatic steering control, configuring a lateral control module, located in the steering assist unit, to cease applying steering control torque for the automatic steering assist based on a determined phase of steering control operation and the determined driver intervention level by the amount of driver applied torque and the torque rate.

[0014] In at least one exemplary embodiment, the method includes configuring the mission planning module located in the steering support unit to use the lane data and the vehicle sensor interpretation information to determine a desired trajectory path of the vehicle.

[0015] In at least one exemplary embodiment, the method includes configuring the vehicle path prediction module located in the steering assistance unit to use the lane data and the interpretation of the vehicle sensor information to determine the current predicted path of the vehicle.

[0016] In at least one exemplary embodiment, the method includes configuring the driver override determination module with a vehicle curvature determination module to generate a set of curvature parameters and a vehicle heading and position determination module to generate a set of heading parameters to be stored in a decision matrix to determine when to override the currently active automatic steering control for different amounts of driver input, such as driver applied steering torque and torque rate.

[0017] In at least one exemplary embodiment, the method includes configuring an offline data labeling module to send labeled data to the DNN, and in response to a command assist torque signal, to determine the corresponding classified phase of the steering control event and use the classified phase information to modify the steering override torque thresholds by the driver override determination module.

[0018] In at least one exemplary embodiment, the method includes configuring a lateral control module to generate a steering control torque based at least on inputs including the desired path of the vehicle, the predicted dynamics of the vehicle, and a driver override indicator.

[0019] In at least one exemplary embodiment, the driver influence indicator is generated by the driver influence determination module.

[0020] In another exemplary embodiment, a system is provided. The system comprises a processing unit disposed in a vehicle and including one or more processors configured by programming instructions encoded on non-transitory computer-readable media in communication with a deep neural network (DNN), the processing unit configured to: receive a set of vehicle inputs including lane data interpretation information and driver steering inputs; determine at least one desired path of the vehicle based on the lane data interpretation information and vehicle sensors; determine at least one set of predicted dynamics of the vehicle based on the lane data interpretation information and vehicle sensors;Overriding the automatic steering control torque based on a phase of the steering control event determined in part by the desired path and predicted dynamics of the vehicle, and by information about a corresponding intermediate phase classified in the DNN; and in response to a determination to override the automatic steering control, ceasing application of the automatic steering control torque for steering assistance based on a determined phase of the steering control event and the determined driver intervention level, including the amount of torque applied by the driver and the torque rate.

[0021] In at least one exemplary embodiment, the system includes the processing unit configured to determine a desired path of the vehicle based on the lane data and the interpretation of the vehicle sensor information.

[0022] In at least one exemplary embodiment, the system includes the processing unit configured to determine a path of the vehicle based on the lane data interpretation information.

[0023] In at least one exemplary embodiment, the system includes the processing unit configured to generate a set of curvature parameters and a set of curve parameters stored in a decision matrix to determine when the automatic steering torque is overridden by the application of a variable driver override threshold applied by driver inputs, including steering torque and torque rate.

[0024] In at least one exemplary embodiment, the system includes the processing unit configured to send tagged data obtained offline to the DNN and, in response to a torque assist signal of the control command, send classified engagement phase information corresponding to a range in which the variable driver steering torque override threshold is to be applied.

[0025] In at least one exemplary embodiment, the system includes the processing unit configured to: generate a steering control torque based on at least one set of inputs including the desired path of the vehicle, the predicted dynamics of the vehicle, and a driver override indicator.

[0026] In at least one example embodiment, the system includes the processing unit configured to: generate a driver override indicator based on at least one set of inputs including the desired path of the vehicle, the predicted dynamics of the vehicle, and the driver input.

[0027] In another exemplary embodiment, a vehicle device is provided. The device includes a steering assistance unit comprising one or more processors and non-transitory computer-readable media encoded with programming instructions, the steering assistance unit configured to receive a set of vehicle inputs including lane data and vehicle sensor interpretation information, and driver inputs; determine at least one desired path of the vehicle based on use of the lane data and the vehicle sensor interpretation information; determine at least one set of predicted dynamics of the vehicle based on the lane data and the vehicle sensor interpretation information;Overriding the automatic steering torque based on a phase of the steering control event determined in part by the desired path and predicted dynamics of the vehicle, and by information from a corresponding intermediate phase classified in a DNN; and in response to a determination to override the automatic steering torque, terminating the application of a variable amount of steering torque for steering assist based on a particular phase of the steering control event and the variable amount of driver input to the vehicle control systems.

[0028] In at least one exemplary embodiment, the vehicle device further includes the steering assist unit configured to determine a mixed path of the vehicle based on the lane data and the vehicle sensor interpretation information; and determine a path of the vehicle based on the lane data and the vehicle sensor interpretation information.

[0029] In at least one exemplary embodiment, the vehicle device further comprises the steering assist unit configured to generate a set of curvature parameters and a set of heading parameters to be stored in a decision matrix to determine when to override the automatic steering control torque based on a variable amount of driver input, the input including the driver steering torque and the torque rate.

[0030] In at least one exemplary embodiment, the vehicle device further comprises the steering assist unit configured to send labeled data obtained offline to the DNN and, in response to a control command torque assist signal, send classified engagement phase information corresponding to a range in which driver override thresholds vary, including driver applied steering torque and torque rate.

[0031] In at least one exemplary embodiment, the vehicle device further comprises the steering assist unit configured to: generate a steering control torque based on at least one set of inputs including the desired path of the vehicle, the predicted dynamics of the vehicle, and a driver override indicator.

[0032] In at least one exemplary embodiment, the vehicle device further comprises the steering assist unit configured to generate a driver override flag based on at least one set of inputs including the desired path of the vehicle, the predicted dynamics of the vehicle, and a driver override flag. BRIEF DESCRIPTION OF THE CHARACTERS

[0033] The exemplary embodiments are described below in conjunction with the following figures, wherein like reference numerals denote like elements and wherein: Fig. 1 is a block diagram illustrating an example vehicle that may include a processor for path-based automatic driving support of vehicle operation implemented by an adaptive driver monitoring system according to an example embodiment; Fig. 2 shows an exemplary diagram of parameters calculated for path-based automatic driving support of vehicle operations implemented by an adaptive driver monitoring system according to an exemplary embodiment; Fig. 3 shows an exemplary diagram of the control architecture for path-based automated driving support of vehicle operation implemented by an adaptive driver monitoring system according to an exemplary embodiment; Fig. 4 shows an exemplary diagram for path-based automatic driving support of vehicle operations implemented by an adaptive driver monitoring system according to an exemplary embodiment; Fig. 5 shows an exemplary diagram of parameters for path-based automated driving support of vehicle operations implemented by an adaptive driver monitoring system according to an exemplary embodiment; Fig. 6 shows an exemplary diagram of offline data labeling used for path-based automatic driving support of vehicle operations implemented by an adaptive driver monitoring system according to an embodiment; Fig. 7A and Fig. 7B show exemplary diagrams of parameters for path-based automated driving support of vehicle operation implemented by an adaptive driver monitoring system; and Fig. 8 shows an exemplary flowchart for path-based automatic driving support of vehicle operations implemented by an adaptive driver monitoring system according to one embodiment. DETAILED DESCRIPTION

[0034] The following detailed description is merely exemplary and is not intended to limit the application and uses. Furthermore, there is no intention to be bound by any express or implied theories presented in the preceding technical field, background, summary, or the following detailed description.

[0035] As used herein, the term "module" refers to any hardware, software, firmware, electronic control component, processing logic, and / or processor device, individually or in any combination, including, but not limited to: application-specific integrated circuits (ASICs), a field-programmable gate array (FPGA), an electronic circuit, a processor (common, dedicated, or group) and memory executing one or more software or firmware programs, a combinational logic circuit, and / or other suitable components that provide the described functionality.

[0036] Embodiments of the present disclosure may be described herein in terms of functional and / or logical block components and various processing steps. Such block components may be implemented by any number of hardware, software, and / or firmware components configured to perform the specified functions. For example, an embodiment of the present disclosure may utilize various integrated circuit components, such as memory elements, digital signal processing elements, logic elements, lookup tables, or the like, capable of performing a variety of functions under the control of one or more microprocessors or other control devices.Furthermore, those skilled in the art will understand that embodiments of the present disclosure may be used in connection with any number of systems and that the systems described herein are merely exemplary embodiments of the present disclosure.

[0037] Autonomous and semi-autonomous vehicles are capable of sensing their surroundings and navigating based on the sensed environment. Such vehicles sense their surroundings using multiple types of sensors, such as optical cameras, radar, lidar, other image sensors, and the like. In such vehicles, the sensed data can be combined with map data and vehicle sensors (inertial measurement device, speed sensors, etc.) to determine and track the vehicle's trajectory tracking performance based on road geometry.

[0038] The present disclosure describes methods, systems, and apparatus for intuitive steering override of a vehicle through an interface that at least automatically sets a required steering override effort, classifies an intervention phase of the steering assist function based on learned behavior, determines and sets the driver's steering override effort based on operating conditions in a continuous range, and sets the steering override effort based on control, path planning, and prediction parameters.

[0039] Fig. 1 shows a block diagram illustrating an example vehicle that may include a processor for determining a variable range of driver steering override effort, including the feature of continuously varying the override effort in response to road geometry and vehicle dynamics, continuously varying the override effort in response to the geometry of a desired path and predicted vehicle dynamics, continuously varying the override effort in response to differences between the vehicle's current operating conditions and a predicted path, training a DNN and classifying an intervention phase of a steering assist event similar to what is perceived by a human driver, and adjusting the steering override effort based on the classified steering assist phase.which is learned by a DNN through the driver assistance system 100. The system 100 continuously implements practical automated steering functions (e.g., SuperCruise) that correspond to the driver's sense of safety and change based on the vehicle's position within the lane boundaries as well as the road geometry and conditions.

[0040] For example, when the vehicle is in a safe operating condition, the driver will perceive steering as less necessary and expect less effort to stop. Accordingly, the system 100 reduces the driver's effort when using a convenient automated steering function. This, in turn, reduces the overall driver annoyance of manual steering control when the driver's intent differs from that of the automated driving function implemented by the system 100 when the vehicle is operating in a safe condition.

[0041] In an exemplary embodiment, system 100 continuously implements convenient automated steering features (e.g., SuperCruise) that can respond to the driver's perception of "safe" changes based on vehicle position and roadway conditions. When the vehicle is in a safer operating condition due to vehicle position within the lane boundaries and / or road geometry and roadway conditions, system 100 responds to the driver's perceived reduced need for steering control and the expected reduction in effort required to stop. System 100 can also reduce driver effort when using other related automated hand-steering features to reduce the overall annoyance of hand-steering when the vehicle is in such safe conditions.

[0042] As in Fig. 1, the vehicle 10 generally includes a chassis 12, a body 14, front wheels 16, and rear wheels 18. The body 14 is disposed on the chassis 12 and substantially encloses the components of the vehicle 10. The body 14 and the chassis 12 may together form a frame. The vehicle wheels 16, 18 are each pivotally connected to the chassis 12 near a corner of the body 14. The vehicle 10, in the illustrated embodiment, is shown as a passenger vehicle. However, it should be recognized that any other vehicle, including motorcycles, trucks, sport utility vehicles (SUVs), recreational vehicles (RVs), watercraft, aircraft, etc., may also be used.

[0043] As illustrated, the vehicle 10 generally includes a drive system 20, a transmission system 22, a steering system 24, a braking system 26, a sensor system 28, an actuator system 30, at least one data storage device 32, at least one controller 34, and a communication system 36. The drive system 20, in this example, may include an electric machine, such as a permanent magnet (PM) motor. The transmission system 22 is configured to transfer the power of the drive system 20 to the vehicle wheels 16 and 18 according to selectable gear ratios.

[0044] The braking system 26 is configured to apply braking torque to the vehicle wheels 16 and 18. In various exemplary embodiments, the braking system 26 may include friction brakes, a wire brake, a regenerative braking system such as an electric machine, and / or other suitable braking systems.

[0045] The steering system 24 influences the position of the vehicle wheels 16 and / or 18. Although a steering wheel 25 is shown for illustrative purposes, the steering system 24 may not include a steering wheel in some exemplary embodiments contemplated by the present disclosure.

[0046] The sensor system 28 includes one or more sensing devices 40a-40n that sense observable conditions of the external environment and / or the internal environment of the vehicle 10 and generate sensor data related thereto.

[0047] The actuator system 30 includes one or more actuator devices 42a-42n that control one or more vehicle functions, such as, but not limited to, the drive system 20, the transmission system 22, the steering system 24, and the braking system 26. In various exemplary embodiments, the vehicle 10 may also include interior and / or exterior vehicle features that are Fig. 1 are not shown, such as various doors, a trunk, and cabin features such as air, music, lighting, touchscreen display components, and the like.

[0048] The data storage device 32 stores data for controlling the vehicle 10. The data storage device 32 may be part of the control unit 34, separate from the control unit 34, or part of the control unit 34 and part of a separate system.

[0049] The controller 34 includes at least one processor 44 (integrated into or connected to the system 100) and a computer-readable storage device or medium 46. The processor 44 may be any custom or off-the-shelf processor, a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC) (e.g., a custom ASIC implementing a neural network), a field-programmable gate array (FPGA), an auxiliary processor among multiple processors connected to the controller 34, a semiconductor-based microprocessor (in the form of a microchip or chipset), any combination thereof, or generally any device for executing instructions. The computer-readable storage device or media 46 may include volatile and non-volatile memories, e.g.,Read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). KAM is a persistent or non-volatile memory used to store various operating variables while the processor 44 is powered off. The computer-readable storage device(s) 46 may be implemented using various known storage devices such as PROMs (programmable read-only memory), EPROMs (electrical PROMs), EEPROMs (electrically erasable PROMs), flash memory, or other electrical, magnetic, optical, or combination storage devices capable of storing data, some of which are executable instructions used by the controller 34 in controlling the vehicle 10.

[0050] The instructions may comprise one or more separate programs, each containing an ordered list of executable instructions for implementing logical functions. When executed by processor 44, the instructions receive and process signals (e.g., sensor data) from sensor system 28, execute logic, calculations, methods, and / or algorithms to automatically control the components of vehicle 10, and generate control signals that are transmitted to actuator system 30 to automatically control the components of vehicle 10 based on the logic, calculations, methods, and / or algorithms. Although in Fig. 1 only one controller 34 is shown, embodiments of the vehicle 10 may include any number of controllers 34 that communicate and cooperate via any suitable communication medium or combination of communication media to process the sensor signals, perform logic, calculations, methods and / or algorithms, and generate control signals to automatically control features of the vehicle 10.

[0051] For example, the system 100 may include any number of additional sub-modules embedded within the controller 34, which may be combined and / or further subdivided to implement the systems and methods described herein in a similar manner. Furthermore, inputs to the system 100 may be received from the sensor system 28, from other control modules (not shown) connected to the vehicle 10, and / or from other sub-modules (not shown) within the controller 34 of Fig. 1. In addition, the inputs can also be subjected to preprocessing, such as subsampling, denoising, normalization, feature extraction, missing data reduction, and the like.

[0052] Fig. 2 shows an exemplary diagram of parameters calculated for path-based automatic driving support of vehicle operations implemented by an adaptive driver monitoring system according to an exemplary embodiment. Fig. 2 shows the calculation of the torque parameters by the system 100 (from Fig. 1) as follows: τOvrdLt=τCal∗GLt(X,YHdng,YisCurve); τOvrdRt=τCal∗GRt(X,YHdng,YisCurve,) and X=[Δy Δρ ΔyVPP ΔρVPP ΔyBP ΔρBP y(0) ρ(0) P].

[0053] In an exemplary embodiment, the determination of the direction of curvature by the system 100 is performed as follows: YHdng={−1(inner)KHdng<α0(center)α≤KHdng≤β1(outer)KHdng>β where KHdng=f(X)=AHdngX YisCurve=1

[0054] For right curves α, β opposite signs compared to left curves

[0055] In an exemplary embodiment, the steering force is continuously adjusted by the system 100 as follows: τOvrdLt=τCAL+τAdj(X,P) τOvrdRt=−τCAL−τAdj(X,P) τOvrdLt=τCAL+τAdjCrv(X,YHdng,P) τOvrdRt=−τCAL−τAdjCrv(X,YHdng,P) τ Adj can be defined as linear (functions or gains) or nonlinear (stepwise functions or lookup tables).

[0056] In an exemplary embodiment, the curvature determination is carried out as follows: YisCurve={1KisCurve(X)≥C0KisCurve(X) <C

[0057] The direction of curvature is determined as follows: YHdng={−1(inner)KHdng(X)<α0(center)α≤KHdng(X)≤β1(outer)KHdng(X)>β

[0058] In an exemplary embodiment, the input state is defined as follows: X=[ΔyΔρΔyVPPΔρVPPΔyBPΔρBPy(0)ρ(0)P]T=[yBP(tLA)−yVPP(tLA)ρBP(tLA)−ρVPP (tLA)yVPP(tLA)−y(0)ρVPP(tLA)−ρ(0)yBP(tLA)−y(0)ρBP(tLA)−ρ(0)y(0)ρ(0)P]T where τ OvrdLt , τ OvrdRt is the total value of the torque of the steering wheel of the oversteering driver on the left and right sides; where τ Cal is a statically assigned value for the steering wheel torque of the oversteering driver; where Y Hdng , Y isCurve Logic / state parameters that indicate the status of the vehicle direction and the vehicle in a curve; where K isCurve is a function of the state X for determining the vehicle curvature coefficient; where C is a threshold value for the curvature coefficient that determines the state value of Y isCurve describes; where K Hdng is a function of the state X to determine the heading coefficient of the vehicle; and where α, β is a category of coefficient thresholds that determine the state value of Y Hdng describe.

[0059] Furthermore, y is a lateral position of the vehicle with respect to the calculated center of the lane as a function of time, relative to the current event; ρ is a curvature of the trajectory line, a function of the time until the current event; P is phase classification information; t LA is a predetermined time value in the future used as a look-ahead time for steering control; X is an input matrix of the input state; G Lt (), G Rt () are configurable states for amplifying the input matrix of the input state X; BP is a subscript indicating the “fade path” or the desired predicted trajectory of the vehicle; and VPP is a subscript indicating the “predicted path” or vehicle dynamics based on a predicted trajectory.

[0060] Fig. Figure 3 shows an exemplary diagram of the control architecture for path-based automated driving support of vehicle operation implemented by an adaptive driver monitoring system according to an exemplary embodiment. In Fig. 3, the system comprises 100 (of Fig. 1) Input lane data and vehicle sensor interpretations 305 into system 100 to a mission planning module 310, a vehicle path prediction module 320, and a driver override determination module 330. The driver override determination module 330 also receives input of a driver input 315. The mission planning module generates the desired path and sends the information to the driver override determination module 330 and a lateral control module 340. The vehicle path prediction module 320 generates a prediction of the vehicle dynamics and path, which is communicated to the lateral control module 340 and the driver override determination module 330.

[0061] The driver override determination module 330 receives a set of multiple inputs for processing, consisting of the driver input 315, the lane data and interpretation of the vehicle sensors 305, the predicted vehicle dynamics and path information generated by the vehicle path prediction module 320, the desired path information generated by the mission planning module 310, and the classified intervention phase information contained in a deep neural network 350 based in part on offline labeled data from the offline labeled data module 360. The driver override determination module 330 processes the multiple input information and generates driver override information for the lateral control module 340.

[0062] The lateral control module 340 processes the set of inputs to generate the steering torque 345 based on information processed from the set of inputs consisting of driver steering input information 315, driver flag information generated by the driver oversteer determination module 330, predicted vehicle dynamics information and path information generated by the vehicle path prediction module 320, and desired path information generated by the mission planning module 310.

[0063] Fig. Figure 4 shows an exemplary diagram for path-based automated driving support of vehicle operations implemented by an adaptive driver monitoring system according to an exemplary embodiment. In Fig. 4 are elements of module 330 (from Fig. 3) to determine the driver overhang in more detail. In Fig. 4, the input of lane data and the interpretation of the vehicle sensors from the vehicle sensors 405 are received by the mission planning module 310 and the vehicle path prediction module 320. Both modules, the mission planning module 310 and the vehicle path prediction module 320, generate desired path information and information about the predicted vehicle dynamics (i.e., information contained in Fig. 4 as being at position "Z"), which are passed directly and indirectly for processing to a number of elements of the oversteer determination module 330, including a vehicle curvature determination module 415, a vehicle heading and position module 420, a gain planning module 430, and a decision and indexing module 425.

[0064] In an exemplary embodiment, the vehicle curvature determination module 415 receives inputs from the position “Z” and generates Y isCurve Information for a decision and indexing module 425 and the vehicle heading and position module 420. The vehicle heading and position module 420 generates Y Hdng Information to the decision and indexing module 425. The Y isCurve Information, and the Y Hdng The information and the information are indexed for retrieval in a matrix contained in the decision and indexing module 425 for sending to the gain scheduling module 430. The gain scheduling module 430 also receives as input information about the desired path and the predicted dynamics of the vehicle over position "Z". The output of the gain scheduling module 430 is combined with the input of the torque τ in function 445 for processing by function 455 (multiplier).Cal Calculations 450 derived from vehicle speed V x The output of function 455 is compared with the driver steering input τ Drvr compared to determine an oversteer flag to be used by the lateral control module 340 ( Fig. 3) is used for the steering torque.

[0065] Fig. 5 shows an exemplary diagram of functions for path-based automated driving support of vehicle operations implemented by an adaptive driver monitoring system according to an exemplary embodiment. In Fig. 5 shows functions that are configured offline or online in the system 100. In Fig. 5 is an offline labeling module 510 (in Fig. 6) for an offline configured and online updated label for a range of regions and settings of vehicle operation. Module 530 represents a vehicle state to intervene in lane operation and apply torque assist in real time. Module 520 represents a neural network configured to store labeled intent phase data that is trained and validated offline. Further, module 520 generates classified intervention phase information in real time based in part on real-time (online) operational information of vehicle operations such as an intervention state and applied assist torque in module 530.Finally, module 540 presents a classified engagement and assist torque graph that is sent to a driver override determination 550 for use in configuring the engagement state and assist torque in module 530 in real time (online).

[0066] Fig. 6 shows an example diagram of offline data labeling used for path-based automated driving support of vehicle operations implemented by an adaptive driver monitoring system according to one embodiment. Fig. 6 shows in the exemplary diagram a series of intervention phases 610 with maximum correction, steering back, inactivity, acceleration, approach and settling, which correspond to a diagram 620 and a bar chart 630 with torque values.

[0067] The Fig. 7A and Fig. 7B show example diagrams of features for path-based automatic driving support of vehicle operations implemented by an adaptive driver monitoring system. In Fig. Figure 7A shows a vehicle 710 in a safe position configured with reduced driver thresholds and a new detection stop control. The vehicle's safe position corresponds to a graph 720 that represents the torque-to-drive torque ratio and oversteer limits, graph 730 that represents the curvature to the driver's oversteer indicators, and graph 740 that indicates the distance to the vehicle position. Fig. Figure 7B illustrates a vehicle 750 in an unsafe position configured with a large engagement torque, elevated driver thresholds, and a new detection similar to the previous behavior. The following graphs are shown according to the vehicle's unsafe position: torque versus drive torque and override limits 760, curvature versus driver override indicator 770, and distance to vehicle position 780.

[0068] Fig. Figure 8 shows an exemplary flowchart for path-based automated driving assistance of vehicle operations implemented by an adaptive driver monitoring system according to one embodiment. Flowchart 800 illustrates the path-based automated assistance operation tasks for the adaptive driver monitoring system, which include the following tasks: Task 810 for determining a variable range of driver override effort based on a set of vehicle inputs, including lane data and vehicle sensor interpretation information, such asDriver inputs, by a driver oversteer determination module in communication with the DNN, to oversteer the automatic steering control-based torque based on a phase of the steering control event determined by the desired path and predicted vehicle dynamics, and by information from a corresponding intermediate phase classified in the DNN. Task 820 to continuously vary the oversteer effort in response to road geometry and vehicle dynamics by determining desired paths and predicted vehicle dynamics.Task 830: Continuously varying the override effort as a function of the geometry of a predicted path and the predicted vehicle dynamics using a lateral control module located in the steering assist unit, terminating the application of the automated and the determined driver intervention level, e.g., by the amount of torque applied by the driver and the torque rate, to override the steering control torque for the automated steering assist based on a determined phase of the steering control operation and the driver input, e.g., the steering torque and the torque rate.

[0069] Task 840: Continuously vary the oversteer effort in response to differences between the vehicle's current operating conditions and a predicted path. Task 850: Train a DNN and classify the intervention phase of a steering assist event, similar to what a human driver perceives. Task 860: Adjust the steering oversteer effort based on the classified steering assist phase learned by a DNN using labeled data created offline and sent to the DNN, and send classified intervention phase information corresponding to a range to vary the driver steering torque threshold by the driver oversteer determination module in response to a control command torque assist signal.

[0070] The deep neural network is used in the adaptive driver monitoring system to determine the torque characteristics and is configured as a pre-trained neural network. Therefore, in certain embodiments, the torque prediction system process is configured in only one operating mode. For example, in various embodiments, the deep neural network is trained in a training mode before being deployed or deployed in the vehicle (or other vehicles). Once the deep neural network is trained, it can be deployed in a vehicle (e.g., the vehicle 10 of Fig. 1) be implemented in an operating mode in which the vehicle is operated autonomously, semi-autonomously or manually.

[0071] In various alternative embodiments, the neural network may also be implemented in a vehicle in both training mode and operating mode and trained during an initial operating period in conjunction with the application of a time delay or similar methodology for torque control predictions. A vehicle may also be operated exclusively in operating mode with neural networks that have already been trained via a training mode of the same vehicle and / or other vehicles in various embodiments.

[0072] As briefly mentioned above, the various modules and systems described above can be implemented as one or more machine learning models that undergo supervised, unsupervised, semi-supervised, or reinforcement learning. Such models can be trained to perform classification (e.g., binary or multi-class classification), regression, clustering, dimensionality reduction, and / or similar tasks. Examples of such models include artificial neural networks (ANNs) (e.g., recurrent neural networks (RNNs) and convolutional neural networks (CNNs)), decision tree models (e.g., classification and regression trees (CARTs)), ensemble learning models (e.g., boosting, bootstrapped aggregation, gradient boosting machines, and random forests), Bayesian network models (e.g.,, Naive Bayes), principal component analysis (PCA), support vector machines (SVM), clustering models (such as K-nearest neighbor, K-means, expectation maximization, hierarchical clustering, etc.) and linear discriminant analysis models.

[0073] In various exemplary embodiments, the present disclosure describes a method for implementing phases of steering override in a vehicle using a deep neural network (DNN). This method includes receiving, by a steering assistance unit located in the vehicle, a set of vehicle inputs comprising lane data and vehicle sensor interpretation information, including a driver input; configuring a mission planning module located in the steering assistance unit to use the lane data and vehicle sensor interpretation information to determine at least one desired path of the vehicle; configuring a vehicle path prediction module located in the steering assistance unit to use the lane data and vehicle sensor interpretation information to determine at least one set of predicted dynamics of the vehicle;Configuring a driver override determination module, located in the steering assist unit and communicating with the DNN, to override an automatic steering control torque based on a phase of the steering control event determined in part by the desired path and predicted dynamics of the vehicle and by information about a corresponding intermediate phase classified in the DNN; in response to a determination by a lateral control module located in the steering assist unit, enabling automatic configuration of an override steering control module to terminate application of automatic steering;and in response to a determination of the lateral control module disposed in the steering assist unit, determining a driver intervention level of a magnitude of driver applied torque and a torque rate and a phase of steering control operation and a driver input, wherein the driver input includes a steering torque and a driver torque rate to override the steering control torque for automatic steering assist;

[0074] The present disclosure also describes a method that includes configuring the mission planning module located in the steering assistance unit to use the lane data and vehicle sensor interpretation information to determine a mixed path of the vehicle, and configuring the vehicle path prediction module located in the steering assistance unit to use the lane data and vehicle sensor interpretation information to determine a path of the vehicle.

[0075] The present disclosure also describes the configuration of the driver oversteer determination module with a vehicle curvature determination module to generate a set of curvature parameters and a vehicle heading and position determination module to generate a set of heading parameters to be stored in a decision matrix, both sets of parameters being used to determine when an oversteer action should be performed by the steering assist unit, wherein the automatic steering control torque is applied based on a particular phase of the steering control operation and the driver inputs, wherein the driver inputs include the driver steering torque and the torque rate.

[0076] The present disclosure also describes configuring a labeling module to label data offline, to send labeled data to the DNN in response to a control command assist torque signal, sending classified intervention phase information corresponding to a range to apply a varying amount of steering override torque by the driver override determination module, and configuring a lateral control module to generate a steering control torque based at least on inputs including the desired path of the vehicle, the predicted dynamics of the vehicle, and a driver override indicator, wherein the driver override indicator is generated by the driver override determination module.

[0077] It should be noted that the process of Fig. 1-8 may include any number of additional or alternative tasks, that the Fig. The tasks shown in 1-8 do not have to be carried out in the order shown and that the process of Fig. 1-8 can be integrated into a more comprehensive procedure or process with additional functionality not described in detail here. In addition, one or more of the Fig. 1-8 in an embodiment of the method described in the Fig. 1-8 may be omitted as long as the intended overall functionality is retained.

[0078] The foregoing detailed description is merely illustrative and is not intended to limit the embodiments of the subject matter or the application and uses of such embodiments. As used herein, the word "exemplary" means "serving as an example, instance, or illustration." Any implementation described herein as exemplary is not necessarily to be construed as preferred or advantageous over other implementations. Furthermore, there is no intention to be bound by any express or implied theory presented in the foregoing technical field, background, or detailed description.

[0079] Although at least one exemplary embodiment has been presented in the foregoing detailed description, it should be understood that numerous variations exist. It should also be appreciated that the exemplary embodiment or exemplary embodiments are merely examples and are not intended to limit the scope, applicability, or configuration of the disclosure in any way. Rather, the foregoing detailed description is intended to provide those skilled in the art with practical guidance for implementing the exemplary embodiment or exemplary embodiments.

[0080] It is to be understood that various changes in the function and arrangement of elements may be made without departing from the scope of the disclosure as set forth in the appended claims and their legal equivalents.

Claims

[1] A method for implementing steering override phases in a vehicle (10) using a Deep Neural Network, DNN, comprising: Receiving, by a steering assist unit disposed in the vehicle (10), a set of vehicle inputs comprising lane data and interpretation information from vehicle sensors, including the driver's steering input; Configuring a mission planning module (310) disposed in the steering support unit to use the lane data and vehicle sensor interpretation information to determine at least one desired path of the vehicle (10); Configuring a vehicle path prediction module disposed in the steering assistance unit to use the lane data and vehicle sensor interpretation information to determine at least one set of predicted dynamics of the vehicle (10); Configuring a driver override determination module (330) located in the steering assistance unit and communicating with the DNN to override an automatic steering control torque based on a phase of the steering control event determined in part by the desired path and predicted dynamics of the vehicle (10) and by information of a corresponding intermediate phase classified in the DNN; in response to a determination of a lateral control module disposed in the steering assist unit, enables automatic configuration of an override steering control module to terminate application of the automatic steering; and in response to a determination by the lateral control module disposed in the steering assistance unit, to determine a driver intervention level of a driver applied torque and a torque rate, as well as a phase of the steering control operation and a driver input, wherein the driver input includes a steering torque and a driver torque rate to override the steering control torque for automatic steering assistance, further comprising: Configuring the mission planning module (310) arranged in the steering support unit to use the lane data and vehicle sensor interpretation information to determine a mixed path of the vehicle (10), further comprising: Configuring the vehicle path prediction module located in the steering support unit to use the lane data and vehicle sensor interpretation information to determine a path of the vehicle, further comprising: Configuring the driver oversteer determination module (330) with a vehicle curvature determination module (415) to generate a set of curvature parameters and a vehicle heading and position module (420) to generate a set of heading parameters to be stored in a decision matrix, wherein both sets of parameters are used to determine when an oversteer action is implemented by the steering assist unit, wherein the automated steering control torque is applied based on a particular phase of the steering control process and the driver inputs, wherein the driver inputs include a driver steering torque and a torque rate, further comprising: Configuring a labeling module for offline data labeling to send labeled data to the DNN, and in response to a control command assist torque signal, sending classified engagement phase information corresponding to a range in which the driver override determination module (330) applies a variable steering override torque, further comprising: Configuring a lateral control module (340) to generate a steering control torque based at least on inputs including the desired path of the vehicle (10), the predicted dynamics of the vehicle, and a driver override indicator. wherein the driver override flag is generated by the driver override determination module (330).

Images