An intelligent driving method and device for drivers with lower limb disabilities

Abstract

The present invention belongs to the technical field of autonomous driving testing, and specifically relates to an intelligent driving method and device for lower-limb disabled drivers. The system includes a lower-limb disabled driver operation module, an autonomous driving module, a human-machine integration module, and a control execution module; the method can convert the steering wheel position and speed change information into digital signals through a vertically movable steering wheel, obtain the driving intentions of acceleration and deceleration, and develop a longitudinal decision-making algorithm for autonomous driving based on deep learning, and achieve effective human-machine integration through driving right allocation to complete the control of intelligent vehicles.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving testing, and specifically relates to an intelligent driving method and device for lower-limb disabled drivers. Background Art

[0002] The research and development of intelligent vehicles for the disabled not only has great market potential, but also is of great significance for protecting the travel rights of the disabled, creating economic value, and improving the people's happiness index.

[0003] With the continuous development of autonomous driving technology, the travel convenience of the disabled has been greatly improved. However, at the present stage, mass-produced autonomous vehicles have not achieved L5-level high-level autonomous driving, and autonomous driving or assisted driving is achieved under specific working conditions. Even still, an auxiliary driver is needed to act as a safety officer, which greatly limits the travel range of disabled drivers. In fact, the disabled do not completely lose their ability to control the vehicle. For example, lower-limb disabled drivers have normal thinking and hands controlled by the brain, and can control the rotation of the vehicle steering wheel, making it possible for individuals to be the main body of vehicle control. Therefore, developing an intelligent driving system for lower-limb disabled drivers and realizing the high integration of disabled drivers and the intelligent driving system can improve driving safety during driving, enhance the trust in the intelligent driving system, and also ensure the driving experience of disabled drivers.

[0004] However, there is still a blank in the driving assistance devices and intelligent driving systems for lower-limb disabled drivers. Therefore, it is still very crucial to develop an intelligent driving method and device for lower-limb disabled drivers. Summary of the Invention

[0005] The present invention provides an intelligent driving method and device for lower-limb disabled drivers, which can convert the steering wheel position and speed change information into digital signals through a vertically movable steering wheel, obtain the driving intentions of acceleration and deceleration, and develop a longitudinal decision-making algorithm for autonomous driving based on deep learning to achieve effective human-machine integration through driving right allocation and complete the control of intelligent vehicles.

[0006] The technical solution of the present invention is described in conjunction with the accompanying drawings as follows:

[0007] In a first aspect, the present invention provides an intelligent driving method for lower-limb disabled drivers, including the following steps:

[0008] Step 1: The driver with lower limb disability manipulates the steering wheel 1 to rotate and move vertically according to the surrounding driving environment information. The vertical position sensor of the steering wheel obtains the vertical displacement of the steering wheel, and the steering wheel angle sensor 4 and the steering wheel torque sensor 5 obtain the angle and torque of the steering wheel. The driver control module obtains the acceleration, braking, and steering intentions of the driver; The VCU 6 obtains the acceleration, braking, and steering signals of the driver control module;

[0009] Step 2: The VCU 6 receives the environment information obtained by the perception system in the autonomous driving module and outputs the vehicle longitudinal control signal through the end-to-end decision-making model;

[0010] Step 3: The human-machine integration module receives the control signals output by the driver control module and the autonomous driving module, obtains the final control signal through the allocation of human-machine driving rights, and the VCU 6 sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus;

[0011] Step 4: The control execution module controls the drive and brake execution motors and the steering execution motor to work, thereby realizing autonomous driving.

[0012] Furthermore, the autonomous driving module includes a perception system, a positioning and navigation system, and a decision-making and planning system.

[0013] Furthermore, the specific method of Step 2 is as follows:

[0014] S21: The perception system perceives and collects the original data of the surrounding environment through cameras, millimeter-wave radars, lidars, and ultrasonic radars; The original data is fused through Kalman filtering after preprocessing to obtain the target-level data x of traffic participants, x = {l h, l v, v h, v v, a s, a v, a h}, where l h is the lateral position of the traffic participant; l v is the longitudinal position; v h is the lateral speed; v v is the longitudinal speed; a s is the steering wheel angle of the host vehicle; a v is the longitudinal acceleration of the host vehicle; a h is the lateral acceleration, and then the target machine data is transmitted to the decision-making and planning system;

[0015] S22: The positioning information collected by the positioning and navigation system is synchronously transmitted to the decision-making and planning system; The positioning system obtains the positioning information of the vehicle in real time and transmits the positioning information to the decision-making and planning system;

[0016] S23. The decision-making and planning system, through the longitudinal decision-making model for autonomous driving based on deep learning, transmits the collected perception data and positioning data to the VCU. After calculation, it automatically makes a decision and outputs the longitudinal control signal of the vehicle.

[0017] Furthermore, the camera collects RGB image information; the millimeter-wave radar collects raw ADC data; the lidar collects 3D point cloud information; the ultrasonic radar collects raw data.

[0018] Furthermore, the specific method for fusing the raw data after preprocessing through Kalman filtering is as follows:

[0019] Obtain the measurement noises of the camera, millimeter-wave radar, lidar, and ultrasonic radar respectively as , , , , then the measurement signals from different sensors are expressed as:

[0020]

[0021]

[0022]

[0023]

[0024] In the formula, , , , are the measurement signals of the camera, millimeter-wave radar, lidar, and ultrasonic radar respectively; , , , are the corresponding measurement matrices, and the obtained measurement signal is ;

[0025] Among them, ;

[0026] The state vector of the dynamic system is , and the state transition equation and measurement equation are expressed as:

[0027]

[0028]

[0029] In the formula, is the system state transition matrix; is the system control input matrix; is the system control input; is the process noise, and the corresponding covariance matrix is ; is the measurement matrix; is the measurement noise, and the corresponding covariance matrix is ; Predict and update the system state. The prediction steps are as follows:

[0030] S1. State prediction:

[0031] In the formula, is the state at the past k - 1 moments, and is the state prediction at the kth moment; is the state estimate at the (k - 1)th moment;

[0032] S2. Covariance prediction:

[0033] In the formula, is the state at the past k - 1 moments, and is the covariance matrix of the state prediction at the kth moment, which is used to evaluate the accuracy of the estimated value, , representing the covariance matrix of the state estimate at the (k - 1)th moment;

[0034] S3. Update and correct the system according to the state prediction result, specifically as follows:

[0035] Calculation of the Kalman gain:

[0036]

[0037] In the formula, is the Kalman gain;

[0038] State update:

[0039]

[0040] Covariance update:

[0041]

[0042] Furthermore, the specific method of step three is as follows:

[0043] S31. Design a longitudinal control signal game formula considering multiple factors, as follows:

[0044]

[0045] In the formula, and are weight coefficients, which are used to balance the influence of factors such as vehicle state and road environment on the game result, and ; is the control signal for a driver with lower limb disabilities, including the depth of the downward position of the steering wheel indicating the intention to accelerate and the depth of the upward position of the steering wheel indicating the intention to decelerate ; is the control signal for the autonomous driving system, which also includes an acceleration control signal and a deceleration control signal ; is the vehicle speed; Vehicle acceleration; Road gradient; Coefficient of road surface friction; is the distance between the host vehicle and the vehicle ahead; is the collision risk coefficient; is a function based on the vehicle state and the control signals of the driver with lower limb disabilities and the autonomous driving system, expressed as:

[0046]

[0047] In the formula, is the maximum acceleration control signal; is the maximum deceleration control signal; is the given speed The maximum acceleration at; , , are weight coefficients used to balance the influence of the depth of the downward position of the steering wheel, the depth of the upward position of the steering wheel, and the acceleration factor on the game result, and ;

[0048] is a function based on road safety and environmental factors, expressed as:

[0049]

[0050] In the formula, is the given speed The ideal road gradient required for the vehicle to stay in the lane at; is the maximum gradient that the vehicle can safely pass; is the given speed The minimum distance from the vehicle ahead required for the vehicle to ensure safety at; , , , are weight coefficients used to balance the influence of road gradient, coefficient of road surface friction, distance from the vehicle ahead, and collision risk factor on the game result, and ;

[0051] S32. Send the game result of the longitudinal control signal to VCU6;

[0052] S33. The VCU sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus to control the drive and brake execution motors and the steering execution motor to work, thereby realizing autonomous driving.

[0053] In a second aspect, the present invention also provides an intelligent driving system for lower-limb disabled drivers, which is used to implement the intelligent driving method for lower-limb disabled drivers described above, and includes:

[0054] A lower-limb disabled driver manipulation module, which is used for a lower-limb disabled driver to manipulate the rotation and vertical movement of the steering wheel 1 according to the surrounding driving environment information. The vertical position sensor of the steering wheel obtains the vertical displacement of the steering wheel, and the steering wheel angle sensor 4 and the steering wheel torque sensor 5 obtain the angle and torque of the steering wheel. The driver manipulation module obtains the acceleration, braking and steering intentions of the driver.

[0055] An autonomous driving module, which is used to receive the environmental information obtained by the perception system in the autonomous driving module and output a vehicle longitudinal control signal through an end-to-end decision model.

[0056] A human-machine integration module, which is used to receive the control signals output by the driver manipulation module and the autonomous driving module, obtain the final control signal through human-machine driving right allocation, and the VCU sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus.

[0057] A control execution module, which is used to control the drive and brake execution motors and the steering execution motor to work, thereby realizing autonomous driving.

[0058] Further, the lower-limb disabled driver manipulation module includes a steering wheel 1, a vertical position sensor of the steering wheel 2, a main controller 3, a steering wheel angle sensor 4, a steering wheel torque sensor 5, a road feeling motor 7, a drive and brake motor 8, a steering motor 9, an axle 10, a redundant motor 11, a steering column 12, a slide bar 13, a guiding slider 14, a first sleeve 15, a brush 16, a second sleeve 17, a wire 18, a pin 19, a plug socket 20, a bottom plate 21, a substrate 22, a cushion plate 23, a spring 24, a magnet 25, a housing 26, a third sleeve 27 and a fourth sleeve 28;

[0059] The steering wheel 1 is connected to the steering column 12 by a pin shaft; the steering wheel angle sensor 4 is installed above the steering column 12, below the steering wheel, and connected to the top of the steering column 12; the steering wheel vertical position sensor 2 is embedded inside the steering column 12; the road feel motor 7 is installed at the lower end of the steering column 12 and is mechanically connected to the steering column 12 through a spline; the main controller 3, the steering wheel torque sensor 5, and the redundant motor 11 are installed at the lower end of the steering column 12; the redundant motor 11 is connected to the steering column 12 and directly applies torque to the steering column 12 when the main motor fails.

[0060] A driver with lower limb disabilities manipulates the steering wheel 1 to rotate and move vertically up and down. The steering wheel vertical position sensor 2 obtains the longitudinal driving intention and transmits it to the main controller 3; the steering wheel angle sensor 4 and the steering wheel torque sensor 5 obtain the lateral driving intention signals and transmit them to the main controller 3; the VCU 6 receives the longitudinal and lateral control signals of the driver with lower limb disabilities from the main controller 3 and then transmits them to the drive and brake motor 8 and the steering motor 9 respectively to control the drive force distribution and corner change of the wheel and the axle 10; at the same time, the VCU 6 controls the road feel motor 7 based on the road condition information detected and collected in real time, outputs the road feel feedback torque command in real time, applies an appropriate road feel to the driver, and returns the steering wheel 1 to the straight position.

[0061] The guide slider 14 meshes with the housing 26; the first sleeve 15 and the third sleeve 27 are arranged on both sides of the guide slider 14; the inner side of the first sleeve 15 is in contact with the outer side of the second sleeve 17 on the steering column 12; the inner side of the third sleeve 27 is in contact with the outer side of the fourth sleeve 28 on the steering column 12; the steering column 12 and the backing plate 23 are connected through a card slot; two springs 24 are arranged between the first sleeve 15 and the third sleeve 27 and the housing 26; the steering wheel 1 is connected to the guide slider 14 through a slide bar 13, and through the longitudinal movement of the steering wheel 1, the guide slider 14 is driven to perform longitudinal displacement on the guide rail of the housing 26; there is a magnet 25 on the guide slider 14, and the longitudinal displacement of the guide slider 14 drives the magnet 25 to move, causing a change in the surrounding magnetic field, thereby generating a longitudinal displacement signal; the longitudinal displacement signal is received by the brush 16; the bottom plate 21 is a convex structure and meshes with the backing plate 23 with a groove structure to form a cavity chamber; there is a substrate 22 in the cavity chamber; a wire 18 is inserted on the substrate 22 and connected to the brush 16; one end of the pin 19 is also connected to the substrate 22, and the other end is connected to the plug socket 20. The longitudinal displacement signal is collected by sensing the magnetic field change through the brush 16, and then transmitted to the pin 19 through the wire 18 and the substrate 22, and finally the vertical displacement signal is transmitted to the plug socket 20, and the longitudinal control signal is output by the plug socket 20 and transmitted to the main controller 3 through the signal transmission line.

[0062] The groove on the sliding rod 13 meshes with the card slot on the steering column 12. The torque exerted by the driver on the steering wheel is transmitted through the card slot, and then transmitted to the steering wheel angle sensor 4 and the steering wheel torque sensor 5 through the housing 26, and the steering control signal is output, and then transmitted to the main controller 3 through the signal transmission line;

[0063] The main controller 3 encodes the longitudinal and lateral control signals of the lower limb disabled driver and transmits them to the VCU 6. The VCU 6 directly transmits the encoded lateral control signal to the steering motor 9 and the drive braking motor, realizing the control of the vehicle by the lower limb disabled driver.

[0064] Further, the guiding slider 14 meshes with the limiting slot on the housing 26; when the limiting slot is closed, the guiding slider 14 cannot move, and the lower limb disabled driver cannot generate a longitudinal control signal; when the limiting slot is opened, the guiding slider 14 can move, and the lower limb disabled driver can generate a longitudinal control signal.

[0065] The beneficial effects of the present invention are as follows:

[0066] 1. The intelligent driving method and device for lower limb disabled drivers described in the present invention provide an intelligent driving method for lower limb disabled drivers to control the vehicle longitudinally and laterally, improve the driving safety of lower limb disabled drivers, and enhance the driving experience of the drivers;

[0067] 2. The intelligent driving method and device for lower limb disabled drivers described in the present invention provide a longitudinal driving intention acquisition device for lower limb disabled drivers, which converts the steering wheel position and speed change information into digital signals through the steering wheel position sensor to obtain the driving intentions of acceleration and deceleration;

[0068] 3. The intelligent driving method and device for lower limb disabled drivers described in the present invention provide a human-machine integration method for lower limb disabled drivers. Combining the longitudinal decision-making method of autonomous driving based on deep learning and the longitudinal driving intention of the driver, a human-machine driving right allocation strategy is designed to achieve effective human-machine integration. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0070] Figure 1 It is a logic schematic diagram of the intelligent driving device for lower limb disabled drivers described in the present invention;

[0071] Figure 2 Schematic flow chart of the intelligent driving method for lower-limb disabled drivers according to the present invention;

[0072] Figure 3 Schematic structural diagram of the control system for lower-limb disabled drivers according to the present invention;

[0073] Figure 4 Schematic assembly diagram of the control system for lower-limb disabled drivers according to the present invention;

[0074] Figure 5 Schematic perspective partial sectional view of the vertical position sensor of the steering wheel according to the present invention;

[0075] Figure 6 Schematic plan sectional view of the vertical position sensor of the steering wheel according to the present invention;

[0076] Figure 7 Enlarged schematic plan sectional view of the vertical position sensor of the steering wheel according to the present invention.

[0077] In the figure:

[0078] 1. Steering wheel; 2. Vertical position sensor of the steering wheel; 3. Main controller; 4. Steering wheel angle sensor; 5. Steering wheel torque sensor; 6. VCU; 7. Road feel motor; 8. Driving and braking motor; 9. Steering motor; 10. Axle; 11. Redundant motor; 12. Steering column; 13. Slide bar; 14. Guide slider; 15. First sleeve; 16. Brush; 17. Second sleeve; 18. Wire; 19. Pin; 20. Plug socket; 21. Base plate; 22. Substrate; 23. Pad; 24. Spring; 25. Magnet; 26. Housing; 27. Third sleeve; 28. Fourth sleeve. Specific embodiments

[0079] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. Additionally, it should be noted that for the sake of description, only parts related to the present invention are shown in the accompanying drawings, rather than all the structures.

[0080] Embodiment 1

[0081] This embodiment provides an intelligent driving method for lower-limb disabled drivers, including the following steps:

[0082] Step 1: The driver with lower limb disability manipulates the rotation and vertical movement of the steering wheel 1 according to the surrounding driving environment information. The vertical position sensor 2 of the steering wheel obtains the vertical displacement of the steering wheel. The steering wheel angle sensor 4 and the steering wheel torque sensor 5 obtain the angle and torque of the steering wheel. The driver control module obtains the acceleration, braking, and steering intentions of the driver; the VCU 6 obtains the acceleration, braking, and steering signals from the driver control module.

[0083] Step 2: The VCU 6 receives the environment information obtained by the perception system in the autonomous driving module and outputs the vehicle longitudinal control signal through an end-to-end decision model. Among them, the autonomous driving module includes a perception system, a positioning and navigation system, and a decision-making and planning system, specifically as follows:

[0084] S21: The perception system perceives and collects the original data of the surrounding environment through cameras, millimeter-wave radars, lidars, and ultrasonic radars; the original data is preprocessed and then fused through Kalman filtering to obtain the target-level data x of traffic participants, where x = {l h, l v, v h, v v, a s, a v, a h}, where l h is the lateral position of the traffic participant; l v is the longitudinal position; v h is the lateral speed; v v is the longitudinal speed; a s is the steering wheel angle of the host vehicle; a v is the longitudinal acceleration of the host vehicle; a h is the lateral acceleration, and then the target machine data is transmitted to the decision-making and planning system;

[0085] Among them, the perception system first preprocesses the original signals of different sensors to handle the noise between different sensors. Reasonably allocate the working frequency bands of sensors and use frequency modulation anti-interference technology to solve the channel conflict problem. For the signals from the camera, first use a Gaussian kernel to smooth the image for noise reduction to reduce Gaussian noise. Adopt an edge detection filter to highlight the edges and details in the image for enhancement processing, and use a convolutional neural network to extract image features; for the signals from the millimeter-wave radar, perform a fast Fourier transform on the echo signal to extract the distance, speed, and angle information of the target. Specifically, convert the echo signal in the time domain into a frequency domain signal and analyze the frequency components of the signal. Calculate the frequency components of the signal and use the operating frequency of the radar and the signal propagation time to calculate the distance of the target. Utilize the Doppler effect to calculate the frequency offset of the echo signal. Calculate the phase difference of the signal received by the phased array antenna and use the angle-of-arrival algorithm to calculate the angle of the target; for the signals from the lidar, the system filters and envelopes the scanned point cloud data to extract point cloud features including curvature and normal. Remove outliers and noise through statistical filtering and Gaussian filtering and retain the point cloud structure features. Calculate the normal using local plane fitting and extract the geometric features of the point cloud through principal curvature and mean curvature; for the ultrasonic radar signals, the system uses a low-pass filter to remove high-frequency noise and retain the useful echo signals. Perform a Hilbert transform on the filtered signal to extract the envelope of the signal, reflecting the change in the amplitude of the echo signal over time. Measure the time delay of the ultrasonic wave from transmission to reception of the echo. Calculate the relative distance between different targets and the vehicle according to the echo time delay.

[0086] The specific method for fusing the preprocessed raw data through Kalman filtering is as follows:

[0087] Affected by environmental interferences such as rain, snow, and fog, the accuracy of the sensors themselves, and the interference between sensors or external electromagnetic interference, the measurement noises of the camera, millimeter-wave radar, lidar, and ultrasonic radar are respectively , , , , then the measurement signals from different sensors are expressed as:

[0088]

[0089]

[0090]

[0091]

[0092] In the formula, , , , Measurement signals from a camera, a millimeter-wave radar, a lidar, and an ultrasonic radar, respectively; , , , are the corresponding measurement matrices, and the obtained measurement signal is ;

[0093] Among them, ;

[0094] The state vector of the dynamic system is , and the state transition equation and the measurement equation are expressed as:

[0095]

[0096]

[0097] In the formula, is the system state transition matrix; is the system control input matrix; is the system control input; is the process noise, and the corresponding covariance matrix is ; is the measurement matrix; is the measurement noise, and the corresponding covariance matrix is ; Predict and update the system state, and the prediction steps are as follows:

[0098] State prediction:

[0099]

[0100] Covariance prediction:

[0101]

[0102] Update and correct the system according to the state prediction result, and the update steps are as follows:

[0103] Kalman gain calculation:

[0104]

[0105] State update:

[0106]

[0107] Covariance update:

[0108]

[0109] Through this operation, the accuracy and robustness of the vehicle's real-time perception of the surrounding environment can be improved, ensuring that the target-level data of traffic participants obtained by the perception system is input into the decision-making and planning system.

[0110] S22. The positioning information collected by the positioning and navigation system is synchronously transmitted into the decision-making and planning system; the positioning system obtains the vehicle's positioning information in real time and transmits the positioning information into the decision-making and planning system;

[0111] S23. The decision-making and planning system passes through the longitudinal decision-making model for autonomous driving based on deep learning. The longitudinal decision-making model for autonomous driving transmits the collected perception data and positioning data to the VCU, makes decisions automatically through calculation, and outputs the longitudinal control signal of the vehicle.

[0112] Step 3. The human-machine fusion module receives the control signals output by the driver control module and the autonomous driving module, obtains the final control signal through human-machine driving right allocation, and the VCU6 sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus, specifically as follows:

[0113] S31. Design a game formula for longitudinal control signals considering multiple factors, as shown below:

[0114]

[0115] In the formula, and are weight coefficients used to balance the influence of factors such as vehicle state and road environment on the game result, and . represents the control signal of a driver with lower limb disability, including the depth of the downward position of the steering wheel indicating the intention of acceleration (range 0~1, 0 represents the initial position of the steering wheel, 1 represents the maximum downward travel position of the steering wheel) and the depth of the upward position of the steering wheel indicating the intention of deceleration (range -1~0, 0 represents the initial position of the steering wheel, -1 represents the maximum upward travel position of the steering wheel). represents the control signal of the autonomous driving system, which also includes the acceleration control signal and the deceleration control signal . The vehicle speed is represented by , the vehicle acceleration is , the road gradient is , the road surface friction coefficient is , the distance from the vehicle in front is represented by , and the collision risk coefficient is represented by .

[0116] is a function based on the vehicle state and the control signals of the lower-limb disabled driver and the automatic driving system, expressed as:

[0117]

[0118] In the formula, is the maximum acceleration control signal; is the maximum deceleration control signal; is the given speed The maximum acceleration at; , , are weight coefficients used to balance the influence of the downward position depth of the steering wheel, the upward position depth of the steering wheel, and the acceleration factor on the game result, and ;

[0119] is a function based on road safety and environmental factors, expressed as:

[0120]

[0121] In the formula, is the given speed The ideal road slope required for the vehicle to stay in the lane at; is the maximum slope that the vehicle can safely pass; is the given speed The minimum distance from the vehicle in front required to ensure safety at; , , , are weight coefficients used to balance the influence of road slope, road surface friction coefficient, distance from the vehicle in front, and collision risk factor on the game result, and ;

[0122] S32. Send the game result of the longitudinal control signal to VCU6;

[0123] S33. VCU6 sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus to control the drive, brake execution motor, and steering execution motor to work, so as to realize automatic driving.

[0124] Step 4. The control execution module controls the drive, brake execution motor, and steering execution motor to work, so as to realize automatic driving.

[0125] Embodiment 2

[0126] Refer to Figures 2 - 7, the present invention also provides an intelligent driving device for lower-limb disabled drivers, which is used to implement the intelligent driving method for lower-limb disabled drivers described in Embodiment 1, and includes:

[0127] A. Lower-limb disabled driver control module, which is used for the lower-limb disabled driver to manipulate the steering wheel 1 to rotate and move vertically according to the surrounding driving environment information. The vertical position sensor 2 of the steering wheel obtains the vertical displacement of the steering wheel, and the steering wheel angle sensor 4 and the steering wheel torque sensor 5 obtain the angle and torque of the steering wheel. The driver control module obtains the acceleration, braking, and steering intentions of the driver;

[0128] The lower-limb disabled driver control module includes a steering wheel 1, a vertical position sensor 2 of the steering wheel, a main controller 3, a steering wheel angle sensor 4, a steering wheel torque sensor 5, a road feel motor 7, a drive and brake motor 8, a steering motor 9, an axle 10, a redundant motor 11, a steering column 12, a slide bar 13, a guide slider 14, a first sleeve 15, a brush 16, a second sleeve 17, a wire 18, a pin 19, a plug socket 20, a base plate 21, a substrate 22, a backing plate 23, a spring 24, a magnet 25, a housing 26, a third sleeve 27, and a fourth sleeve 28;

[0129] The steering wheel 1 is connected to the steering column 12 through a pin shaft. When the steering wheel 1 rotates, it drives the steering column 12 to rotate synchronously. The steering wheel angle sensor 4 is installed above the steering column 12, below the steering wheel 1, and is connected to the top of the steering column 12. The vertical position sensor 2 of the steering wheel is embedded inside the steering column 12. The road feel motor 7 is installed at the lower end of the steering column 12 and is mechanically connected to the steering column 12 through a spline. The main controller 3, the steering wheel torque sensor 5, and the redundant motor 11 are installed at the lower end of the steering column 12. The lower-limb disabled driver manipulates the steering wheel 1 to rotate and move vertically up and down. The vertical position sensor 2 of the steering wheel obtains the longitudinal driving intention and transmits it to the main controller 3. The steering wheel angle sensor 4 and the steering wheel torque sensor 5 obtain the lateral driving intention signals and transmit them to the main controller 3. The VCU 6 receives the longitudinal and lateral control signals of the lower-limb disabled driver from the main controller 3, and then transmits them to the drive and brake motor 8 and the steering motor 9 respectively to control the drive force distribution and angle change between the wheels and the axle 10. At the same time, the VCU 6 also controls the road feel motor 7 based on the real-time traffic condition information detected and collected, and outputs the road feel feedback torque command in real time to apply appropriate road feel to the driver and return the steering wheel to the neutral position.

[0130] The three-dimensional partial cross-sectional view of the vertical position sensor 2 of the steering wheel is as Figure 5As shown in the figure. The internally slidable guiding slider 14 engages with the limiting groove on the housing 26. When the limiting groove is closed, the guiding slider 14 cannot move, and the lower-limb disabled driver cannot generate a vertical control signal; when the limiting groove is opened, the guiding slider 14 can move, and the lower-limb disabled driver can generate a vertical control signal. Additionally, on both sides of the guiding slider 14, there are a first sleeve 15 and a third sleeve 27. The inner side of the first sleeve 15 is tightly connected to the outer side of the second sleeve 17 on the steering column 12, and the inner side of the third sleeve 27 is tightly connected to the outer side of the fourth sleeve 28 on the steering column 12. The steering column 12 and the backing plate 23 are connected through a slot structure, ensuring the longitudinal movement of the guiding slider. At the same time, between the first sleeve 15, the third sleeve 27 and the housing 26, two springs 24 are installed to ensure that when no external force is applied by the lower-limb disabled driver, the guiding slider 14 can return to the middle position on the guide rail of the housing 26. The steering wheel 1 is connected to the guiding slider 14 through a slide bar 13. Through the longitudinal movement of the steering wheel 1, the guiding slider 14 is driven to perform longitudinal displacement on the guide rail of the housing 26. And there is a magnet 25 on the guiding slider 14. Through the longitudinal displacement of the guiding slider 14, the magnet 25 is driven to move, causing a change in the surrounding magnetic field, and thus generating a longitudinal displacement signal. The brush 16 receives this longitudinal displacement signal. When the guiding slider 14 drives the magnet 25 to move backward, the magnetic field intensity received by the brush 16 weakens, generating a decelerating longitudinal control signal. When the guiding slider 14 drives the magnet 25 to move forward, the magnetic field intensity received by the brush 16 increases, generating an accelerating longitudinal control signal.

[0131] Amplified plan sectional view of the vertical position sensor of the steering wheel, as Figure 7 As shown in the figure, the bottom plate 21 is a convex structure, which engages with the backing plate 23 with a groove structure to form a cavity chamber. There is a substrate 22 in the cavity chamber. A wire 18 is inserted on the substrate 22 and connected to the brush 16. At the same time, one end of the pin 19 is also connected to the substrate 22, and the other end is connected to the plug socket 20. The longitudinal displacement signal is collected by sensing the magnetic field change through the brush 16, and then transmitted to the pin 19 through the wire 18 and the substrate 22, and finally the longitudinal displacement signal is transmitted to the plug socket 20. The plug socket 20 outputs a longitudinal control signal, which is transmitted to the main controller 3 through a signal transmission line.

[0132] The groove on the sliding rod 13 meshes with the clamping groove on the steering column 12. The torque exerted by the driver on the steering wheel is transmitted through the clamping groove, and then transmitted to the steering wheel angle sensor 4 and the steering wheel torque sensor 5 through the housing 26 to output a steering control signal, which is then transmitted to the main controller 3 through a signal transmission line. Finally, the main controller 3 encodes the longitudinal and lateral control signals of the lower-limb disabled driver into the CAN bus communication protocol format. Each signal has a unique identifier and data format. The longitudinal signal is encoded into a frame containing information such as vehicle speed and acceleration request. The lateral signal is encoded into a frame containing information such as steering angle and steering speed. Then it is transmitted to the VCU 6, and the VCU 6 directly transmits the encoded lateral control signal to the steering motor 9 to achieve the control of the vehicle steering system by the lower-limb disabled driver. For the vertical control signal, the VCU 6 analyzes its rationality with the longitudinal control signal from the autonomous driving system, and after completing the allocation of the driving right, it is transmitted to the drive and brake motor to achieve the acceleration and deceleration of the vehicle.

[0133] B. An autonomous driving module, which is used to receive the environmental information obtained by the perception system in the autonomous driving module and output the longitudinal control signal of the whole vehicle through an end-to-end decision-making model;

[0134] C. A human-machine integration module, which is used to receive the control signals output by the driver operation module and the autonomous driving module, obtain the final control signal through the allocation of the human-machine driving right, and the VCU sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus;

[0135] D. A control execution module, which is used to control the drive and brake execution motors and the steering execution motor to work, so as to achieve autonomous driving.

[0136] In summary, the present invention can obtain the driving intention of acceleration and deceleration by converting the steering wheel position and speed change information into digital signals through the steering wheel position sensor with the steering wheel that can move up and down, and develop an autonomous driving longitudinal decision-making algorithm based on deep learning. Through the allocation of the driving right, the effective integration of humans and machines is achieved to complete the control of the intelligent vehicle.

[0137] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent driving method for lower-limb disabled drivers, characterized in that, It includes the following steps: Step 1: The driver with lower limb disability manipulates the steering wheel (1) to rotate and move vertically according to the surrounding driving environment information. The vertical position sensor of the steering wheel obtains the vertical displacement of the steering wheel, and the steering wheel angle sensor and the steering wheel torque sensor obtain the angle and torque of the steering wheel. The driver manipulation module obtains the acceleration, braking, and steering intentions of the driver, that is, the steering wheel position and speed change information are converted into digital signals through the vertical position sensor of the steering wheel (2) to obtain the driving intentions of acceleration and deceleration; the VCU (6) obtains the acceleration, braking, and steering signals of the driver manipulation module; Step 2: The VCU (6) receives the environment information obtained by the perception system in the autonomous driving module and outputs the vehicle longitudinal control signal through an end-to-end decision model; Step 3: The human-machine fusion module receives the control signals output by the driver manipulation module and the autonomous driving module, obtains the final control signal through the allocation of human-machine driving rights, and the VCU (6) sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus; Step 4: The control execution module controls the drive and brake execution motors and the steering execution motor to work, thereby realizing autonomous driving.

2. The intelligent driving method for lower-limb disabled drivers according to claim 1, characterized in that The autonomous driving module includes a perception system, a positioning and navigation system, and a decision-making and planning system.

3. The intelligent driving method for lower-limb disabled drivers according to claim 1, characterized in that The specific method of the above Step 2 is as follows: S21. The perception system senses and collects the raw data of the surrounding environment through cameras, millimeter-wave radars, lidars, and ultrasonic radars; the raw data is preprocessed and then fused through Kalman filtering to obtain the target-level data x of traffic participants, where x = {l h, l v, v h, v v, a s, a v, a h}, where l h is the lateral position of the traffic participant; l v is the longitudinal position; v h is the lateral speed; v v is the longitudinal speed; a s is the steering wheel angle of the host vehicle; a v is the longitudinal acceleration of the host vehicle; a h is the lateral acceleration, and then the target machine data is passed to the decision-making and planning system; S22: The positioning information collected by the positioning and navigation system is synchronously transmitted into the decision-making and planning system; the positioning system obtains the vehicle positioning information in real time and transmits the positioning information into the decision-making and planning system; S23: The decision-making and planning system passes through an autonomous driving longitudinal decision model based on deep learning. The autonomous driving longitudinal decision model transmits the collected perception data and positioning data to the VCU, makes a decision automatically after calculation, and outputs the vehicle longitudinal control signal.

4. The intelligent driving method for lower-limb disabled drivers according to claim 3, wherein The camera collects RGB image information; the millimeter-wave radar collects raw ADC data; the lidar collects 3D point cloud information; the ultrasonic radar collects raw data.

5. The intelligent driving method for lower-limb disabled drivers according to claim 3, wherein The specific method for fusing the raw data after preprocessing through Kalman filtering is as follows: The measurement noises of the camera, millimeter-wave radar, lidar, and ultrasonic radar are obtained as v c , v r , v l , v u . Then the measurement signals from different sensors are expressed as: z k,1 = H c x + v c z k,2 = H r x + v r z k,3 = H l x + v l z k,4 = H u x + v u where z k,1 , z k,2 , z k,3 , z k,4 are the measurement signals of the camera, millimeter wave radar, lidar, and ultrasonic radar respectively; H c , H r , H l , H u are the corresponding measurement matrices, and the obtained measurement signal is z k ; Among them, The state vector of the dynamic system is x k , and the state transition equation and measurement equation are expressed as: x k = F k x k-1 + B k u k + w k z k = H k x k + v k where F k is the system state transition matrix; B k is the system control input matrix; u k is the system control input; w k is the process noise, and the corresponding covariance matrix is Q k ; H k is the measurement matrix; v k is the measurement noise, and the corresponding covariance matrix is R k ; Predict and update the system state, and the prediction steps are as follows: S1. State prediction: x k|k-1 = F k x k-1|k-1 + B k u k where x k|k-1 is the state at the past k - 1 time instants, for predicting the state at time instant k; x k-1|k-1 is the state estimate at time instant k - 1; S2. Covariance prediction: where, P k|k-1 is the state at the past k - 1 moments, the covariance matrix of the state prediction at the kth moment, which is used to evaluate the accuracy of the estimated value, P k-1|k-1 ; represents the covariance matrix of the state estimation at the k - 1 moments; S3: Update and correct the system according to the state prediction result, specifically as follows: Kalman gain calculation: where K k is the Kalman gain; State update: x k|k = x k|k-1 + K k (z k - H k x k|k-1 ) Covariance update: P k|k = (I - K k H k )P k|k-1 6. The intelligent driving method for lower limb disabled drivers according to claim 1, wherein The specific method of the above Step 3 is as follows: S31: Design a longitudinal control signal game formula considering multiple factors, as shown below: S = α·f(S d , S a , v, a x ) + β·g(θ, μ, d f , R c ) Where α and β are weight coefficients used to balance the influence of factors such as vehicle state and road environment on the game result, and α + β = 1; S d is the control signal of the driver with lower limb disability, including the depth a of the downward position of the steering wheel indicating the intention of acceleration d and the depth b of the upward position of the steering wheel indicating the intention of deceleration d ; S a is the control signal of the automatic driving system, which also includes the acceleration control signal a a and the deceleration control signal b a ; v is the vehicle speed; a x is the vehicle acceleration; θ is the road gradient; μ is the road surface friction coefficient; d f is the distance between the host vehicle and the vehicle ahead; R c is the collision risk coefficient; f(S d , S a , v, a x ) is a function based on the vehicle state and the control signals of the driver with lower limb disability and the automatic driving system, expressed as: where a max is the maximum acceleration control signal; b max is the maximum deceleration control signal; is the maximum acceleration at the given speed v; ω1, ω2, and ω3 are weight coefficients used to balance the influence of the downward position depth of the steering wheel, the upward position depth of the steering wheel, and the acceleration factor on the game result, and ω1 + ω2 + ω3 = 1; g(θ,μ,d f ,R c ) is a function based on road safety and environmental factors, expressed as: where θ req(v) is the ideal road gradient required for the vehicle to stay within the lane at a given speed v; θ max is the maximum gradient that the vehicle can safely pass; is the minimum distance from the vehicle in front required to ensure safety at a given speed v; ω4, ω5, ω6, ω7 are weight coefficients used to balance the impacts of road gradient, road surface friction coefficient, distance from the vehicle in front, and collision risk factors on the game result, and ω4 + ω5 + ω6 + ω7 = 1; S32: Send the longitudinal control signal game result to the VCU; S33: The VCU sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus to control the drive and brake execution motors and the steering execution motor to work, thereby realizing autonomous driving.

7. An intelligent driving device for lower-limb disabled drivers, which is used to implement an intelligent driving method for lower-limb disabled drivers according to any one of claims 1-6, characterized in that, It includes: A driver manipulation module for drivers with lower limb disabilities, which is used for drivers with lower limb disabilities to manipulate the steering wheel (1) to rotate and move vertically according to the surrounding driving environment information. The vertical position sensor of the steering wheel (2) obtains the vertical displacement of the steering wheel, and the steering wheel angle sensor (4) and the steering wheel torque sensor (5) obtain the angle and torque of the steering wheel. The driver manipulation module obtains the acceleration, braking, and steering intentions of the driver; An autonomous driving module, which is used to receive the environmental information obtained by the perception system in the autonomous driving module and output a vehicle longitudinal control signal through an end-to-end decision-making model; A human-machine integration module, which is used to receive the control signals output by the driver operation module and the autonomous driving module, obtain the final control signal through human-machine driving authority allocation, and the VCU sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus; A control execution module, which is used to control the drive and brake execution motors and the steering execution motor to work, so as to realize autonomous driving.

8. An intelligent driving device for lower-limb disabled drivers according to claim 7, characterized in that, The lower-limb disabled driver operation module includes a steering wheel (1), a steering wheel vertical position sensor (2), a main controller (3), a steering wheel angle sensor (4), a steering wheel torque sensor (5), a road feel motor (7), a drive and brake motor (8), a steering motor (9), an axle (10), a redundant motor (11), a steering column (12), a slide bar (13), a guide slider (14), a first sleeve (15), a brush (16), a second sleeve (17), a wire (18), a pin (19), a plug socket (20), a bottom plate (21), a substrate (22), a backing plate (23), a spring (24), a permanent magnet (25), a housing (26), a third sleeve (27) and a fourth sleeve (28); The steering wheel (1) is connected to the steering column (12) through a pin shaft; the steering wheel angle sensor (4) is installed above the steering column (12) and below the steering wheel (1), and is connected to the top of the steering column (12); the steering wheel vertical position sensor (2) is embedded inside the steering column (12); the road feel motor (7) is installed at the lower end of the steering column (12) and is mechanically connected to the steering column (12) through a spline; the main controller (3), the steering wheel torque sensor (5) and the redundant motor (11) are installed at the lower end of the steering column (12); the redundant motor (11) is connected to the steering column (12), and when the main motor fails, it directly applies torque to the steering column (12); the lower-limb disabled driver operates the steering wheel (1) to rotate and move vertically up and down, and the steering wheel vertical position sensor (2) obtains the longitudinal driving intention and transmits it to the main controller (3); the steering wheel angle sensor (4) and the steering wheel torque sensor (5) obtain the lateral driving intention signal and transmit it to the main controller (3); the VCU (6) receives the longitudinal and lateral control signals of the lower-limb disabled driver from the main controller (3), and then transmits them to the drive and brake motor (8) and the steering motor (9) respectively to control the drive force distribution and angle change of the wheel and the axle (10); at the same time, the VCU (6) controls the road feel motor (7) based on the road condition information detected and collected in real time, and outputs a road feel feedback torque command in real time to apply road feel to the driver and return the steering wheel (1) to the straight position; The guiding slider (14) is engaged with the housing (26); the first sleeve (15) and the third sleeve (27) are arranged on both sides of the guiding slider (14); the inner side of the first sleeve (15) is in contact with the outer side of the second sleeve (17) on the steering column body (12); the inner side of the third sleeve (27) is in contact with the outer side of the fourth sleeve (28) on the steering column body (12); the steering column body (12) and the backing plate (23) are connected through a clamping groove; two springs (24) are arranged between the first sleeve (15) and the third sleeve (27) and the housing (26); the steering wheel (1) is connected with the guiding slider (14) through a slide bar (13), and through the longitudinal movement of the steering wheel (1), the guiding slider (14) is driven to perform longitudinal displacement on the guide rail of the housing (26); a magnet (25) is arranged on the guiding slider (14), and through the longitudinal displacement of the guiding slider (14), the magnet (25) is driven to move, causing a change in the surrounding magnetic field, and further generating a longitudinal displacement signal; the longitudinal displacement signal is received by the brush (16); the bottom plate (21) is a convex structure and is engaged with the groove-structured backing plate (23) to form a cavity chamber; a substrate (22) is arranged in the cavity chamber; a wire (18) is inserted on the substrate (22) and is connected with the brush (16); one end of the pin (19) is also connected with the substrate (22), and the other end is connected with the plug socket (20). The longitudinal displacement signal is collected by sensing the magnetic field change through the brush (16), and then is transmitted to the pin (19) through the wire (18) via the substrate (22), and finally the vertical displacement signal is transmitted to the plug socket (20), and the longitudinal control signal is output by the plug socket (20) and is transmitted to the main controller (3) through the signal transmission line; The groove on the slide bar (13) is engaged with the clamping groove on the steering column body (12), and the torque applied by the driver to the steering wheel is transmitted through the clamping groove, and then is transmitted to the steering wheel angle sensor (4) and the steering wheel torque sensor through the housing (26), and the steering control signal is output and then is transmitted to the main controller (3) through the signal transmission line; The main controller (3) encodes the longitudinal and lateral control signals of the lower-limb disabled driver and transmits them to the VCU (6), and the VCU (6) directly transmits the encoded lateral control signal to the steering motor (9) and the drive and brake motor, so as to realize the control of the vehicle by the lower-limb disabled driver.

9. An intelligent driving device for lower-limb disabled drivers according to claim 8, characterized in that, The guiding slider (14) is engaged with the limiting groove on the housing (26); when the limiting groove is closed, the guiding slider (14) cannot move, and the lower-limb disabled driver cannot generate a longitudinal control signal; when the limiting groove is opened, the guiding slider (14) can move, and the lower-limb disabled driver can generate a longitudinal control signal.

Images