Intelligent driving method and device for driver with disabled lower limbs

Abstract

The invention belongs to the technical field of automatic driving testing, and particularly relates to an intelligent driving method and device for a driver with disabled lower limbs. The system comprises a lower limb disabled driver operation module, an automatic driving module, a man-machine fusion module and a control execution module. According to the method, the position and speed change information of a steering wheel can be converted into digital signals through the steering wheel moving up and down and a steering wheel position sensor, the driving intention of acceleration and deceleration is obtained, an automatic driving longitudinal decision algorithm based on deep learning is developed, effective man-machine fusion is achieved through driving right distribution, and the driving efficiency is improved. And the control of the intelligent vehicle is completed.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving testing, and specifically relates to an intelligent driving method and device for disabled lower-limb drivers. Background Art

[0002] Driving a car by disabled people is an equal right and an important condition for them to participate in social life, and it is an important manifestation of the country's civilization progress, social security and management service level. With the continuous development of socialization, since disabled people do not have the ability to drive a vehicle independently, existing means of transportation cannot meet the travel needs of disabled drivers, causing great difficulties in daily travel and also affecting their mental health to a certain extent. According to statistics, there are tens of millions of disabled people in the country, and most of them have the will to drive a vehicle. Developing intelligent vehicles for disabled people not only has huge market potential, but also is of great significance for protecting the travel rights of disabled people, creating economic value and improving the people's happiness index.

[0003] With the continuous development of autonomous driving technology, the convenience of travel for disabled people has been greatly improved. However, at the present stage, mass-produced autonomous driving vehicles have not achieved high-level autonomous driving of L5 level, and autonomous driving or assisted driving is achieved under specific working conditions, and even still requires an assisted driver to act as a safety officer, which greatly limits the travel range of disabled drivers. In fact, disabled people have not completely lost the ability to control the vehicle. For example, a disabled lower-limb driver has normal thinking and hands controlled by the brain, and can control the rotation of the vehicle steering wheel, and has the possibility of realizing the individual as the main body of vehicle control. Therefore, developing an intelligent driving system for disabled lower-limb drivers and realizing the high integration of disabled drivers and the intelligent driving system can improve the driving safety during driving, enhance the trust in the intelligent driving system, and also ensure the driving experience of disabled drivers.

[0004] However, there is still a blank in the driving assistance device and intelligent driving system for disabled lower-limb drivers. Therefore, it is still very crucial to develop an intelligent driving method and device for disabled lower-limb drivers. Summary of the Invention

[0005] The present invention provides an intelligent driving method and device for disabled lower-limb drivers, which can convert the steering wheel position and speed change information into digital signals through a vertically movable steering wheel, obtain the driving intentions of acceleration and deceleration, and develop a longitudinal decision-making algorithm for autonomous driving based on deep learning to achieve effective human-machine integration through driving right allocation and complete the control of intelligent vehicles.

[0006] The technical solution of the present invention is described in conjunction with the accompanying drawings as follows:

[0007] In a first aspect, the present invention provides an intelligent driving method for lower-limb disabled drivers, comprising the following steps:

[0008] Step 1: According to the surrounding driving environment information, the lower-limb disabled driver manipulates the steering wheel 1 to rotate and move vertically. The vertical position sensor of the steering wheel obtains the vertical displacement of the steering wheel, and the steering wheel angle sensor 4 and the steering wheel torque sensor 5 obtain the angle and torque of the steering wheel. The driver manipulation module obtains the driver's acceleration, braking, and steering intentions; the VCU 6 obtains the acceleration, braking, and steering signals of the driver manipulation module;

[0009] Step 2: The VCU 6 receives the environment information obtained by the perception system in the autonomous driving module and outputs the vehicle longitudinal control signal through an end-to-end decision model;

[0010] Step 3: The human-machine integration module receives the control signals output by the driver manipulation module and the autonomous driving module, obtains the final control signal through human-machine driving right allocation, and the VCU 6 sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus;

[0011] Step 4: The control execution module controls the drive and brake execution motors and the steering execution motor to work, thereby realizing autonomous driving.

[0012] Further, the autonomous driving module includes a perception system, a positioning and navigation system, and a decision and planning system.

[0013] Further, the specific method of Step 2 is as follows:

[0014] S21: The perception system senses and collects the original data of the surrounding environment through cameras, millimeter-wave radars, lidars, and ultrasonic radars; the original data is fused through Kalman filtering after preprocessing to obtain the target-level data x of traffic participants, x = {l h, l v, v h, v v, a s, a v, a h}, where l h is the lateral position of the traffic participant; l v is the longitudinal position; v h is the lateral speed; v v is the longitudinal speed; a s is the steering wheel angle of the vehicle itself; a v is the longitudinal acceleration of the vehicle itself; a h is the lateral acceleration, and then the target machine data is transmitted to the decision and planning system;

[0015] S22. The positioning information collected by the positioning and navigation system is synchronously transmitted into the decision-making and planning system; the positioning system obtains the vehicle's positioning information in real time and transmits the positioning information into the decision-making and planning system.

[0016] S23. The decision-making and planning system uses a deep learning-based longitudinal decision-making model for autonomous driving. The autonomous driving longitudinal decision-making model transmits the collected perception data and positioning data to the VCU, makes a decision automatically after calculation, and outputs the vehicle's longitudinal control signal.

[0017] Furthermore, the camera collects RGB image information; the millimeter-wave radar collects raw ADC data; the lidar collects 3D point cloud information; the ultrasonic radar collects raw data.

[0018] Furthermore, the specific method for fusing the raw data after preprocessing through Kalman filtering is as follows:

[0019] Obtain the measurement noises of the camera, millimeter-wave radar, lidar, and ultrasonic radar as , , , , then the measurement signals from different sensors are expressed as:

[0020]

[0021]

[0022]

[0023]

[0024] In the formula, , , , are the measurement signals of the camera, millimeter-wave radar, lidar, and ultrasonic radar respectively; , , , are the corresponding measurement matrices, and the obtained measurement signal is ;

[0025] Among them, ;

[0026] The state vector of the dynamic system is , and the state transition equation and measurement equation are expressed as:

[0027]

[0028]

[0029] In the formula, is the system state transition matrix; is the system control input matrix; is the system control input; is the process noise, and the corresponding covariance matrix is ; is the measurement matrix; is the measurement noise, and the corresponding covariance matrix is ; Predict and update the system state, and the prediction steps are as follows:

[0030] S1. State prediction:

[0031] In the formula, is the state at the past k - 1 moments, for predicting the state at the kth moment; is the state estimate at the (k - 1)th moment;

[0032] S2. Covariance prediction:

[0033] In the formula, is the state at the past k - 1 moments, for predicting the covariance matrix of the state at the kth moment, which is used to evaluate the accuracy of the estimated value, , representing the covariance matrix of the state estimate at the (k - 1)th moment;

[0034] S3. Update and correct the system according to the state prediction result, specifically as follows:

[0035] Calculation of the Kalman gain:

[0036]

[0037] In the formula, is the Kalman gain;

[0038] State update:

[0039]

[0040] Covariance update:

[0041]

[0042] Furthermore, the specific method of step three is as follows:

[0043] S31. Design a longitudinal control signal game formula considering multiple factors, as follows:

[0044]

[0045] In the formula, and are weight coefficients, used to balance the influence of factors such as vehicle state and road environment on the game result, and ; is the control signal of the driver with lower limb disability, including the depth of the downward position of the steering wheel indicating the intention of acceleration and the depth of the upward position of the steering wheel indicating the intention of deceleration ; is the control signal of the autonomous driving system, also including the acceleration control signal and the deceleration control signal ; is the vehicle speed; Vehicle acceleration; Road gradient; Road surface friction coefficient; is the distance between the host vehicle and the preceding vehicle; is the collision risk coefficient; is a function based on the vehicle state and the control signals of the driver with lower limb disability and the autonomous driving system, expressed as:

[0046]

[0047] In the formula, is the maximum acceleration control signal; is the maximum deceleration control signal; is the given speed The maximum acceleration at; , , are weight coefficients used to balance the influence of the depth of the downward position of the steering wheel, the depth of the upward position of the steering wheel, and the acceleration factor on the game result, and ;

[0048] is a function based on road safety and environmental factors, expressed as:

[0049]

[0050] In the formula, is the given speed The ideal road gradient required for the vehicle to stay in the lane at; is the maximum gradient that the vehicle can safely pass through; is the given speed The minimum distance required from the preceding vehicle to ensure the safety of the vehicle at; , , , is a weight coefficient used to balance the influences of road slope, road surface friction coefficient, distance from the vehicle ahead, and collision risk factors on the game result, and ;

[0051] S32. Send the game result of the longitudinal control signal to VCU6;

[0052] S33. The VCU sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus to control the operation of the drive and brake execution motors and the steering execution motor, thereby realizing autonomous driving.

[0053] In a second aspect, the present invention also provides an intelligent driving system for lower-limb disabled drivers, which is used to implement the intelligent driving method for lower-limb disabled drivers, including:

[0054] A lower-limb disabled driver manipulation module, which is used for a lower-limb disabled driver to manipulate the rotation and vertical movement of the steering wheel 1 according to the surrounding driving environment information. The vertical position sensor of the steering wheel obtains the vertical displacement of the steering wheel, and the steering wheel angle sensor 4 and the steering wheel torque sensor 5 obtain the angle and torque of the steering wheel. The driver manipulation module obtains the acceleration, braking, and steering intentions of the driver;

[0055] An autonomous driving module, which is used to receive the environmental information obtained by the perception system in the autonomous driving module and output the vehicle longitudinal control signal through an end-to-end decision model;

[0056] A human-machine integration module, which is used to receive the control signals output by the driver manipulation module and the autonomous driving module, obtain the final control signal through human-machine driving right allocation, and the VCU sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus;

[0057] A control execution module, which is used to control the operation of the drive and brake execution motors and the steering execution motor, thereby realizing autonomous driving.

[0058] Further, the lower-limb disabled driver manipulation module includes a steering wheel 1, a vertical position sensor of the steering wheel 2, a main controller 3, a steering wheel angle sensor 4, a steering wheel torque sensor 5, a road feeling motor 7, a drive and brake motor 8, a steering motor 9, an axle 10, a redundant motor 11, a steering column 12, a slide bar 13, a guiding slider 14, a first sleeve 15, a brush 16, a second sleeve 17, a wire 18, a pin 19, a plug socket 20, a bottom plate 21, a substrate 22, a backing plate 23, a spring 24, a permanent magnet 25, a housing 26, a third sleeve 27, and a fourth sleeve 28;

[0059] The steering wheel 1 is connected to the steering column 12 by a pin shaft; the steering wheel angle sensor 4 is installed above the steering column 12, below the steering wheel, and connected to the top of the steering column 12; the steering wheel vertical position sensor 2 is embedded inside the steering column 12; the road feel motor 7 is installed at the lower end of the steering column 12 and is mechanically connected to the steering column 12 through a spline; the main controller 3, the steering wheel torque sensor 5, and the redundant motor 11 are installed at the lower end of the steering column 12; the redundant motor 11 is connected to the steering column 12, and when the main motor fails, it directly applies torque to the steering column 12.

[0060] The driver with lower limb disability manipulates the steering wheel 1 to rotate and move vertically up and down. The steering wheel vertical position sensor 2 obtains the longitudinal driving intention and transmits it to the main controller 3; the steering wheel angle sensor 4 and the steering wheel torque sensor 5 obtain the lateral driving intention signals and transmit them to the main controller 3; the VCU 6 receives the longitudinal and lateral control signals of the driver with lower limb disability from the main controller 3 and then transmits them to the drive brake motor 8 and the steering motor 9 respectively to control the drive force distribution and corner change of the wheel and the axle 10; at the same time, the VCU 6 controls the road feel motor 7 based on the road condition information detected and collected in real time, and outputs the road feel feedback torque command in real time to apply a suitable road feel to the driver and return the steering wheel 1 to the straight position.

[0061] The guiding slider 14 meshes with the housing 26; on both sides of the guiding slider 14, there are a first sleeve 15 and a third sleeve 27; the inner side of the first sleeve 15 is in contact with the outer side of the second sleeve 17 on the steering column 12; the inner side of the third sleeve 27 is in contact with the outer side of the fourth sleeve 28 on the steering column 12; the steering column 12 and the backing plate 23 are connected through a slot; between the first sleeve 15 and the third sleeve 27 and the housing 26, there are two springs 24; the steering wheel 1 is connected to the guiding slider 14 through a slide bar 13, and through the longitudinal movement of the steering wheel 1, the guiding slider 14 is driven to perform longitudinal displacement on the guide rail of the housing 26; there is a magnet 25 on the guiding slider 14, and through the longitudinal displacement of the guiding slider 14, the magnet 25 is driven to move, causing a change in the surrounding magnetic field, and thus generating a longitudinal displacement signal; the longitudinal displacement signal is received by the brush 16; the bottom plate 21 is a convex structure and meshes with the backing plate 23 with a groove structure to form a cavity chamber; there is a substrate 22 in the cavity chamber; a wire 18 is inserted on the substrate 22 and connected to the brush 16; one end of the pin 19 is also connected to the substrate 22, and the other end is connected to the plug socket 20. The longitudinal displacement signal is collected by sensing the magnetic field change through the brush 16, and then transmitted to the plug socket 20 through the wire 18 via the substrate 22 and the pin 19, and the longitudinal control signal is output by the plug socket 20 and transmitted to the main controller 3 through the signal transmission line.

[0062] The groove on the sliding rod 13 meshes with the clamping groove on the steering column 12. The torque exerted by the driver on the steering wheel is transmitted through the clamping groove, and then transmitted to the steering wheel angle sensor 4 and the steering wheel torque sensor 5 through the housing 26 to output a steering control signal, which is then transmitted to the main controller 3 through a signal transmission line.

[0063] The main controller 3 encodes the longitudinal and lateral control signals of the lower-limb disabled driver and transmits them to the VCU 6. The VCU 6 directly transmits the encoded lateral control signal to the steering motor 9 and the drive braking motor to realize the control of the vehicle by the lower-limb disabled driver.

[0064] Furthermore, the guiding slider 14 meshes with the limiting groove on the housing 26. When the limiting groove is closed, the guiding slider 14 cannot move, and the lower-limb disabled driver cannot generate a longitudinal control signal. When the limiting groove is opened, the guiding slider 14 can move, and the lower-limb disabled driver can generate a longitudinal control signal.

[0065] The beneficial effects of the present invention are as follows:

[0066] 1. The intelligent driving method and device for lower-limb disabled drivers provided by the present invention provide an intelligent driving method for lower-limb disabled drivers to control the vehicle longitudinally and laterally, improve the driving safety of lower-limb disabled drivers, and enhance the driving experience of the drivers.

[0067] 2. The intelligent driving method and device for lower-limb disabled drivers provided by the present invention provide a longitudinal driving intention acquisition device for lower-limb disabled drivers, which converts the steering wheel position and speed change information into digital signals through the steering wheel position sensor to obtain the driving intentions of acceleration and deceleration.

[0068] 3. The intelligent driving method and device for lower-limb disabled drivers provided by the present invention provide a human-machine integration method. By combining the longitudinal decision-making method of autonomous driving based on deep learning and the longitudinal driving intention of the driver, a human-machine driving right allocation strategy is designed to realize the effective integration of humans and machines. Description of the Drawings

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, so they should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0070] Figure 1 It is a logic schematic diagram of the intelligent driving device for lower-limb disabled drivers described in the present invention.

[0071] Figure 2 Schematic flowchart of the intelligent driving method for lower-limb disabled drivers according to the present invention;

[0072] Figure 3 Schematic structural diagram of the control system for lower-limb disabled drivers according to the present invention;

[0073] Figure 4 Schematic assembly diagram of the control system for lower-limb disabled drivers according to the present invention;

[0074] Figure 5 Schematic perspective partial sectional view of the vertical position sensor of the steering wheel according to the present invention;

[0075] Figure 6 Schematic plan sectional view of the vertical position sensor of the steering wheel according to the present invention;

[0076] Figure 7 Enlarged schematic plan sectional view of the vertical position sensor of the steering wheel according to the present invention.

[0077] In the figure:

[0078] 1. Steering wheel; 2. Vertical position sensor of the steering wheel; 3. Main controller; 4. Steering wheel angle sensor; 5. Steering wheel torque sensor; 6. VCU; 7. Road feel motor; 8. Drive and brake motor; 9. Steering motor; 10. Axle; 11. Redundant motor; 12. Steering column; 13. Slide bar; 14. Guide slider; 15. First sleeve; 16. Brush; 17. Second sleeve; 18. Wire; 19. Pin; 20. Plug socket; 21. Base plate; 22. Substrate; 23. Pad; 24. Spring; 25. Magnet; 26. Housing; 27. Third sleeve; 28. Fourth sleeve. Specific embodiments

[0079] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. In addition, it should be noted that, for the sake of description, only parts related to the present invention are shown in the drawings, rather than all structures.

[0080] Embodiment 1

[0081] This embodiment provides an intelligent driving method for lower-limb disabled drivers, including the following steps:

[0082] Step 1: The driver with lower limb disability manipulates the steering wheel 1 to rotate and move vertically according to the surrounding driving environment information. The vertical position sensor 2 of the steering wheel obtains the vertical displacement of the steering wheel. The steering wheel angle sensor 4 and the steering wheel torque sensor 5 obtain the angle and torque of the steering wheel. The driver manipulation module obtains the acceleration, braking, and steering intentions of the driver; the VCU 6 obtains the acceleration, braking, and steering signals of the driver manipulation module;

[0083] Step 2: The VCU 6 receives the environment information obtained by the perception system in the autonomous driving module and outputs the vehicle longitudinal control signal through an end-to-end decision-making model. Among them, the autonomous driving module includes a perception system, a positioning and navigation system, and a decision-making and planning system, specifically as follows:

[0084] S21: The perception system perceives and collects the original data of the surrounding environment through cameras, millimeter-wave radars, lidars, and ultrasonic radars; the original data is preprocessed and then fused through Kalman filtering to obtain the target-level data x of traffic participants, x = {l h, l v, v h, v v, a s, a v, a h}, where l h is the lateral position of the traffic participant; l v is the longitudinal position; v h is the lateral speed; v v is the longitudinal speed; a s is the steering wheel angle of the host vehicle; a v is the longitudinal acceleration of the host vehicle; a h is the lateral acceleration, and then the target machine data is passed to the decision-making and planning system;

[0085] Among them, the perception system first preprocesses the original signals of different sensors to handle the noise between different sensors. Reasonably allocate the working frequency bands of the sensors and use frequency modulation anti-interference technology to solve the channel conflict problem. For the signals from the camera, first use a Gaussian kernel to smooth the image for noise reduction to reduce Gaussian noise. Adopt an edge detection filter to highlight the edges and details in the image for enhancement processing, and use a convolutional neural network to extract image features; for the signals from the millimeter-wave radar, perform a fast Fourier transform on the echo signal to extract the distance, speed, and angle information of the target. Specifically, convert the echo signal in the time domain into a frequency domain signal and analyze the frequency components of the signal. Calculate the frequency components of the signal and use the operating frequency of the radar and the signal propagation time to calculate the distance of the target. Utilize the Doppler effect to calculate the frequency shift of the echo signal. Calculate the phase difference of the signal received by the phased array antenna and use the arrival angle algorithm to calculate the angle of the target; for the signals from the lidar, the system filters and envelopes the scanned point cloud data to extract point cloud features including curvature and normal. Remove outliers and noise through statistical filtering and Gaussian filtering and retain the point cloud structure features. Calculate the normal using local plane fitting and extract the geometric features of the point cloud through the principal curvature and the mean curvature; for the ultrasonic radar signals, the system uses a low-pass filter to remove high-frequency noise and retain the useful echo signals. Perform a Hilbert transform on the filtered signal to extract the envelope of the signal, reflecting the change in the amplitude of the echo signal over time. Measure the time delay of the ultrasonic wave from transmission to receiving the echo. Calculate the relative distance between different targets and the vehicle according to the echo time delay.

[0086] The specific method for fusing the preprocessed raw data through Kalman filtering is as follows:

[0087] Affected by environmental interferences such as rain, snow, and fog, the accuracy of the sensors themselves, and the influence of electromagnetic interference between sensors or from the outside, the measurement noises of the camera, millimeter-wave radar, lidar, and ultrasonic radar are respectively , , , , then the measurement signals from different sensors are expressed as:

[0088]

[0089]

[0090]

[0091]

[0092] In the formula, , , , Measurement signals from a camera, a millimeter-wave radar, a lidar, and an ultrasonic radar, respectively; , , , are the corresponding measurement matrices, and the obtained measurement signal is ;

[0093] Among them, ;

[0094] The state vector of the dynamic system is , and the state transition equation and the measurement equation are expressed as:

[0095]

[0096]

[0097] In the formula, is the system state transition matrix; is the system control input matrix; is the system control input; is the process noise, and the corresponding covariance matrix is ; is the measurement matrix; is the measurement noise, and the corresponding covariance matrix is ; Predict and update the system state. The prediction steps are as follows:

[0098] State prediction:

[0099]

[0100] Covariance prediction:

[0101]

[0102] Update and correct the system according to the state prediction result. The update steps are as follows:

[0103] Kalman gain calculation:

[0104]

[0105] State update:

[0106]

[0107] Covariance update:

[0108]

[0109] Through this operation, the accuracy and robustness of the vehicle's real-time perception of the surrounding environment can be improved, ensuring that the target-level data of traffic participants obtained by the perception system is input into the decision-making and planning system.

[0110] S22. The positioning information collected by the positioning and navigation system is synchronously transmitted into the decision-making and planning system; the positioning system obtains the vehicle's positioning information in real time and transmits the positioning information into the decision-making and planning system;

[0111] S23. The decision-making and planning system passes through the longitudinal decision-making model for autonomous driving based on deep learning. The longitudinal decision-making model for autonomous driving transmits the collected perception data and positioning data to the VCU, makes decisions automatically through calculation, and outputs the longitudinal control signal of the vehicle.

[0112] Step 3. The human-machine fusion module receives the control signals output by the driver control module and the autonomous driving module, obtains the final control signal through human-machine driving right allocation, and the VCU6 sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus, specifically as follows:

[0113] S31. Design a game formula for longitudinal control signals considering multiple factors, as shown below:

[0114]

[0115] In the formula, and are weight coefficients used to balance the influence of factors such as vehicle state and road environment on the game result, and . represents the control signal of a driver with lower limb disability, including the depth of the downward position of the steering wheel indicating the intention of acceleration (range 0 - 1, 0 represents the initial position of the steering wheel, 1 represents the maximum downward travel position of the steering wheel) and the depth of the upward position of the steering wheel indicating the intention of deceleration (range - 1 - 0, 0 represents the initial position of the steering wheel, - 1 represents the maximum upward travel position of the steering wheel). represents the control signal of the autonomous driving system, which also includes the acceleration control signal and the deceleration control signal . The vehicle speed is represented by , the vehicle acceleration is , the road gradient is , the road surface friction coefficient is , the distance from the vehicle in front is represented by , and the collision risk coefficient is represented by .

[0116] is a function based on the vehicle state and the control signals of the lower-limb disabled driver and the automatic driving system, expressed as:

[0117]

[0118] In the formula, is the maximum acceleration control signal; is the maximum deceleration control signal; is the given speed The maximum acceleration under; , , Are weight coefficients used to balance the influence of the depth of the steering wheel downward position, the depth of the steering wheel upward position, and the acceleration factor on the game result, and ;

[0119] Is a function based on road safety and environmental factors, expressed as:

[0120]

[0121] In the formula, Is the given speed The ideal road gradient required for the vehicle to stay in the lane under; Is the maximum gradient that the vehicle can safely pass; Is the given speed The minimum distance from the vehicle in front required to ensure safety under; , , , Are weight coefficients used to balance the influence of road gradient, road surface friction coefficient, distance from the vehicle in front, and collision risk factor on the game result, and ;

[0122] S32. Send the game result of the longitudinal control signal to VCU6;

[0123] S33. VCU6 sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus to control the drive, brake execution motor and the steering execution motor to work, so as to realize automatic driving.

[0124] Step 4. The control execution module controls the drive, brake execution motor and the steering execution motor to work, so as to realize automatic driving.

[0125] Embodiment 2

[0126] Refer to Figures 2 - 7, the present invention also provides an intelligent driving device for lower limb disabled drivers, which is used to implement the intelligent driving method for lower limb disabled drivers described in Embodiment 1, including:

[0127] A. Lower limb disabled driver control module, which is used for the lower limb disabled driver to manipulate the steering wheel 1 to rotate and move vertically according to the surrounding driving environment information. The vertical position sensor 2 of the steering wheel obtains the vertical displacement of the steering wheel, and the steering wheel angle sensor 4 and the steering wheel torque sensor 5 obtain the angle and torque of the steering wheel. The driver control module obtains the acceleration, braking and steering intentions of the driver;

[0128] The lower limb disabled driver control module includes a steering wheel 1, a vertical position sensor 2 of the steering wheel, a main controller 3, a steering wheel angle sensor 4, a steering wheel torque sensor 5, a road feeling motor 7, a drive and brake motor 8, a steering motor 9, an axle 10, a redundant motor 11, a steering column 12, a slide bar 13, a guide slider 14, a first sleeve 15, a brush 16, a second sleeve 17, a wire 18, a pin 19, a plug socket 20, a bottom plate 21, a substrate 22, a backing plate 23, a spring 24, a magnet 25, a housing 26, a third sleeve 27 and a fourth sleeve 28;

[0129] The steering wheel 1 is connected to the steering column 12 through a pin shaft. When the steering wheel 1 rotates, it drives the steering column 12 to rotate synchronously. The steering wheel angle sensor 4 is installed above the steering column 12, below the steering wheel 1, and is connected to the top of the steering column 12. The vertical position sensor 2 of the steering wheel is embedded inside the steering column 12. The road feeling motor 7 is installed at the lower end of the steering column 12 and is mechanically connected to the steering column 12 through a spline. The main controller 3, the steering wheel torque sensor 5 and the redundant motor 11 are installed at the lower end of the steering column 12. The lower limb disabled driver manipulates the steering wheel 1 to rotate and move vertically up and down. The vertical position sensor 2 of the steering wheel obtains the longitudinal driving intention and transmits it to the main controller 3. The steering wheel angle sensor 4 and the steering wheel torque sensor 5 obtain the lateral driving intention signals and transmit them to the main controller 3. The VCU 6 receives the longitudinal and lateral control signals of the lower limb disabled driver from the main controller 3, and then transmits them to the drive and brake motor 8 and the steering motor 9 respectively to control the drive force distribution and angle change of the wheels and the axle 10. At the same time, the VCU 6 also controls the road feeling motor 7 based on the real-time detected road condition information, and outputs the road feeling feedback torque command in real time to apply a suitable road feeling to the driver and return the steering wheel to the straight position.

[0130] The three-dimensional partial cross-sectional view of the vertical position sensor 2 of the steering wheel is as Figure 5As shown in the figure. The internally slidable guide slider 14 engages with the limit groove on the housing 26. When the limit groove is closed, the guide slider 14 cannot move, and the disabled lower-limb driver cannot generate a vertical control signal; when the limit groove is open, the guide slider 14 can move, and the disabled lower-limb driver can generate a vertical control signal. Additionally, on both sides of the guide slider 14, there are a first sleeve 15 and a third sleeve 27. The inner side of the first sleeve 15 is tightly connected to the outer side of the second sleeve 17 on the steering column 12, and the inner side of the third sleeve 27 is tightly connected to the outer side of the fourth sleeve 28 on the steering column 12. The steering column 12 and the backing plate 23 are connected through a slot structure, ensuring the longitudinal movement of the guide slider. At the same time, between the first sleeve 15, the third sleeve 27 and the housing 26, two springs 24 are installed to ensure that when no external force is applied by the disabled lower-limb driver, the guide slider 14 can return to the middle position on the guide rail of the housing 26. The steering wheel 1 is connected to the guide slider 14 through a slide bar 13. Through the longitudinal movement of the steering wheel 1, the guide slider 14 is driven to perform longitudinal displacement on the guide rail of the housing 26. And there is a magnet 25 on the guide slider 14. Through the longitudinal displacement of the guide slider 14, the magnet 25 is driven to move, causing a change in the surrounding magnetic field, and thus generating a longitudinal displacement signal. The brush 16 receives this longitudinal displacement signal. When the guide slider 14 drives the magnet 25 to move backward, the magnetic field intensity received by the brush 16 weakens, generating a decelerating longitudinal control signal. When the guide slider 14 drives the magnet 25 to move forward, the magnetic field intensity received by the brush 16 increases, generating an accelerating longitudinal control signal.

[0131] The enlarged plan sectional view of the vertical position sensor of the steering wheel is as Figure 7 shown. The bottom plate 21 is a convex structure that engages with the backing plate 23 with a groove structure to form a cavity chamber. There is a substrate 22 in the cavity chamber. A wire 18 is inserted into the substrate 22 and connected to the brush 16. At the same time, one end of the pin 19 is also connected to the substrate 22, and the other end is connected to the plug socket 20. The longitudinal displacement signal is collected by sensing the magnetic field change through the brush 16, and then transmitted to the pin 19 through the wire 18 via the substrate 22, and finally the longitudinal displacement signal is transmitted to the plug socket 20. The plug socket 20 outputs the longitudinal control signal, which is transmitted to the main controller 3 through the signal transmission line.

[0132] The groove on the sliding rod 13 meshes with the clamping groove on the steering column 12. The torque exerted by the driver on the steering wheel is transmitted through the clamping groove, and then transmitted to the steering wheel angle sensor 4 and the steering wheel torque sensor 5 through the housing 26 to output a steering control signal, which is then transmitted to the main controller 3 through a signal transmission line. Finally, the main controller 3 encodes the longitudinal and lateral control signals of the lower-limb disabled driver into the CAN bus communication protocol format. Each signal has a unique identifier and data format. The longitudinal signal is encoded into a frame containing information such as vehicle speed and acceleration request. The lateral signal is encoded into a frame containing information such as steering angle and steering speed. Then it is transmitted to the VCU 6, and the VCU 6 directly transmits the encoded lateral control signal to the steering motor 9 to achieve the control of the vehicle steering system by the lower-limb disabled driver. For the vertical control signal, the VCU 6 performs a rationality analysis on it and the longitudinal control signal from the autonomous driving system, and after completing the allocation of the driving right, it is transmitted to the drive and brake motor to achieve the acceleration and deceleration of the vehicle.

[0133] B. An autonomous driving module, configured to receive the environmental information obtained by the perception system in the autonomous driving module and output the vehicle longitudinal control signal through an end-to-end decision-making model;

[0134] C. A human-machine integration module, configured to receive the control signals output by the driver operation module and the autonomous driving module, obtain the final control signal through the allocation of the human-machine driving right, and the VCU sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus;

[0135] D. A control execution module, configured to control the drive and brake execution motors and the steering execution motor to work, so as to achieve autonomous driving.

[0136] In summary, the present invention can use a vertically movable steering wheel to convert the steering wheel position and speed change information into digital signals through a steering wheel position sensor, obtain the driving intentions of acceleration and deceleration, and develop a longitudinal decision-making algorithm for autonomous driving based on deep learning. Through the allocation of the driving right, an effective human-machine integration is achieved to complete the control of the intelligent vehicle.

[0137] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent driving method for drivers with lower limb disabilities, characterized in that: The following steps are involved: Step 1: The driver with lower limb disability controls the steering wheel (1) to rotate and move vertically according to the surrounding driving environment information, the steering wheel vertical position sensor obtains the vertical displacement of the steering wheel, the steering wheel angle sensor and the steering wheel torque sensor obtain the steering wheel angle and torque, and the driver control module obtains the driver's acceleration, braking and steering intentions; The VCU (6) obtains the acceleration, braking and steering signals from the driver's control module; Step 2: VCU (6) receives environmental information obtained by the perception system in the autonomous driving module and outputs a longitudinal control signal of the vehicle through an end-to-end decision model; Step 3: The human-machine fusion module receives the control signals output by the driver control module and the automatic driving module, obtains the final control signal after the human-machine driving rights are allocated, and the VCU (6) sends the control instructions to the steering motor controller and the driving and braking motor controller in the control execution module through the CAN bus; Step 4: The control execution module controls the driving, braking execution motors and steering execution motors to operate, thereby realizing automatic driving.

2. The intelligent driving method for drivers with lower limb disabilities according to claim 1, characterized in that: The autonomous driving module includes a perception system, a positioning and navigation system, and a decision-making and planning system.

3. The intelligent driving method for drivers with lower limb disabilities according to claim 1, characterized in that: The specific method of step 2 is as follows: S21, the perception system senses and collects raw data of the surrounding environment through cameras, millimeter wave radars, laser radars, and ultrasonic radars; the raw data is pre-processed and fused through Kalman filtering to obtain target-level data x of traffic participants, x={l h, l v, v h, v v, a s, a v, a h }, where l h is the lateral position of the traffic participant; v is the vertical position; v h is the lateral velocity; v v is the longitudinal velocity; a s is the steering wheel angle of the vehicle; a v is the longitudinal acceleration of the vehicle; a h It is the lateral acceleration, and then the target machine data is transmitted to the decision and planning system; S22, the positioning information collected by the positioning navigation system is synchronously transmitted to the decision-making and planning system; the positioning system obtains the positioning information of the vehicle in real time and transmits the positioning information to the decision-making and planning system; S23, the decision-making and planning system uses an autonomous driving longitudinal decision model based on deep learning. The autonomous driving longitudinal decision model transmits the collected perception data and positioning data to the VCU, automatically makes decisions after calculation, and outputs the longitudinal control signal of the vehicle.

4. The intelligent driving method for drivers with lower limb disabilities according to claim 3, characterized in that: The camera collects RGB image information; the millimeter wave radar collects original ADC data; the laser radar collects 3D point cloud information; and the ultrasonic radar collects original data.

5. The intelligent driving method for drivers with lower limb disabilities according to claim 3, characterized in that: The specific method of fusing the preprocessed raw data through Kalman filtering is as follows: The measurement noise of the camera, millimeter wave radar, lidar, and ultrasonic radar are obtained respectively , , , , then the measurement signals from different sensors are expressed as: ; ; ; ; In the formula, , , , They are the measurement signals of camera, millimeter wave radar, lidar and ultrasonic radar respectively; , , , is the corresponding measurement matrix, and the measurement signal is obtained as ; in, ; The state vector of the dynamic system is , the state transition equation and measurement equation are expressed as: ; ; In the formula, is the system state transfer matrix; Input matrix for system control; It is the system control input; is the process noise, and the corresponding covariance matrix is ; is the measurement matrix; is the measurement noise, and the corresponding covariance matrix is ; Predict and update the system status. The prediction steps are as follows: S1. Status prediction: L; In the formula, The state of the past k-1 moments, and the state prediction at moment k; is the state estimate at time k-1; S2, covariance prediction: ; In the formula, is the state at the past k-1 moments, and the covariance matrix of the state prediction at moment k is used to evaluate the accuracy of the estimated value. , represents the state estimation covariance matrix at k-1 moments; S3. Update and modify the system according to the status prediction results, as follows: Kalman gain calculation: ; In the formula, is the Kalman gain; Status Update: ; Covariance update: 。 6. The intelligent driving method for drivers with lower limb disabilities according to claim 1, characterized in that: The specific method of step three is as follows: S31. Design a game formula for longitudinal control signals that takes multiple factors into consideration, as shown below: ; In the formula, and is the weight coefficient, which is used to balance the impact of factors such as vehicle status and road environment on the game results, and ; Control signals for drivers with lower limb disabilities, including steering wheel down position depth to indicate acceleration intent and the depth of the steering wheel upward position indicating the intention to slow down ; Control signals for the autonomous driving system, including acceleration control signals and deceleration control signal ; is the vehicle speed; Vehicle acceleration; Road slope; Road surface friction coefficient; is the distance between the vehicle and the vehicle in front; is the collision risk factor; is a function based on the vehicle state and the control signal of the lower limb disabled driver and the automatic driving system, expressed as: ; In the formula, is the maximum acceleration control signal; is the maximum deceleration control signal; For a given speed Maximum acceleration under , , is the weight coefficient used to balance the influence of the steering wheel downward position depth, the steering wheel upward position depth, and the acceleration factor on the game result, and ; is a function based on road safety and environmental factors, expressed as: ; In the formula, For a given speed The ideal road gradient required for the vehicle to stay in its lane; The maximum slope that a vehicle can safely pass; For a given speed The minimum distance required by the next vehicle to ensure safety from the vehicle ahead; , , , is the weight coefficient, which is used to balance the impact of road slope, road friction coefficient, distance to the front vehicle and collision risk factors on the game results, and ; S32, sending the longitudinal control signal game result to the VCU; S33, the VCU sends control instructions to the steering motor controller and the drive and brake motor controller in the control execution module through the CAN bus to control the drive, brake execution motor and steering execution motor to work, thereby realizing automatic driving.

7. An intelligent driving device for drivers with lower limb disabilities, used to implement an intelligent driving method for drivers with lower limb disabilities as described in any one of claims 1 to 6, characterized in that: include: The lower limb disabled driver operation module is used for the lower limb disabled driver to operate the steering wheel (1) to rotate and move vertically according to the surrounding driving environment information, the steering wheel vertical position sensor (2) obtains the vertical displacement of the steering wheel, the steering wheel angle sensor (4) and the steering wheel torque sensor (5) obtain the steering wheel angle and torque, and the driver operation module obtains the driver's acceleration, braking and steering intentions; The autonomous driving module is used to receive environmental information obtained by the perception system in the autonomous driving module and output the longitudinal control signal of the whole vehicle through an end-to-end decision model; The human-machine fusion module is used to receive the control signals output by the driver control module and the automatic driving module, obtain the final control signal after the human-machine driving rights are allocated, and the VCU sends control instructions to the steering motor controller and the driving and braking motor controller in the control execution module through the CAN bus; The control execution module is used to control the operation of the drive, brake execution motor and steering execution motor to achieve automatic driving.

8. The intelligent driving device for drivers with lower limb disabilities according to claim 7, characterized in that: The control module for a driver with lower limb disabilities comprises a steering wheel (1), a steering wheel vertical position sensor (2), a main controller (3), a steering wheel angle sensor (4), a steering wheel torque sensor (5), a road sensing motor (7), a driving and braking motor (8), a steering motor (9), an axle (10), a redundant motor (11), a steering column (12), a sliding rod (13), a guide slider (14), a first sleeve (15), a brush (16), a second sleeve (17), a wire (18), a pin (19), a plug socket (20), a bottom plate (21), a base plate (22), a pad (23), a spring (24), a magnetic steel (25), a housing (26), a third sleeve (27) and a fourth sleeve (28); The steering wheel (1) is connected to the steering column (12) via a pin shaft; the steering wheel angle sensor (4) is mounted above the steering column (12), located below the steering wheel (1), and connected to the top of the steering column (12); the steering wheel vertical position sensor (2) is embedded in the steering column (12); the road sensing motor (7) is mounted at the lower end of the steering column (12) and mechanically connected to the steering column (12) via a spline; the main controller (3), the steering wheel torque sensor (5) and the redundant motor (11) are mounted at the lower end of the steering column (12); the redundant motor (11) is connected to the steering column (12), and when the main motor fails, torque is directly applied to the steering column (12); drivers with lower limb disabilities The steering wheel (1) is operated to rotate and move vertically up and down, and the steering wheel vertical position sensor (2) obtains the longitudinal driving intention and transmits it to the main controller (3); the steering wheel angle sensor (4) and the steering wheel torque sensor (5) obtain the lateral driving intention signal and transmit it to the main controller (3); the VCU (6) receives the longitudinal and lateral control signals of the driver with lower limb disabilities from the main controller (3), and transmits them to the driving and braking motor (8) and the steering motor (9) respectively, to control the driving and braking force distribution and the angle change of the wheels and the axle (10); at the same time, the VCU (6) detects and collects the road condition information in real time, controls the road sense motor (7), outputs the road sense feedback torque command in real time, applies the road sense to the driver and makes the steering wheel (1) return to the center position; The guide slider (14) is meshed with the housing (26); a first sleeve (15) and a third sleeve (27) are provided on both sides of the guide slider (14); the inner side of the first sleeve (15) contacts the outer side of the second sleeve (17) on the steering column (12); the inner side of the third sleeve (27) contacts the outer side of the fourth sleeve (28) on the steering column (12); the steering column (12) and the backing plate (23) are connected via a slot; two springs (24) are provided between the first sleeve (15) and the third sleeve (27) and the housing (26); the steering wheel (1) and the guide slider (14) are connected via a slide rod (13); the longitudinal movement of the steering wheel (1) drives the guide slider (14) to move longitudinally on the guide rail of the housing (26); a magnetic steel ( 25), the longitudinal displacement of the guide slider (14) drives the magnetic steel (25) to move, causing the surrounding magnetic field to change, thereby generating a longitudinal displacement signal; the longitudinal displacement signal is received by the brush (16); the bottom plate (21) is a protruding structure, meshing with the backing plate (23) of the groove structure to form a cavity chamber; the cavity chamber contains a substrate (22); a wire (18) is inserted into the substrate (22) and connected to the brush (16); one end of the pin (19) is also connected to the substrate (22), and the other end is connected to the plug socket (20), and the brush (16) senses the change in the magnetic field to collect the longitudinal displacement signal, and then the wire (18) is transmitted to the pin (19) through the substrate (22), and finally the vertical displacement signal is transmitted to the plug socket (20), and the plug socket (20) outputs the longitudinal control signal, which is transmitted to the main controller (3) through the signal transmission line; The groove on the slide bar (13) meshes with the slot on the steering column (12), and the torque applied by the driver to the steering wheel is transmitted through the slot, and then transmitted to the steering wheel angle sensor (4) and the steering wheel torque sensor (5) through the housing (26), and a steering control signal is output, which is then transmitted to the main controller (3) through the signal transmission line; The main controller (3) encodes the longitudinal and lateral control signals of the driver with lower limb disabilities and transmits them to the VCU (6), and the VCU (6) directly transmits the encoded lateral control signals to the steering motor (9) and the driving and braking motors, thereby enabling the driver with lower limb disabilities to control the vehicle.

9. The intelligent driving device for drivers with lower limb disabilities according to claim 8, characterized in that: The guide slider (14) is meshed with a limit slot on the housing (26); when the limit slot is closed, the guide slider (14) cannot move, and the driver with a lower limb disability cannot generate a longitudinal control signal; when the limit slot is open, the guide slider (14) can move, and the driver with a lower limb disability can generate a longitudinal control signal.

Images