Time delay compensation trajectory prediction control method for brain-controlled unmanned vehicle

Abstract

The invention relates to the technical field of brain-computer interface and unmanned vehicle control, and provides a time delay compensation trajectory prediction control method for a brain-controlled unmanned vehicle. The method aims at solving the problems that discrete instructions output by a non-intrusive brain-computer interface have fixed time delay and jitter time delay in an acquisition window, decoding, communication and vehicle execution links, so that response lagging and jamming of an unmanned vehicle are caused. According to the method, a time delay sensing trajectory prediction control framework is constructed; firstly, electroencephalogram signals are collected and decoded to obtain discrete high-level control instructions; secondly, performing estimation or online updating on end-to-end time delay, and establishing an instruction-state time alignment sequence; inputting the current and historical instruction sequences, the vehicle state sequence and the environment information into a trajectory prediction learning model, and predicting a target trajectory or a continuous control quantity sequence at an effective moment after time delay compensation; and finally, carrying out constraint correction on a prediction result in combination with vehicle dynamics and safety constraint, and issuing a control quantity to the vehicle in a rolling time domain manner to realize compensation for time delay and jitter, thereby reducing the sense of lagging and improving the continuity, safety and comfort of a track.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of brain-computer interface and unmanned vehicle motion control, and particularly relates to a time delay compensation trajectory prediction control method for brain-controlled unmanned vehicles. BACKGROUND

[0002] With the development of non-invasive brain-computer interface (BCI) technology, using physiological signals such as electroencephalogram (EEG) to control external devices has become an important research direction in the fields of intelligent auxiliary driving, rehabilitation assistance and intelligent robots. As a typical application of brain-computer interface on a mobile platform, a brain-controlled unmanned vehicle is usually composed of brain signal acquisition, intention decoding, instruction transmission and vehicle execution control.

[0003] In existing non-invasive brain-controlled unmanned vehicle systems, due to the signal-to-noise ratio and the amount of available feature information of the brain-computer interface, the decoding module often outputs a small number of discrete high-level instructions, such as "forward, backward, left turn, right turn, stop" and the like. Due to the low instruction update frequency and the natural mismatch between discrete instructions and continuous vehicle motion, existing systems often use fixed mapping, simple interpolation, low-pass filtering or finite state machine to convert discrete instructions into speed / turning angle control quantities. The above methods can basically work in simple environments with small time delays, but in actual brain-controlled unmanned vehicles, the following problems often exist: 1. End-to-end time delay causes lag: non-invasive brain-computer interfaces usually use a sliding time window for acquisition and decoding (such as hundreds of milliseconds to several seconds), and superimpose decoding calculation, communication and vehicle control cycle factors, which will produce obvious end-to-end time delay; when the user issues a turning or stopping intention, the vehicle response lags, and the user will frequently issue instructions to correct the error, causing a "lag-correct-lag" lagging feeling. 2. Jitter delay causes unstable trajectory: communication links, computing load and task scheduling factors will bring jitter delay that changes over time, making the effective time of the same type of instruction inconsistent at different cycles, resulting in sudden changes in vehicle control quantity, trajectory zigzagging and oscillation. 3. Traditional smoothing is not enough to solve "smoothing + time delay" at the same time: filtering or interpolation of control quantity can improve curve continuity, but cannot eliminate the misalignment of instruction effective time; in the presence of significant time delay and jitter delay, the vehicle will still show a "slow half-beat" response and "poor hand-following" turning, and even pose a safety risk in complex scenarios.

[0004] Therefore, there is an urgent need for a control method for brain-controlled unmanned vehicles that introduces time delay awareness and prediction compensation mechanisms between discrete brain-computer instructions and vehicle execution control, models the timing relationship between instructions and states, predicts the target trajectory or control quantity for a certain period of time in the future, and compensates for time delay and jitter in a rolling control framework, thereby significantly reducing the lagging feeling and improving trajectory smoothness and safety. SUMMARY

[0005] In order to solve the problems of lag, response lag and unstable trajectory caused by end-to-end delay and jitter delay of the existing brain-controlled unmanned vehicle, the application provides a delay compensation trajectory prediction control method for the brain-controlled unmanned vehicle, a control link of "delay estimation-time alignment-trajectory prediction-constraint correction-rolling issuance" is constructed between the brain-computer decoding output and the vehicle bottom control, the prediction compensation of the delay and the jitter delay is realized, and the continuity, hand-following and safety of the vehicle control are improved.

[0006] The delay compensation trajectory prediction control method for the brain-controlled unmanned vehicle has the following specific steps:

[0007] Step 1, build a brain-controlled unmanned vehicle delay compensation and trajectory prediction control system and install it on the unmanned vehicle S to be tested; the system comprises a brain-computer signal acquisition and decoding unit, a vehicle state acquisition unit, a delay estimation and updating unit, an instruction-state time alignment unit, a trajectory prediction learning model unit, a trajectory constraint checking and correction unit, a rolling control issuance unit and an optional actuator interface.

[0008] Among them:

[0009] The brain-computer signal acquisition and decoding unit acquires EEG signals and decodes discrete high-level instructions, the instructions at least include one of forward, backward, left turn, right turn and stop, and a timestamp and a confidence are attached to each instruction;

[0010] The vehicle state acquisition unit acquires vehicle speed, heading angle, angular velocity, position and other state information and attaches a timestamp;

[0011] The delay estimation and updating unit estimates the end-to-end delay Δt of the "brain-computer instruction generation-instruction executed by the vehicle" link, and updates the jitter delay online;

[0012] The instruction-state time alignment unit time-aligns the discrete instruction sequence and the vehicle state sequence according to Δt, and constructs a prediction model input;

[0013] The trajectory prediction learning model unit predicts the target trajectory sequence or continuous control quantity sequence after delay compensation in the future time domain based on the aligned instructions, states and environmental information;

[0014] The trajectory constraint checking and correction unit applies vehicle dynamics constraints and safety constraints to the prediction results, and outputs executable control quantities;

[0015] The rolling control issuance unit issues control quantities to the unmanned vehicle S in a rolling time domain manner, and realizes the continuous mapping of low-frequency brain-computer instructions to high-frequency vehicle control through a buffering mechanism.

[0016] Step two, the brain-computer signal acquisition and decoding unit acquires, preprocesses and decodes the user's electroencephalogram signal, and outputs a discrete high-level control instruction sequence; the preprocessing includes filtering, artifact rejection and feature extraction; the decoding output is a discrete instruction c k , and a timestamp t k is attached k .

[0017] Step three, the vehicle state acquisition unit acquires the state information sequence of the unmanned vehicle S

[0018] The state information can be represented as , where v is the speed, ψ is the heading angle, and P is the position.

[0019] Step four, the time delay estimation and update unit estimates the end-to-end time delay Δt and updates the jitter time delay online; the end-to-end time delay can be decomposed as:

[0020] Δt = Δt win + Δt dwc + Δt com + Δt exec

[0021] where Δt win is the acquisition window delay, Δt dec is the decoding calculation delay, Δt com is the communication delay, and Δt exec is the execution and control cycle delay; the jitter time delay can be obtained by online updating Δt(t) through sliding average or filtering.

[0022] Step five, the instruction-state time alignment unit aligns the instruction sequence and the state sequence according to Δt, and constructs the prediction model input;

[0023] The alignment target is to make the model predict the trajectory or control quantity that "takes effect at t+Δt" at time t within the vehicle control cycle T c , so as to realize phase advance compensation. The model input at least includes:

[0024] The current and historical instruction sequences (c k-m ,..., c k ) and their confidence levels;

[0025] The current and historical vehicle state sequences x(t-n),...,x(t);

[0026] Optional environmental information (relative position of obstacles, road boundaries, local occupancy grid, etc.).

[0027] Step six, the trajectory prediction learning model unit outputs the target trajectory sequence or continuous control quantity sequence after time delay compensation

[0028] After the instruction-state time alignment is completed in step five, the trajectory prediction learning model unit is triggered to perform inference prediction at the vehicle control cycle time t. The input of the trajectory prediction learning model unit at least includes: (1) the aligned current instruction c k , the historical instruction sequence (c k-m ,…,c k ) and their confidence (p k-m ,…,p k ); (2) the current state x(t) and the historical state sequence (x(t-nT c ),…,z(t)) of the vehicle, wherein the state at least includes one or more of speed v, heading angle ψ, yaw rate and position P; (3) optional environmental information e(t), including obstacle relative position / speed, road boundary / lane line information, local occupancy grid or passable area representation, etc.; (4) the end-to-end delay parameter Δt(t) and its fluctuation statistics (such as variance or maximum / minimum range) obtained online, used to guide the prediction compensation strategy.

[0029] 6-1 Prediction time domain and output form

[0030] The prediction time domain is H steps, and the step length is the vehicle control cycle T c . The model output is at least one of the following two types:

[0031] (a) Target trajectory sequence output:

[0032] The output is a trajectory point sequence in the future time domain:

[0033]

[0034] Each trajectory point at least contains position coordinates and / or heading angle and can further contain curvature expected speed , etc. This form is suitable for vehicles using trajectory tracking controllers (such as Pure Pursuit, MPC, etc.) for tracking.

[0035] (b) Continuous control quantity sequence output:

[0036] The output is a control quantity sequence in the future time domain:

[0037]

[0038] The control quantity includes expected speed and steering angle angular velocity This form is suitable for the vehicle bottom layer to directly execute speed and steering instructions.

[0039] 6-2 Time delay compensation mechanism

[0040] In order to compensate for the "instruction effective time dislocation" caused by the end-to-end time delay, the trajectory prediction learning model unit meets the following requirements when outputting the target trajectory / control quantity:

[0041] When predicting at time t, the output corresponds to the trajectory or control quantity that "begins to take effect near t+Δt(t)", that is, the time reference of the prediction result is phase-advanced relative to the current time;

[0042] When Δt(t) has jitter, the model can further adopt a conservative compensation strategy, such as using the upper quantile of Δt(t) as the compensation delay, to reduce the execution jump caused by jitter.

[0043] 6-3 Model structure and training method

[0044] The trajectory prediction learning model is a time series learning model, and the specific implementation adopts a pre-trained model for migration:

[0045] Use a model pre-trained on multi-source trajectory data, driving / teleoperation data, and / or brain control data as a feature encoder, and use transfer learning or fine-tuning to output trajectories / controls for this task. This approach is suitable for cross-scene generalization or rapid adaptation to new road structures. To further improve trajectory continuity, the model output can have built-in smoothing priors or regularization terms, such as constraints on the second derivative of the trajectory or the rate of change of the control quantity, so that the output meets the acceleration and jerk smoothing requirements, reducing the frequent fluctuations of "stutter-correction" from the source.

[0046] Step seven, the trajectory constraint checking and correction unit applies safety and dynamics constraints to the prediction results to obtain the final control quantity.

[0047] Step seven is used to project the predicted trajectory or control quantity obtained in step six to the safe and feasible domain that the vehicle can execute, avoiding dangerous behavior caused by model prediction errors, brain control misinstructions, or time delay jitter. This unit performs the following processing on the output:

[0048] 7-1 Constraint type

[0049] The constraints at least include:

[0050] Steering and steering rate constraints: |δ|≤δ max , To prevent vehicle swaying caused by sudden direction changes.

[0051] Speed, acceleration, and jerk constraints: 0≤v≤v max ,|a|≤amax ,|j|≤j max This ensures smooth longitudinal movement and suppresses sudden jolts during braking / acceleration.

[0052] Road boundary / lane constraints: Ensure that the trajectory remains within the passable area, without crossing the boundary or going out of the road.

[0053] 7-2 Correction Method

[0054] When a collision risk occurs or the road is not feasible, the local path planning module (MPC-based local planning) is invoked to regenerate a short-term feasible trajectory, maintaining similarity in shape / direction to the original predicted trajectory, thus balancing safety and user intent.

[0055] 7-3 Unified Output Format

[0056] If the output of step six is ​​the target trajectory sequence, then step seven further converts the target trajectory into the final control variable u through the trajectory tracking controller (or integrates trajectory tracking calculation within this unit). * .

[0057] If the output of step six is ​​a control quantity sequence, then step seven directly applies constraints and corrections to the control quantity sequence, outputting the final control quantity u. * .

[0058] Step 8: The rolling control issuing unit issues control quantities to the unmanned vehicle S in a rolling time domain manner, and achieves continuous control through a buffering mechanism.

[0059] Step eight addresses the continuous control challenges posed by low brain-computer interface command frequency, high vehicle control frequency, and fluctuating end-to-end latency. This unit employs a rolling time-domain execution mechanism, specifically including:

[0060] 8-1 Multi-rate Execution and Control Buffer

[0061] Due to the brain-computer decoding frequency f BCI Typically below the vehicle control frequency f veh In each control cycle T c At time t, steps six and seven are triggered once to generate the future H-step control quantity sequence u. * System settings control buffer will u * Write to the buffer and output to the vehicle actuators in chronological order.

[0062] Between two consecutive brain-computer interface command updates, vehicle control still presses T. c The control quantities in the high-frequency execution buffer are used to achieve the mapping of "low-frequency brain control - high-frequency continuous execution", thereby reducing control breaks and stuttering caused by discrete instructions.

[0063] 8-2 Rolling update rule

[0064] In each control period t:

[0065] If a new brain-machine command is received or the time delay estimation changes significantly, update the alignment sequence immediately and re-predict, refreshing the subsequent control quantities in the buffer;

[0066] If no new command is received, maintain the rolling update of the prediction model: use the latest vehicle state x(t) and the current buffer execution results to perform a correction prediction, ensuring that the trajectory adapts to environmental changes;

[0067] Output the first control quantity in the buffer as the current period execution command, and roll the buffer forward by one period.

[0068] 8-3 Time delay surge and confidence deficiency safety strategy

[0069] When the brain-machine decoding confidence p k is below the threshold or the estimated time delay Δt(t) surges or the jitter amplitude exceeds the preset threshold within a short time; the system enters the safety strategy mode, which includes: (a) speed limit: reduce the upper limit of the desired speed to a safe speed; (b) maintain: maintain the current low-speed straight driving or maintain the current turning angle; (c) stop: output brake / zero speed control quantity and maintain.

[0070] Through the above mechanism, even if the brain control command is low frequency and there is a fixed time delay and jitter time delay in the link, the vehicle can still continuously output stable control quantities, significantly reducing the lag and improving safety and control experience.

[0071] The present application has the following advantages:

[0072] 1. Relieve lag caused by time delay and jitter time delay: through time delay estimation, time alignment and prediction compensation, the vehicle execution is aligned to the user intention effective time, significantly reducing the lag of "slow half a beat";

[0073] 2. Improve trajectory continuity and comfort: prediction output combined with acceleration / jerk constraints makes the control quantity change smooth, reducing the zigzag and swing;

[0074] 3. Reduce user brain control burden and improve safety: high-frequency continuous control can be generated under low-frequency discrete brain control input, while the risk of misinstruction and environmental mutation is reduced through constraint correction and safety strategy. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 is a flowchart of a time delay compensation trajectory prediction control method for brain-controlled unmanned vehicles of the present application;

[0076] Figure 2This is a schematic diagram of the time delay compensation and trajectory prediction control system for the brain-controlled unmanned vehicle of the present invention; Detailed Implementation

[0077] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0078] The aforementioned time-delay compensation trajectory prediction control method for a brain-controlled unmanned vehicle, such as Figure 1 As shown, the specific steps are as follows.

[0079] Step 1: Build a time delay compensation trajectory prediction and control system for brain-controlled unmanned vehicles and install it on the unmanned vehicle S under test.

[0080] like Figure 2 As shown, the system of the present invention includes at least: a brain-computer signal acquisition and decoding unit, a vehicle state acquisition unit, an end-to-end delay estimation and update unit, an instruction-state time alignment unit, a trajectory prediction learning model unit, a trajectory constraint checking and correction unit, a rolling control issuance and buffer execution unit, and an optional actuator interface and environment perception unit.

[0081] in:

[0082] 1) Brain-computer signal acquisition and decoding unit (module 1): Acquires non-invasive brain-computer signals (e.g., EEG), performs preprocessing, feature extraction and decoding, and outputs discrete high-level control commands; the discrete commands include at least one of "forward, backward, left turn, right turn, stop", and each command is appended with a timestamp and confidence level.

[0083] 2) Vehicle Status Acquisition Unit (Module 2): Acquires the status information of the unmanned vehicle S during the vehicle control cycle and adds a timestamp. The status information includes at least one or more of the following: speed, heading angle, angular velocity, and position.

[0084] 3) End-to-end delay estimation and update unit (module 3): Estimates and updates the end-to-end delay parameter Δt(t) from “brain-computer instruction generation” to “chassis execution effect” online, including fixed delay and jitter delay.

[0085] 4) Command-State Time Alignment Unit (Module 4): Based on Δt(t), the discrete command sequence and the vehicle state sequence are time aligned to construct the input sequence of the trajectory prediction model.

[0086] 5) Trajectory prediction learning model unit (Module 5): Based on the aligned instructions, vehicle state and optional environmental information, predict the target trajectory sequence and / or continuous control quantity sequence of the future step, and compensate the prediction result in time by Δt(t).

[0087] 6) Trajectory constraint checking and correction unit (Module 6): Apply dynamics and safety constraints to the predicted trajectory / control quantity, and perform projection optimization or local replanning if necessary, output executable control quantity u * .

[0088] 7) Rolling control issuance and buffer execution unit (Module 7): Issue control quantity to the chassis in a rolling time domain manner, and set up a control buffer to realize continuous execution of "low-frequency brain-machine - high-frequency vehicle control"; trigger safety strategy when confidence is insufficient or time delay increases sharply.

[0089] 8) Optional environmental perception unit (Module 8): Obtain environmental information such as obstacles, road boundaries or occupancy grids, and provide input for prediction and constraint checking.

[0090] 9) Optional actuator interface (Module 9): Convert control quantity into steering, driving, braking and other chassis execution signals, and collect feedback for closed-loop update.

[0091] Step two, collect brain-machine signals and decode output discrete instruction sequence.

[0092] Module 1 collects and pre-processes brain-machine signals in each decoding period, which at least includes band-pass filtering, power frequency suppression, artifact suppression and feature extraction; then the features are input into the decoder to output discrete high-level control instructions c k , and record the instruction generation timestamp t k and confidence p k .

[0093] In an embodiment, if p k is lower than the preset threshold, module 1 outputs "keep / no new instruction" or outputs "stop / decelerate" and other conservative instructions to reduce the disturbance of misinstruction to vehicle motion.

[0094] Step three, obtain vehicle state information sequence and add timestamp.

[0095] Module 2 samples vehicle state x(t) with vehicle control period T c , and adds timestamp. The state vector can be represented as:

[0096]

[0097] Where v(t) is the speed, ψ(t) is the heading angle, is the yaw rate, P(t) is the position coordinate; in practical applications, one part or other state variables can be selected according to the chassis and positioning conditions.

[0098] Step four, end-to-end delay estimation and online update.

[0099] Module 3 estimates the end-to-end delay, which can be decomposed as:

[0100] Δt(t) = Δt win + Δt dec + Δt com + Δt exec

[0101] where Δt win is the inherent delay caused by the collection window and the sliding window, Δt dec is the decoding calculation delay, Δt com is the communication transmission delay, and Δt exec is the control period and actuator response delay.

[0102] In an embodiment, module 3 can obtain Δt meas (t) and update it online in one or more of the following ways:

[0103] 1) Timestamp link measurement method: timestamps are recorded at the decoding output end and the chassis effective end, respectively, and the measured delay Δt meas (t) is directly calculated.

[0104] 2) Response correlation identification method: the instruction change (such as left / right turn instruction) is correlated with the vehicle state change (such as the change of yaw rate or the change of heading ψ, and the delay is estimated by the peak position.

[0105] 3) Filter update method (jitter delay modeling): Δt meas (t) is filtered to obtain a smoothed Δt(t), for example, exponential moving average:

[0106] Δt(t) = αΔt(t-1) + (1-α)Δt meas (t)

[0107] where α ∈ (0, 1) is the filter coefficient; Kalman filtering and other methods can also be used to estimate the jitter delay.

[0108] Step five, instruction-state time alignment and construction of trajectory prediction model input.

[0109] Module 4 aligns the instruction sequence and the state sequence according to Δt(t), so that the instruction information used at the current time (t) can correspond to the intention that will affect the vehicle behavior at t+Δt(t) time.

[0110] In an embodiment, module 4 performs the following processing:

[0111] 1) Instruction alignment: map the time-stamped discrete instructions c k to the control timeline, resulting in an aligned instruction sequence and retaining instruction confidences

[0112] 2) State window construction: extract the vehicle state sequence x(t-nT c ),…,x(t) in the last n control periods.

[0113] 3) Input vector concatenation: form the trajectory prediction learning model input:

[0114]

[0115] where e(t) is optional environment information (obstacles, road boundaries, occupancy grid, etc.); when there is no environment sensor, e(t) can be empty or replaced by simplified features.

[0116] Step six, the trajectory prediction learning model outputs the target trajectory sequence or continuous control quantity sequence after time delay compensation.

[0117] Module 5 predicts the future H-step target trajectory or control quantity based on the input z(t) at each control period t, and makes the output compensate for Δt(t) on the time reference. The prediction time domain step length is T c , and the prediction output is at least one of the following two types:

[0118] (1) Target trajectory sequence:

[0119]

[0120] Each trajectory point contains at least position and / or heading angle and can optionally contain curvature expected speed , etc.

[0121] (2) Continuous control quantity sequence:

[0122]

[0123] where includes expected speed and steering angle angular velocity

[0124] To mitigate the impact of jitter latency on control stability, module 5 can further adopt a conservative compensation strategy: when Δt(t) fluctuates greatly, set the compensation latency to the upper quantile of Δt(t), and suppress the control rate of change at the output end.

[0125] The trajectory prediction learning model is a time series learning model, which can be a pre-trained model migrated for this task. To improve trajectory smoothness, the model training objective can include a trajectory error term and a smoothing regularization term, such as a penalty term on the second-order difference of the trajectory or on the control rate of change, to suppress output jitter and mutations.

[0126] Step seven, impose safety and dynamics constraints on the prediction results and output executable control quantities.

[0127] Module 6 performs constraint checking and correction on the output of step six or Constraints include:

[0128] 1) Steering and steering rate constraints: |δ|≤δ max ,

[0129] 2) Speed, acceleration and jerk constraints: 0≤v≤v max , |a|≤a max , |j|≤j max ;

[0130] 3) Road boundary / lane constraints: trajectory points are within the drivable area.

[0131] When the prediction results do not meet the constraints, module 6 can use the following local re-planning method to correct them:

[0132] When there is a risk of collision or the road is not drivable, call the local re-planning module to generate a short-term feasible trajectory, and maintain consistency with the original prediction trajectory in direction and intent, and then output u * from the trajectory tracking controller.

[0133] If step six outputs a trajectory sequence Module 6 can further call the trajectory tracking controller (such as MPC) to convert the trajectory into executable control quantities for the unmanned vehicle; if step six outputs a control quantity sequence Module 6 directly corrects the control quantity sequence and outputs u * .

[0134] Step eight, roll control issuance and buffer execution, realizing multi-rate continuous control and safety degradation.

[0135] Module 7 performs control with a receding horizon mechanism: at each control period t, steps five to seven are repeated to generate a sequence of future H-step control quantities u based on the latest state and time delay estimate, which is written into a control buffer * The chassis updates its state at period T c The current step control quantity is read from the buffer and executed, after which the buffer is rolled forward by one period and refreshed with the subsequent control quantity.

[0136] When the brain-machine command update frequency is lower than the vehicle control frequency, module 7 uses the prediction model to generate intermediate control quantities between two adjacent brain-machine commands, ensuring the high-frequency continuity of vehicle control, thereby reducing the "pause-jump" phenomenon caused by discrete commands.

[0137] In addition, when any of the following conditions occurs: (a) the brain-machine command confidence is below a threshold; (b) Δt(t) suddenly increases or the jitter amplitude exceeds a threshold; (c) the obstacle distance is too close or the collision risk is high, module 7 triggers a safety strategy, which includes but is not limited to speed limiting, maintaining a low speed straight, maintaining the current turning angle, outputting a parking control quantity (only safe obstacle avoidance and emergency braking are retained).

[0138] Embodiment:

[0139] As shown in Figure 2 , this embodiment illustrates the brain-controlled unmanned vehicle S in an indoor / garden scenario. The brain-machine device uses a non-invasive EEG headset, and the decoder outputs discrete commands such as "forward / left turn / right turn / stop"; the unmanned vehicle S is equipped with a vehicle-mounted computing unit and a chassis controller, and can optionally be equipped with a camera or laser radar to obtain obstacle information.

[0140] During operation, the system first estimates the end-to-end time delay Δt(t) from brain-machine command output to vehicle execution and updates the jitter delay online through filtering; then the system time-aligns the command-state and predicts the future H-step compensation trajectory / control quantity; finally, through constraint correction and rolling delivery mechanism, the vehicle still maintains continuous, smooth and safe motion in the presence of fixed and jitter delays, thereby significantly reducing the lag and improving the control experience.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent substitutions for some technical features; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.​

Claims

1. A time delay compensation trajectory prediction control method for brain-controlled unmanned vehicles, characterized in that, The specific steps are as follows: Step one, build a brain-controlled unmanned vehicle time delay compensation trajectory prediction control system and install it on the unmanned vehicle S to be tested. The system includes a brain-computer signal acquisition and decoding unit, a vehicle state acquisition unit, an end-to-end time delay estimation and updating unit, an instruction-state time alignment unit, a trajectory prediction learning model unit, a trajectory constraint checking and correction unit, a rolling control issuing and buffer execution unit, and optional environmental perception unit and / or actuator interface; Step two, the brain-computer signal acquisition and decoding unit acquires user non-invasive brain-computer signals and performs preprocessing and decoding, outputs discrete high-level control instruction sequence, and adds instruction timestamp and instruction confidence to each discrete high-level control instruction; Step three, the vehicle state acquisition unit acquires the vehicle state information sequence of the unmanned vehicle S, and adds the state timestamp to the vehicle state information; Step four, the end-to-end time delay estimation and updating unit estimates and updates the end-to-end time delay parameter Δt(t) from the generation of discrete high-level control instructions to the execution of unmanned vehicle S execution mechanism, wherein Δt(t) includes fixed time delay and / or jitter time delay; Step five, the instruction-state time alignment unit aligns the discrete high-level control instruction sequence and the vehicle state information sequence in time according to the end-to-end time delay parameter Δt(t), and constructs the input sequence of the trajectory prediction learning model unit; Step six, the trajectory prediction learning model unit predicts the target trajectory sequence and / or continuous control quantity sequence of the future H steps based on the input sequence, and compensates the prediction output for the end-to-end time delay parameter Δt(t) on the time reference; Step seven, the trajectory constraint checking and correction unit imposes vehicle dynamics constraints and safety constraints on the target trajectory sequence and / or continuous control quantity sequence, and modifies the prediction results if necessary to obtain the final control quantity u executable by the chassis * ; Step eight, scroll control is issued and the buffer execution unit is issued to the unmanned vehicle S in a rolling time domain manner Final control quantity u * And through the control of the buffer, continuous execution is realized to compensate for the response lag and lag caused by end-to-end delay and jitter delay.

2. The method of claim 1, wherein, The data interaction between the units in step one includes: The brain-computer signal acquisition and decoding unit outputs discrete high-level control instructions, instruction timestamps and instruction confidence to the end-to-end time delay estimation and updating unit and the instruction-state time alignment unit; The vehicle state acquisition unit outputs vehicle state information and state timestamp to the end-to-end time delay estimation and updating unit and the instruction-state time alignment unit; The end-to-end time delay estimation and updating unit outputs the end-to-end time delay parameter Δt(t) to the instruction-state time alignment unit and the trajectory prediction learning model unit; The instruction-state time alignment unit outputs the aligned instruction-state input sequence to the trajectory prediction learning model unit; The trajectory prediction learning model unit outputs the target trajectory sequence and / or continuous control quantity sequence of the future H steps to the trajectory constraint checking and correction unit; The trajectory constraint checking and correction unit outputs the final control quantity u * To the roll control issuing and buffer executing unit; The roll control issuing and buffer executing unit issues the final control quantity u * to the steering, driving and braking executing mechanism of the unmanned vehicle S, and collects the executing feedback for the state updating and time delay updating of the next control cycle.

3. The method of claim 1, wherein, The discrete high-level control instruction in step two includes at least one of forward, backward, left turn, right turn and stop, and the instruction confidence is used to represent the confidence degree of the decoding result.

4. The method of claim 1, wherein, The end-to-end time delay parameter Δt(t) in step four is obtained and updated by one or more of the following ways: (1) Link measurement based on timestamp, using instruction generation timestamp and execution effective timestamp to calculate end-to-end time delay measurement value; (2) Vehicle response recognition based, estimating end-to-end time delay measurement value according to the correlation peak value between discrete high-level control instruction change and vehicle state change; (3) Based on the filter update mode, the end-to-end delay measurement value is exponentially moving average or Kalman filter to obtain Δt(t) to represent the jitter delay.

5. The method of claim 1, wherein, The time alignment of step five includes: Map the discrete high-level control instructions to the vehicle control time axis according to the end-to-end delay parameter Δt(t) to obtain the aligned instruction sequence; And the aligned instruction sequence and the vehicle state information sequence in the recent several control periods form a time sequence input sequence, so that the trajectory prediction learning model unit predicts the target trajectory and / or continuous control quantity that takes effect near t+Δt(t) at the current control time t.

6. The method of claim 1, wherein, The prediction time domain in step six is H steps, the step length is the vehicle control period Tc, and the prediction output satisfies one or both of the following forms: Target trajectory sequence: wherein each trajectory point contains a position and / or a heading angle; a sequence of continuous control quantities: wherein the control quantities include one or more of desired speed and steering angle / angular velocity / curvature; And the time reference of the prediction output is phase-advanced by Δt(t) to realize delay compensation.

7. The method of claim 1, wherein, The trajectory prediction learning model unit adopts a time series learning model, including a time series Transformer; or a model pre-trained for this task through transfer learning / fine-tuning.

8. The method of claim 1, wherein, The vehicle dynamics constraints and safety constraints in step seven include one or more of the following: maximum steering angle constraint, maximum steering rate constraint, maximum speed constraint, maximum acceleration constraint, maximum jerk constraint, minimum safety distance constraint, and road boundary constraint.

9. The method of claim 1, wherein, The modification of the prediction result in step seven includes one or both of the following: (1) Projection optimization correction: the final control variable u is solved by minimizing the deviation between the corrected control variable and the predicted control variable, under the conditions of satisfying the vehicle dynamics constraints and safety constraints * ; (2) Local replanning correction: when the predicted trajectory has collision risk or is infeasible, call the local path planning strategy to generate a short-time feasible trajectory, and output the final control quantity u by the trajectory tracking controller * .

10. The method of claim 1, wherein, The step eight includes a multi-rate buffer execution mechanism: when the brain-machine instruction update frequency is lower than the vehicle control frequency, an intermediate control quantity is generated by the trajectory prediction learning model unit between adjacent two brain-machine instructions and written into the control buffer, so that the unmanned vehicle S executes with the vehicle control period T c continuously.

11. The method of claim 1, wherein, The step eight further includes safety policy triggering and degradation control: when the instruction confidence is lower than the threshold, the end-to-end delay parameter Δt(t) suddenly increases or the jitter amplitude exceeds the threshold, or the environmental perception determines that there is high risk, output speed limit control, hold control or parking control.

12. The method of claim 1, wherein, Further including a model adaptation step: collecting sample data composed of discrete high-level control instructions, vehicle state information, execution feedback and end-to-end delay parameter Δt(t) during operation, and used for online or offline updating of the trajectory prediction learning model unit to realize adaptive compensation for different users and different delay distributions.

13. A teleoperated vehicle time delay compensated trajectory predictive control system, comprising: Including: The brain-computer signal acquisition and decoding unit, the vehicle state acquisition unit, the end-to-end delay estimation and updating unit, the instruction-state time alignment unit, the trajectory prediction learning model unit, the trajectory constraint checking and modification unit, the rolling control issuing and buffer execution unit, and the optional environmental perception unit and / or actuator interface; wherein each unit is configured to perform the method steps of any one of claims 1 to 12.

Images