Vehicle-mounted ground-air interconnection sensing device and method based on model predictive control and vehicle

Abstract

The invention discloses a vehicle-mounted ground-air interconnection sensing device and method based on model predictive control and a vehicle, and relates to the technical field of auxiliary driving, and the device is applied to a vehicle. The device comprises an aircraft carrying a camera, an image sensor, an image data processor and an airborne communication module which are connected in sequence; the airborne communication module is wirelessly connected with the vehicle-mounted communication module; the camera obtains an external environment image in a visual angle range; the image sensor updates the parameters of the state-space equation based on the vehicle exterior environment image and the measurement data of the vehicle-mounted sensor; and the image data processor solves the target function based on the constraint condition and the state prediction sequence to obtain a control variable sequence of the vehicle in the prediction time domain. Model prediction control is performed on the vehicle based on comprehensively acquired road condition information, and the rationality and precision of vehicle control are improved.

Description

Technical Field

[0001] The present application relates to the technical field of assisted driving, and particularly to an in-vehicle ground-air interconnection sensing device, method and vehicle based on model predictive control. Background Art

[0002] With the rapid economic development, cars have become an essential means of transportation for people. In today's society, the popularity of cars has changed people's travel and lifestyle, providing convenient, efficient and comfortable transportation options. At present, people's requirements for vehicles not only stay on driving performance, but also put forward higher requirements for driving safety. As a result, model predictive control (MPC) has emerged. MPC is an optimization control method, first proposed by Richalet and Cutler, and is currently mainly applied in the fields of industry and intelligent control. In recent years, due to the rapid development of the automotive industry, it has begun to be applied to autonomous driving technology. The control principle of MPC can be divided into model prediction, rolling optimization and error compensation.

[0003] There are many factors affecting driving safety in actual driving: the driver's line of sight is blocked by large vehicles or buildings in front, the driver is not familiar with the road conditions, etc. In addition, current small and medium-sized cars cannot meet people's needs for safety-assisted driving. For example, in urban congested road conditions, vehicles start and stop frequently, the driver is prone to fatigue, and the line of sight may be blocked by surrounding vehicles; on the highway, the vehicle travels at a high speed. Once an emergency occurs ahead, such as a vehicle breakdown or foreign object, the driver may have an accident due to insufficient reaction. Moreover, the current sensing devices only provide lighting and images of the front road section during vehicle driving. In the case of blocked vision, intelligent perception and driving cannot be achieved during driving, and accurate decision-making cannot be provided in case of emergency to help the owner get out of danger. Such cars have great safety hazards and cannot be put into use well. Therefore, new cars equipped with ground-air interconnection sensing devices have gradually attracted more and more attention from scholars.

[0004] The utility model patent with the publication number of CN211001842U discloses a drone vehicle lamp, which can monitor the road conditions in real time at night, illuminate the vision blind area in advance, and keep a safe distance from the vehicle body, effectively observing the road surface conditions and reducing the danger. It has a good effect on improving safe driving, but does not consider the high cost of using instruments and the inability to help the owner make decisions in case of emergency. In addition, this patent mainly focuses on improving the vision at night, and has limited intelligent perception and decision-making assistance capabilities in complex road conditions during the day. Moreover, its monitoring range of the vehicle's surrounding environment is relatively narrow, and it cannot obtain comprehensive road condition information. Summary of the Invention

[0005] The purpose of this application is to provide a vehicle-mounted ground-air interconnection sensing device, method and vehicle based on model predictive control, which can comprehensively obtain road condition information and perform model predictive control on the vehicle based on the comprehensively obtained road condition information, so as to improve the rationality and accuracy of vehicle control.

[0006] To achieve the above object, this application provides the following solutions:

[0007] In the first aspect, this application provides a vehicle-mounted ground-air interconnection sensing device based on model predictive control. The vehicle-mounted ground-air interconnection sensing device based on model predictive control is applied to a vehicle. The vehicle includes an in-vehicle display screen and an in-vehicle communication module.

[0008] The vehicle-mounted ground-air interconnection sensing device based on model predictive control includes: an aircraft.

[0009] A camera, an image sensor, an image data processor and an airborne communication module are sequentially connected to the aircraft.

[0010] The airborne communication module is wirelessly connected to the in-vehicle communication module.

[0011] The camera is used to acquire an external environment image within the viewing angle range.

[0012] The image sensor is used to update the parameters of the state space equation based on the external environment image and the measurement data of the vehicle-mounted sensor, determine the state prediction sequence of the vehicle within the prediction time domain by using the updated state space equation, and correct the state prediction sequence by using the real-time state and the state prediction sequence of the vehicle.

[0013] The image data processor is used to solve the objective function based on the constraint conditions and the state prediction sequence to obtain the control variable sequence of the vehicle within the prediction time domain.

[0014] The airborne communication module is used to send the control variable sequence to the in-vehicle communication module.

[0015] Optionally, the aircraft includes a tracking module.

[0016] The tracking module is used to control the aircraft to fly above the vehicle and follow the vehicle.

[0017] Optionally, the state space equation is:

[0018] x k+1 =Ax k +Bu k +ω k ;

[0019] Among them, x k+1 is the state vector at time k + 1; A is the system matrix; xk is the state vector at time k, B is the input matrix; u k is the control input vector; ω k is the process noise vector.

[0020] Optionally, the objective function is:

[0021]

[0022] where J is the objective function; N is the prediction horizon; is the transpose matrix of the state prediction at time k for the future time k+i; Q is the state weight matrix; x k+i|k is the state prediction at time k for the future time k+i; is the transpose matrix of the control variable prediction at time k for the future time k+i; R is the control input weight matrix; u k+i|k is the control variable prediction at time k for the future time k+i; is the transpose matrix of the state prediction at time k for the future time k+N.

[0023] Optionally, the constraint condition is:

[0024] u1 ≤ u ≤ u2;

[0025] where u1 is the lower limit of the control variable; u is the control variable; u2 is the upper limit of the control variable.

[0026] In a second aspect, the present application provides a vehicle-to-air interconnection sensing method based on model predictive control, and the vehicle-to-air interconnection sensing method based on model predictive control is applied to the vehicle-to-air interconnection sensing device based on model predictive control; the vehicle-to-air interconnection sensing method based on model predictive control includes:

[0027] Obtain an external vehicle environment image within the camera's field of view;

[0028] Update the parameters of the state space equation based on the external vehicle environment image and the vehicle sensor measurement data, and use the updated state space equation to determine the state prediction sequence of the vehicle within the prediction horizon;

[0029] Use the real-time state of the vehicle and the state prediction sequence to correct the state prediction sequence;

[0030] Solve the objective function based on the constraint condition and the state prediction sequence to obtain the control variable sequence of the vehicle within the prediction horizon;

[0031] Send the control variable sequence to the vehicle-mounted communication module.

[0032] Optionally, the state space equation is:

[0033] x k+1 = Ax k + Bu k + ω k ;

[0034] where x k+1 is the state vector at time k + 1; A is the system matrix; x k is the state vector at time k, B is the input matrix; u k is the control input vector; ω k is the process noise vector.

[0035] Optionally, the objective function is:

[0036]

[0037] where J is the objective function; N is the prediction horizon; is the transpose matrix of the state prediction at time k for future time k + i; Q is the state weight matrix; x k+i|k is the state prediction at time k for future time k + i; is the transpose matrix of the control variable prediction at time k for future time k + i; R is the control input weight matrix; u k+i|k is the control variable prediction at time k for future time k + i; is the transpose matrix of the state prediction at time k for future time k + N.

[0038] Optionally, the constraint condition is:

[0039] u1 ≤ u ≤ u2;

[0040] where u1 is the lower limit of the control variable; u is the control variable; u2 is the upper limit of the control variable.

[0041] In a third aspect, the present application provides a vehicle, and the vehicle applies the on-vehicle ground-air interconnection sensing device based on model predictive control;

[0042] The vehicle interacts with the driver, and when an auxiliary driving start instruction is obtained, the on-vehicle ground-air interconnection sensing device based on model predictive control is started, and the on-vehicle sensor measurement data is sent to the airborne communication module by using the on-vehicle communication module;

[0043] The on-vehicle communication module receives a sequence of control variables; the sequence of control variables is obtained by the on-vehicle ground-air interconnection sensing device based on model predictive control updating the parameters of the state space equation based on the external vehicle environment image and the on-vehicle sensor measurement data, determining a sequence of state predictions of the vehicle within the prediction horizon by using the updated state space equation, and solving the objective function based on the constraint condition and the sequence of state predictions;

[0044] The in-vehicle display screen displays the control variable sequence.

[0045] According to the specific embodiments provided by the present application, the following technical effects are disclosed in the present application:

[0046] The present application provides an in-vehicle ground-air interconnection sensing device, method and vehicle based on model predictive control. The camera module mounted on the sensor and the aircraft (small unmanned aerial vehicle) cooperate to scan the road section ahead and the surrounding environment. The aircraft flies in the height range of 8m to 10m directly above the vehicle, which can not only effectively avoid obstacles to ensure flight safety, but also obtain comprehensive and clear road condition information. The information collected by the sensor is quickly given the best driving strategy for the current period by the MPC. At the same time, the device has a low cost and can meet people's requirements for safe and intelligent driving. Description of the Drawings

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0048] Figure 1 It is a schematic structural diagram of an in-vehicle ground-air interconnection sensing device based on model predictive control in an embodiment of the present application. Detailed Embodiments

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0050] To make the above objects, features and advantages of the present application more obvious and understandable, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0051] In an exemplary embodiment, as Figure 1As shown in the figure, a vehicle-mounted ground-air interconnection sensing device based on model predictive control is provided. The vehicle-mounted ground-air interconnection sensing device based on model predictive control is applied to a vehicle. The vehicle includes a vehicle-mounted display screen and a vehicle-mounted communication module. The vehicle-mounted ground-air interconnection sensing device based on model predictive control includes: an aircraft. An aircraft is equipped with a camera, an image sensor, an image data processor, and an airborne communication module that are connected in sequence. The airborne communication module is wirelessly connected to the vehicle-mounted communication module. The camera is used to acquire an image of the vehicle exterior environment within the viewing range. The image sensor is used to update the parameters of the state space equation based on the vehicle exterior environment image and the measurement data of the vehicle-mounted sensor, determine the state prediction sequence of the vehicle within the prediction time domain using the updated state space equation, and correct the state prediction sequence using the real-time state and the state prediction sequence of the vehicle. The image data processor is used to solve the objective function based on the constraint conditions and the state prediction sequence to obtain the control variable sequence of the vehicle within the prediction time domain. The airborne communication module is used to send the control variable sequence to the vehicle-mounted communication module. The aircraft includes a tracking module. The tracking module is used to control the aircraft to fly above the vehicle and follow the vehicle.

[0052] The image data processor is connected to the vehicle-mounted display screen through the airborne communication module and the vehicle-mounted communication module; among them, the aircraft and the vehicle are communicatively connected through vehicle-mounted Bluetooth. The aircraft flies within the height range of 8m to 10m directly above the vehicle. This height range is determined through a large number of experiments and theoretical analyses. At this height, the aircraft can effectively avoid the vehicle itself and possible surrounding obstacles to ensure flight safety, and can also obtain sufficiently comprehensive and clear road condition information to provide reliable data support for subsequent processing and decision-making. The working principles of the various modules in the vehicle-mounted ground-air interconnection sensing device based on model predictive control in this embodiment are as follows:

[0053] I. Camera.

[0054] The camera is used to collect images of the vehicle exterior environment; its performance parameters have been selected and optimized, including resolution, viewing range, sensitivity, etc.

[0055] 1. Resolution selection and optimization.

[0056] Selection basis: Considering that different objects at different distances need to be clearly identified during vehicle driving, such as traffic signs in the distance, vehicle outlines, and details of pedestrians and obstacles nearby.

[0057] Specific selection: According to actual tests and the processing capabilities of the image sensor, select a high-definition resolution, such as 1080p or higher (e.g., 4K). High-definition resolution can provide more image details, which helps to more accurately identify object features, judge distances, relative speeds, and other information in subsequent image processing and analysis. In scenarios where the vehicle is traveling at a relatively high speed, high resolution can ensure that the target object can be clearly captured even at a long distance, providing sufficient information for making driving decisions in advance.

[0058] 2. Selection and optimization of the viewing angle range.

[0059] Considerations: It is necessary to cover the key areas in front of and around the vehicle to cope with various complex road conditions. When the vehicle is in motion, it is not only necessary to pay attention to the road conditions directly in front, but also to have an understanding of the dynamics of vehicles and pedestrians within a certain range on both the left and right sides. At the same time, ensure a comprehensive view of the road conditions within a relatively large angle range in front to avoid blind spots in the field of vision.

[0060] Optimized design: Through simulation and actual tests, determine the appropriate viewing angle range. For common vehicle models such as sedans, the camera viewing angle can be designed to be within 180° - 210° in the front to ensure that most of the road information in the front can be captured. At the same time, the viewing angle range on each side is 30° - 45° for monitoring the dynamics around the vehicle. Such a combination of viewing angles can effectively reduce the blind spots in the vehicle's field of vision, improve the perception ability of the surrounding environment, especially when turning, changing lanes, or driving at complex intersections, providing more comprehensive road condition information for the driver to assist in decision-making.

[0061] 3. Selection and optimization of the sensitivity.

[0062] Analysis of influencing factors: The sensitivity affects the imaging effect of the camera under different lighting conditions, which is directly related to the clarity and recognizability of the image. Under strong sunlight during the day, it is necessary to avoid overexposure resulting in the loss of image details; while at night or in low-light environments, it is necessary to ensure that the camera can clearly image and capture sufficient light information to identify objects.

[0063] Parameter determination: Select a camera component with an automatic sensitivity adjustment function and good high-sensitivity performance. During the day, the sensitivity is automatically adjusted to a lower level to ensure image quality and color reproduction; when entering a low-light environment, such as at night, in a tunnel, or on a rainy day, the camera automatically increases the sensitivity to enhance the sensitivity to light and ensure clear imaging. At the same time, through algorithm optimization and the cooperation of the image sensor, the noise that may be generated under high sensitivity is suppressed to further improve the clarity and usability of the image.

[0064] 4. Design and testing of the installation position.

[0065] Installation location selection: Based on the vehicle structure and vision requirements, install the camera on the above-mentioned aircraft.

[0066] Testing and adjustment: Through a large number of actual driving tests and simulation experiments, analyze and compare the images obtained by cameras at different installation locations. Observe whether the images captured by the camera can comprehensively cover key areas, whether there are blind spots, and whether the image quality meets the requirements of subsequent processing and decision-making under different road conditions (such as urban roads, highways, curves, uphill and downhill, etc.). According to the test results, fine-tune the installation angle and position of the camera to achieve the best vision coverage effect and image acquisition quality, ensuring that comprehensive and accurate environmental image information can be provided for the vehicle in various driving scenarios.

[0067] II. Image sensor.

[0068] The camera transmits the captured external environment images to the image sensor, and the image sensor analyzes the vehicle's state for predictive equation design. Assume that the dynamic model of the vehicle can be represented by the following discrete-time state-space equation:

[0069] x k+1 =Ax k +Bu k +ω k .

[0070] Where x k+1 is the state vector at time k + 1. A is the system matrix, which describes the internal dynamic characteristics of the system. It reflects the evolution law of the vehicle's state from one moment to the next when there is no external control input (only considering the system's own dynamic changes). It embodies the kinematic and dynamic characteristics of the vehicle itself in discrete time; x k is the state vector at time k, B is related to the input matrix and the control input vector, and it represents the degree and manner of the influence of control inputs (such as the vehicle's steering angle, acceleration, etc.) on the vehicle's state. It is the bridge connecting the control commands of the driver or the autonomous driving system and the actual change of the vehicle's motion state. For example, control quantities such as the vehicle's steering angle and acceleration will affect the vehicle's state through this matrix; u k is the control input vector (such as the vehicle's steering angle, acceleration, etc.); ω k is the process noise vector, which is used to consider the inevitable noise interference in the actual environment. Based on the collected image information, the image sensor uses advanced MPC algorithms and models to accurately estimate the current state of the vehicle. The following explains how the MPC algorithm and model achieve advanced effects:

[0071] 1. High-precision vehicle state prediction.

[0072] Dynamic model construction and real-time update: An accurate dynamic model is constructed based on the physical characteristics and kinematic principles of the vehicle, incorporating key state variables such as the vehicle's position, speed, acceleration, and steering angle into the model. For example, the vehicle's motion change law under different control inputs (such as throttle, brake, and steering) is described by vehicle dynamics equations. During driving, the environmental information around the vehicle collected by the image sensor and the data from in-vehicle sensors (such as vehicle speed sensor, inertial measurement unit, etc.) are used to update the parameters in the model in real time to adapt to the changes in the actual driving state of the vehicle.

[0073] Multi-step prediction to improve accuracy: The MPC algorithm estimates the future state of the vehicle through multi-step prediction within the prediction horizon. For example, based on the current vehicle state and the known control input sequence, the dynamic model is used to predict the vehicle's position, speed, and attitude changes at several future sampling times (such as every 0.5 seconds within the next 5 - 10 seconds). Compared with the decision-making based only on the current state, this multi-step prediction can detect potential dangers or opportunities to optimize the driving path in advance, providing a more forward-looking decision-making basis for the driver, thereby improving driving safety and efficiency.

[0074] 2. Generation of real-time optimal control strategy.

[0075] Objective function optimization considering multiple factors: The objective function design of the MPC algorithm comprehensively considers multiple key factors, such as vehicle driving safety (maintaining a safe distance from surrounding obstacles and avoiding collisions), comfort (reducing sudden acceleration, sudden braking, and sharp steering), efficiency (optimizing fuel consumption or power utilization), and accuracy of the driving path (accurately following the preset route or navigation guidance), etc. By assigning appropriate weight coefficients to these factors, the multi-objective optimization problem is transformed into a single objective function, and the control input sequence that minimizes (or maximizes) the objective function is solved in each control cycle. For example, in urban congested traffic conditions, the weights of comfort and fuel economy may be increased; when driving on the highway, more attention is paid to safety and accuracy of the driving path.

[0076] Dynamically adjust weights to adapt to different road conditions: According to the real-time road condition information during vehicle driving, such as road type (urban road, highway, rural road, etc.), traffic flow, weather conditions, etc., the weight coefficients of each factor in the objective function are dynamically adjusted. The environmental images collected by the image sensor and the data from other in-vehicle sensors (such as rain sensor, light sensor, etc.) can be used for road condition recognition and classification. In this way, the MPC algorithm can flexibly generate the most suitable control strategy for the current road condition according to the changes in the actual driving environment, ensuring that the vehicle always drives in an optimal state.

[0077] 3. Constraint condition handling to ensure driving safety and stability.

[0078] Physical constraints limit the vehicle's motion range: Considering the physical limitations of the vehicle itself, such as the maximum steering angle, maximum acceleration, maximum deceleration, etc., corresponding hard constraint conditions are set in the MPC algorithm. These constraints ensure that the generated control inputs do not exceed the limits of the vehicle's physical performance, preventing the vehicle from losing control or being damaged due to excessive operation. For example, when the vehicle is traveling at high speed, by limiting the rate of change of the steering angle, it is possible to avoid rollover accidents caused by sudden steering; during braking, according to the road adhesion and the performance of the vehicle braking system, the maximum deceleration is reasonably limited to prevent the wheels from locking and losing the steering ability.

[0079] Environmental constraints avoid collision risks: Combining the data from image sensors and other environmental perception sensors, obstacles, road boundaries, and other traffic participants (vehicles, pedestrians, etc.) around the vehicle are identified, and maintaining a safe distance from these objects is incorporated as a constraint condition into the MPC algorithm. For example, ensuring that the vehicle maintains a safe following distance from the vehicle in front to avoid rear-end collisions; when changing lanes or turning, ensuring that the vehicle does not intrude into adjacent lanes or collide with roadside obstacles. By continuously monitoring the environment and adjusting the constraint conditions, the MPC algorithm can effectively avoid collisions between the vehicle and surrounding objects during driving, improving driving safety.

[0080] 4. Feedback correction improves control accuracy and adaptability.

[0081] Predictive error corrects the prediction model: After each control cycle, the actual measured vehicle states (such as precise position, speed, etc. obtained through on-vehicle sensors) are compared with the states predicted by the MPC algorithm to calculate the prediction error. Using this error information, a suitable correction method (such as Kalman filtering or other state estimation techniques) is adopted to correct the prediction model, updating the parameters in the model to make the prediction model more accurately reflect the actual dynamic characteristics of the vehicle. For example, if it is found that there is a deviation between the actual change in vehicle speed and the predicted value, it may be due to factors such as road slope and friction changes not being accurately considered in the original model. Through the feedback correction mechanism, the parameters related to speed in the model can be adjusted to improve the accuracy of subsequent predictions.

[0082] Online adjustment optimizes the control strategy: Based on the prediction model after feedback correction and the new vehicle state information, it is re-evaluated whether the current control strategy is still optimal. If it is found that the actual road conditions or vehicle states have changed significantly (such as encountering sudden obstacles, road construction, etc.), the MPC algorithm will online adjust parameters such as the objective function, constraint conditions, or prediction horizon, re-solve the optimization problem, and generate a new optimal control input sequence. This real-time feedback and adjustment mechanism enables the MPC algorithm to quickly adapt to various complex and changing driving scenarios, always providing the best control strategy for the vehicle to ensure stable and safe driving of the vehicle under different road conditions.

[0083] 5. Efficient computation for real-time control.

[0084] Optimized algorithms to accelerate the solution process: To meet the stringent real-time requirements of in-vehicle systems, the MPC algorithm employs efficient numerical optimization algorithms to solve optimization problems. For example, interior point methods, sequential quadratic programming (SQP) algorithms, etc. These algorithms can quickly find the optimal control input sequence that satisfies the constraint conditions within limited computing resources and time. Through the careful design and optimization of the optimization algorithms, the computational complexity is reduced, and the computational efficiency is improved, ensuring that the MPC algorithm can complete the calculation and output control instructions within each control cycle (usually from dozens of milliseconds to hundreds of milliseconds), achieving real-time control of the vehicle.

[0085] Hardware acceleration to enhance computational performance: Combining the hardware characteristics of the in-vehicle computing platform, hardware acceleration technology is utilized to further improve the computational speed of the MPC algorithm. For example, hardware devices such as graphics processing units (GPUs) or field-programmable gate arrays (FPGAs) are employed to accelerate computationally intensive tasks such as matrix operations and optimization solutions in the MPC algorithm. By reasonably allocating computational tasks to different hardware units and fully leveraging the parallel computing capabilities of the hardware, efficient algorithm execution is achieved, thereby ensuring that the in-vehicle ground-air interconnection sensing device can promptly respond to various changes during vehicle driving and provide accurate and timely decision-making support for the driver.

[0086] Through the above advanced features and functions in aspects such as vehicle state prediction, control strategy generation, constraint handling, feedback correction, and computational efficiency improvement, the MPC algorithm has achieved advanced effects in the in-vehicle ground-air interconnection sensing device, effectively improving the intelligent driving level and safety of the vehicle and providing strong technical support for the driver to cope with complex road conditions.

[0087] III. Image data processor.

[0088] The image data processor constructs the objective function. The goal of MPC is to determine the optimal control sequence {u k , u k+1 … u k+N-1} by solving the following finite-horizon optimization problem at each sampling time k:

[0089]

[0090] where J is the objective function. N is the prediction horizon. is the transpose matrix of the state prediction at time k for future time k + i. Q is the state weight matrix. x k+i|k is the state prediction at time k for future time k + i. is the transpose matrix of the control variable prediction at time k for future time k + i. R is the control input weight matrix. uk+i|k For predicting the control variable at future time k + i at time k. It is the transposed matrix for predicting the state at future time k + N at time k.

[0091] 1. N (prediction horizon).

[0092] Definition: It represents the number of time steps for predicting the future system state and control input when the MPC algorithm performs optimization calculations.

[0093] Significance: The selection of the prediction horizon directly affects the performance of the MPC algorithm. A shorter prediction horizon may lead to insufficient consideration of future changes by the algorithm, making it unable to make effective control decisions in a timely manner, especially when the vehicle is traveling at a high speed or the road conditions are complex. While a longer prediction horizon can consider future situations more comprehensively, it will increase the computational complexity, require higher on-vehicle computing resources, and may result in inaccurate prediction results in the more distant future due to the accumulation of prediction errors. Therefore, it is necessary to reasonably select the value of the prediction horizon N according to factors such as the vehicle's driving speed, road condition complexity, and the performance of the on-vehicle computing platform. For example, when driving at a high speed on a highway, since the vehicle speed is high and the road conditions are relatively simple, the prediction horizon can be appropriately extended to plan a more reasonable driving strategy in advance; in urban congested road conditions, where the vehicle speed is slow and the surrounding environment changes frequently, a shorter prediction horizon may be more appropriate, and at the same time, the computational amount can be reduced to ensure the real-time performance of the algorithm.

[0094] 2. Q (state weight matrix).

[0095] Definition: It is a symmetric positive definite matrix, and its elements q ij (i, j = 1, 2,..., n, where n is the dimension of the state vector) represent the trade-off of the relative importance between different state variables in the state vector.

[0096] Significance: By adjusting the element values of matrix Q, according to the actual needs and driving scenarios of the vehicle, the influence of certain state variables on the objective function can be emphasized or weakened. In scenarios that focus on the accuracy of the vehicle's driving path, such as highway autonomous driving or navigation-guided driving, the weights of state variables related to the vehicle's position (such as longitudinal and lateral position deviations) should be relatively high, that is, the values of elements q 11 、q 22 etc. are relatively large to ensure that the vehicle can closely follow the preset path. While in the case of emphasizing vehicle stability, such as when the vehicle is driving on a curve or encountering side wind interference, the weights of state variables related to the vehicle's attitude (such as yaw rate, roll angle, etc.) should be increased, so that the control strategy is more inclined to maintain the stable state of the vehicle and avoid rollover or loss of control.

[0097] 3. R (control input weight matrix).

[0098] Definition: It is also a symmetric positive definite matrix, and its element r ij (i, j = 1, 2, …, m, where m is the dimension of the control input vector) is used to measure the relative importance between different control variables in the control input vector and the penalty degree for the magnitude of the control action.

[0099] Significance: The role of matrix R is to balance the energy consumption of the control input and the control effect when optimizing the control strategy. A larger value means being more sensitive to the change of the corresponding control variable, which will prompt the MPC algorithm to generate a smoother and less variable control input sequence to reduce the intensity of the control action, thereby improving the comfort of vehicle driving and avoiding operations such as frequent hard acceleration, hard braking, and large - amplitude steering. At the same time, reasonably setting matrix R also helps to protect the mechanical components of the vehicle and extend its service life. For example, when driving on urban roads, due to the complex traffic conditions and frequent stops and starts, in order to improve the comfort of passengers, a relatively large r 11 value is usually set to limit the excessive change of acceleration; while in scenarios where precise vehicle steering control is required, such as passing through narrow curves or making emergency evasions, the r 22 value may need to be adjusted according to specific situations to achieve flexible steering control while ensuring safety.

[0100] 4.x k+i|k (State prediction at time k for future time k + i).

[0101] Definition: It is a vector that represents the predicted state of the vehicle at future time k + i based on the state information of the vehicle at the current time k and the system dynamic model. Its dimension depends on the number of selected state variables.

[0102] Significance: State prediction is one of the core parts of the MPC algorithm. Through accurate state prediction, the MPC controller can understand in advance the possible states of the vehicle at different future times, and thus formulate corresponding optimal control strategies according to the objective function and constraint conditions. The accuracy of state prediction directly affects the effectiveness and reliability of the control strategy. It depends on an accurate vehicle dynamic model and an accurate estimation of the current state (such as vehicle position, speed, etc. obtained through image sensors, in - vehicle sensors, etc.). In practical applications, as time goes by, prediction errors may gradually accumulate. Therefore, a feedback correction mechanism (such as using the difference between the actual measurement value and the predicted value to correct the model parameters or prediction results) needs to be combined to continuously improve the accuracy of state prediction to ensure that the MPC algorithm can work continuously and stably in a complex driving environment.

[0103] 5.u k+i|k (Control input prediction at time for future time k + i).

[0104] Definition: It is a vector representing the control input that should be applied to the vehicle at the future time k+i predicted at time k. Its dimension is the same as the number of vehicle control variables.

[0105] Significance: Control input prediction is a sequence of future control strategies optimized by the MPC algorithm based on state prediction results and the objective function. It reflects how the vehicle's control variables (such as acceleration, deceleration, steering, etc.) should be adjusted at different future times to guide the vehicle towards the optimal state. By optimizing the control input prediction sequence, the MPC algorithm can achieve precise control of the vehicle's driving process while meeting various constraints (such as vehicle physical limitations, safety distance requirements, etc.), improving the vehicle's performance and safety. In actual driving, the on-vehicle control system will select the first element in the control input prediction sequence (i.e., u k|k ) as the actual control input at the current time and apply it to the vehicle, and then repeat the above prediction and optimization process at subsequent times to achieve rolling optimization control to adapt to the changing road conditions and vehicle states.

[0106] After the objective function is established, the theoretical rolling optimization of the MPC controller is established. The image data processor determines the appropriate weight matrices Q, R, and P and the prediction horizon N according to the above formula and the actual requirements of the vehicle (such as safety, comfort, efficiency, etc.). When determining the weight matrices, it is necessary to comprehensively consider the importance of various performance indicators of the vehicle in different driving scenarios. For example, when driving at high speed, more attention may be paid to the stability and safety of the vehicle, and the weight of the state deviation may be relatively high at this time; while in urban congested road conditions, more attention may be paid to the comfort and fuel economy of the vehicle, and the weight of the control input energy may be adjusted accordingly. The determination of the prediction horizon N also needs to consider factors such as the vehicle's driving speed, road condition complexity, and driver's reaction time. Generally speaking, the faster the driving speed and the more complex the road conditions, the prediction horizon may need to be appropriately extended to ensure that a reasonable driving strategy can be planned in advance.

[0107] The image data processor establishes constraint conditions. To prevent the vehicle from rolling over, the vehicle's steering angle is restricted to meet the requirements of safe vehicle driving. The constraint conditions include u1≤u≤u2, where u1 is the lower limit of the control variable, u is the control variable, and u2 is the upper limit of the control variable. The determination of these lower and upper limits needs to consider the physical characteristics of the vehicle, such as the vehicle's center of gravity position, the friction between the tires and the ground, the vehicle's wheelbase, and the characteristics of the suspension system. Through precise mechanical analysis and a large amount of experimental data, the safe range of the vehicle's steering angle is determined under different driving speeds and road conditions, thus ensuring the stability and safety of the vehicle during driving.

[0108] After the prediction model, objective function and constraint conditions are established, the image data processor organizes the data and performs secondary optimization to obtain the best driving route. By solving the above finite time domain optimization problem, the first element of the optimal control sequence can be obtained. As the input at the current moment, it is applied to the vehicle. In the secondary optimization solution process, the image data processor will use advanced optimization algorithms, such as gradient descent method and Newton method, to comprehensively consider multiple factors such as prediction model, objective function and constraint conditions. Through continuous iteration and optimization, it gradually approaches the optimal solution and finally obtains the best driving route that can meet multiple goals such as safety, comfort and efficiency to the greatest extent under the current road conditions and vehicle status.

[0109] The image data processor will transmit the best solution to the in-vehicle display screen via the in-vehicle Bluetooth. The synchronization transmission process requires high accuracy and real-time performance to ensure that the driver can obtain the latest driving strategy information at the first time. The display effect of the in-vehicle display screen has also been optimized and designed to clearly and intuitively present the road conditions ahead and the best driving plan, including the vehicle's recommended driving speed, steering angle, lane change suggestions, etc., to provide comprehensive decision support for the driver.

[0110] The driver can check the information on the in-vehicle display screen to obtain information about the road conditions ahead and the best plan, so that he can drive safely. The driver can reasonably adjust his driving behavior based on this information. For example, when the road conditions ahead are complex and it is recommended to slow down, the driver can reduce the speed in time; when it is necessary to change lanes and the display screen gives clear instructions, the driver can change lanes safely. In this way, the driver can better deal with complex road conditions and improve driving safety and comfort.

[0111] This embodiment provides a vehicle-mounted ground-to-air interconnected sensing device based on model predictive control, which specifically solves the two major safety hazards faced by drivers during driving: one is that the line of sight is blocked by large vehicles or buildings in front, and the other is unfamiliarity with road conditions. These situations may cause the driver to be unable to obtain accurate road condition information in a timely manner, thereby increasing the risk of traffic accidents. The camera module mounted on the sensor mainly scans the road section ahead, and MPC quickly gives the best driving strategy for the current period. At the same time, its equipment cost is low and can meet people's requirements for safe and intelligent driving. The specific effects and advantages are as follows:

[0112] 1. Cost advantage: Compared with the existing technology, the equipment of the present invention is low in cost, which makes it have greater advantages in market promotion, allowing more consumers to accept this vehicle-mounted ground-to-air interconnected sensor device, thereby improving the safety performance of the vehicle.

[0113] 2. Intelligent decision-making ability: The device mainly scans the road section ahead through the camera module mounted on the sensor, and the MPC can quickly give the best driving strategy for the current period. This can provide accurate decision-making support for the driver when the driver's line of sight is blocked or unfamiliar with the road conditions, helping the driver better cope with complex road conditions and improving driving safety.

[0114] 3. Comprehensive road condition perception: By flying a small unmanned aerial vehicle at a specific height above the vehicle and collecting the surrounding environment of the vehicle through a camera, the device can obtain more comprehensive road condition information. It includes not only the situation in front of the vehicle but also the monitoring of the surrounding area, providing richer data support for driving decisions. Compared with the existing sensing devices that can only provide images of the road section ahead, it has obvious advantages.

[0115] 4. Accurate vehicle state estimation: Based on the collected image information, the image sensor uses advanced algorithms and models to accurately estimate the current state of the vehicle. Combining the vehicle's dynamic model and known system parameters, the actual state vector of the vehicle can be accurately determined, providing more accurate basic data for subsequent MPC control, thereby improving the accuracy of the driving strategy.

[0116] 5. Personalized driving strategy: When determining the weight matrix and prediction horizon, the image data processor comprehensively considers the importance of various performance indicators of the vehicle in different driving scenarios. This personalized setting enables the device to formulate a driving strategy that better meets the needs of the vehicle and the driver according to the actual situation, improving driving comfort and efficiency.

[0117] 6. Enhancement of safety guarantee: By establishing constraint conditions such as restricting the steering angle of the vehicle, the device can effectively prevent the vehicle from rolling over and ensure the stability and safety of the vehicle during driving. At the same time, it can provide accurate decision-making support for the driver in case of emergencies, further enhancing the safety guarantee of vehicle driving.

[0118] A vehicle-mounted ground-air interconnection sensing device based on model predictive control provided in this embodiment has the following design process:

[0119] 1. Device construction and initialization.

[0120] According to the design requirements, assemble the vehicle-mounted ground-air interconnection sensing device based on MPC. Select a suitable small unmanned aerial vehicle, which should have good flight stability, wind resistance, and communication capabilities. Debug the flight control system of the unmanned aerial vehicle to ensure that it can fly stably in a complex environment and accurately receive and execute instructions from the vehicle.

[0121] Install a camera and ensure that its performance parameters such as resolution, viewing angle range, and sensitivity meet the requirements. According to the type of vehicle and the actual usage scenario, precisely design and adjust the installation position of the camera so that it can cover the key areas in front of and around the vehicle to the greatest extent, such as the areas within the range of 180° - 210° in the front and 30° - 45° on each of the left and right sides, thereby obtaining comprehensive, accurate, and practically valuable environmental images.

[0122] Connect the image sensor, image data processor, and in-vehicle display screen to establish a good communication link. Use high-quality data cables and interfaces to ensure the stability and accuracy of data transmission. At the same time, test the communication link to check for problems such as data loss or transmission delay.

[0123] Connect the aircraft to the vehicle through in-vehicle Bluetooth and set the aircraft to fly within the height range of 8m to 10m directly above the vehicle. Pair and test the Bluetooth connection to ensure stable communication between the aircraft and the vehicle. During this process, the coverage range and interference of the Bluetooth signal need to be considered, and the stability of the Bluetooth connection can be optimized by adjusting the Bluetooth transmission power and selecting a suitable frequency band.

[0124] Perform initialization settings on the image data processor. According to the type of vehicle (such as sedan, truck, SUV, etc.) and common driving scenarios, initially set the values of the weight matrices Q, R, and P, and the prediction horizon N.

[0125] 2. Data collection and processing during driving.

[0126] When the vehicle starts and drives, the camera begins to collect images of the external environment. Under different driving scenarios, the image information collected by the camera has different characteristics. For example, when the vehicle enters a complex urban section with multiple intersections and vehicles, the camera collects image information such as the vehicles, pedestrians, and traffic signs in front, and may also collect information such as the shops and billboards on the roadside; when driving on the highway, the camera mainly collects information such as the distance, speed, and lane information of the vehicles in front.

[0127] The collected images are transmitted to the image sensor. Based on the collected image information, the image sensor uses the above-mentioned advanced MPC algorithm and model to accurately estimate the current state of the vehicle. Assume that the distance between the vehicle and a vehicle in front in the image is gradually decreasing, and the relative speed indicates a collision risk between the two vehicles. The image sensor combines the vehicle's dynamic model to determine the value of the actual state vector x k of the vehicle, which may include information such as the current position, speed, and acceleration of the vehicle.

[0128] Based on the analysis results of the image sensor, the image data processor further determines the appropriate weight matrix and prediction horizon. If the image sensor detects that the vehicle is in a dangerous state, such as the vehicle being too close to the vehicle in front as shown in the image, the image data processor can appropriately adjust the weight matrix, increasing the weight of the state deviation. At the same time, according to the vehicle's driving speed and the complexity of the road conditions, it may be necessary to appropriately extend the prediction horizon to ensure that a reasonable driving strategy can be planned in advance to avoid traffic accidents.

[0129] 3. Determination and display of the optimal driving route.

[0130] After the image data processor establishes the objective function and improves the constraint conditions, it organizes the data and performs a secondary optimization solution. When establishing the objective function, in addition to considering conventional factors such as vehicle safety, comfort, and efficiency, some special objective function terms can also be added according to the actual situation. For example, in certain specific application scenarios, the environmental performance of the vehicle can be considered, and the vehicle's exhaust emissions can be used as an objective function term for optimization.

[0131] By solving the finite-horizon optimization problem, using optimization algorithms such as the gradient descent method, and comprehensively considering various factors such as the prediction model, objective function, and constraint conditions, the optimal solution is gradually approximated. During the solution process, reasonable iteration times and convergence criteria can be set to ensure that an accurate optimal solution can be obtained within a reasonable time. Finally, the optimal driving route under the current road conditions and vehicle state is obtained. For example, a strategy is obtained that the vehicle needs to decelerate and change lanes to the left, or a strategy is obtained that the vehicle needs to maintain a certain vehicle distance and adjust the vehicle speed on the highway.

[0132] The image data processor transmits the obtained optimal solution to the in-vehicle display screen through in-vehicle Bluetooth synchronization. The in-vehicle display screen clearly and intuitively presents the road conditions information ahead and the optimal driving solution, including content such as the recommended driving speed of the vehicle, steering angle, and lane change suggestions. To improve the display effect, the in-vehicle display screen can adopt a high-definition resolution and an adaptive brightness adjustment function, automatically adjusting the screen brightness according to the intensity of the external light, so that the driver can clearly see the information on the display screen. At the same time, the in-vehicle display screen can also set some interactive functions. For example, the driver can obtain more detailed information or fine-tune the driving strategy by touching the screen or voice commands.

[0133] The driver performs safe driving operations based on this information. When performing driving operations, the driver can refer to the information on the in-vehicle display screen, but the final driving decision is still made by the driver himself. If the driver has different views or opinions on the driving strategy, he can adjust it according to his own experience and judgment.

[0134] 4. Maintenance and upgrade of the device.

[0135] Maintain the device regularly and check the working status of components such as the drone, camera, image sensor, image data processor, and in-vehicle display screen. Replace or charge the drone's battery regularly to ensure it has sufficient power for flight. Check whether the lens of the camera is clean. If there is dirt, clean it in time to ensure the quality of image acquisition. Conduct performance tests on the image sensor and image data processor to check for problems such as abnormal data processing or faults. Check the in-vehicle display screen to ensure good display effect without flickering or screen distortion.

[0136] Upgrade the device according to the vehicle's usage conditions and the development of technology. For example, with the emergence of new algorithms and models, the algorithms in the image sensor and image data processor can be updated to improve the accuracy of vehicle state estimation and the rationality of driving strategies. When more advanced drone technology appears, the drone can be replaced to improve the overall performance of the device. At the same time, adjust and optimize the display content and interaction functions of the in-vehicle display screen according to the driver's feedback and requirements to improve the driver's usage experience.

[0137] In another embodiment, a vehicle-ground-air interconnected sensing method based on model predictive control is provided. The vehicle-ground-air interconnected sensing method based on model predictive control is applied to the above-mentioned vehicle-ground-air interconnected sensing device based on model predictive control. The vehicle-ground-air interconnected sensing method based on model predictive control includes:

[0138] Step 1: Obtain the out-of-vehicle environment image within the camera's view range.

[0139] Step 2: Update the parameters of the state space equation based on the out-of-vehicle environment image and the measurement data of the vehicle sensors, and use the updated state space equation to determine the state prediction sequence of the vehicle within the prediction horizon. The state space equation is: x k+1 = Ax k + Bu k + ω k . Wherein, x k+1 is the state vector at time k + 1. A is the system matrix. x k is the state vector at time k, B is the input matrix. u k is the control input vector. ω k is the process noise vector.

[0140] Step 3: Use the real-time state of the vehicle and the state prediction sequence to correct the state prediction sequence.

[0141] Step 4: Solve the objective function based on the constraint conditions and the state prediction sequence to obtain the control variable sequence of the vehicle within the prediction horizon. The objective function is: Wherein, J is the objective function. N is the prediction horizon. is the transpose matrix of the state prediction at time k for future time k+i. Q is the state weight matrix. x k+i|k is the state prediction at time k for future time k+i. is the transpose matrix of the control variable prediction at time k for future time k+i. R is the control input weight matrix. u k+i|k is the control variable prediction at time k for future time k+i. is the transpose matrix of the state prediction at time k for future time k+N. The constraint condition is: u1≤u≤u2. Where, u1 is the lower limit of the control variable. u is the control variable. u2 is the upper limit of the control variable.

[0142] Step 5: Send the control variable sequence to the vehicle-mounted communication module.

[0143] In another embodiment, a vehicle is provided. The vehicle applies the vehicle-mounted ground-air interconnection sensing device based on model predictive control. The vehicle interacts with the driver. When an auxiliary driving start instruction is obtained, the vehicle-mounted ground-air interconnection sensing device based on model predictive control is started, and the vehicle-mounted sensor measurement data is sent to the airborne communication module by using the vehicle-mounted communication module. The vehicle-mounted communication module receives the control variable sequence. The control variable sequence is obtained by the vehicle-mounted ground-air interconnection sensing device based on model predictive control updating the parameters of the state space equation based on the external vehicle environment image and the vehicle-mounted sensor measurement data, determining the state prediction sequence of the vehicle within the prediction time domain by using the updated state space equation, and solving the objective function based on the constraint condition and the state prediction sequence. The vehicle-mounted display screen displays the control variable sequence.

[0144] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0145] In this article, specific examples are used to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A vehicle-mounted ground-air interconnection sensing device based on model predictive control, characterized in that, The vehicle-mounted ground-air interconnected sensing device based on model predictive control is applied to a vehicle; the vehicle includes a vehicle-mounted display screen and a vehicle-mounted communication module; The vehicle-mounted ground-air interconnected sensing device based on model predictive control includes: an aircraft; The aircraft is equipped with a camera, an image sensor, an image data processor, and an airborne communication module connected in sequence; The airborne communication module is wirelessly connected to the vehicle-mounted communication module; The camera is used to obtain an image of the vehicle exterior environment within the viewing angle range; The image sensor is used to update the parameters of the state space equation based on the vehicle exterior environment image and the measurement data of the vehicle-mounted sensors, determine the state prediction sequence of the vehicle within the prediction time domain using the updated state space equation, and correct the state prediction sequence using the real-time state and the state prediction sequence of the vehicle; The image data processor is used to solve the objective function based on the constraint conditions and the state prediction sequence to obtain the control variable sequence of the vehicle within the prediction time domain; The airborne communication module is used to send the control variable sequence to the vehicle-mounted communication module.

2. The vehicle-mounted ground-air interconnection sensing device based on model predictive control according to claim 1, wherein The aircraft includes a tracking module; The tracking module is used to control the aircraft to fly above the vehicle and follow the vehicle.

3. The vehicle-mounted ground-air interconnection sensing device based on model predictive control according to claim 1, characterized in that, The state space equation is: x k+1 = Ax k + Bu k + ω k ; where x k+1 is the state vector at time k + 1; A is the system matrix; x k is the state vector at time k, B is the input matrix; u k is the control input vector; ω k is the process noise vector.

4. The vehicle-mounted ground-air interconnection sensing device based on model predictive control according to claim 3, characterized in that, The objective function is: where, J is the objective function; N is the prediction horizon; is the transposed matrix of the state prediction at time k for future time k+i; Q is the state weight matrix; x k+i|k is the state prediction at time k for future time k+i; is the transposed matrix of the control variable prediction at time k for future time k+i; R is the control input weight matrix; u k+i|k is the control variable prediction at time k for future time k+i; is the transposed matrix of the state prediction at time k for future time k+N.

5. The vehicle-mounted ground-air interconnection sensing device based on model predictive control according to claim 4, wherein The constraint conditions are: u1≤u≤u2; where, u1 is the lower limit of the control variable; u is the control variable; u2 is the upper limit of the control variable.

6. A vehicle-mounted ground-air interconnection sensing method based on model predictive control, characterized in that The vehicle-mounted ground-air interconnected sensing method based on model predictive control is applied to a vehicle-mounted ground-air interconnected sensing device based on model predictive control according to any one of claims 1-5; The vehicle-mounted ground-air interconnected sensing method based on model predictive control includes: Obtaining an image of the vehicle exterior environment within the viewing angle range of the camera; Updating the parameters of the state space equation based on the vehicle exterior environment image and the measurement data of the vehicle-mounted sensors, and determining the state prediction sequence of the vehicle within the prediction time domain using the updated state space equation; Correcting the state prediction sequence using the real-time state and the state prediction sequence of the vehicle; Solving the objective function based on the constraint conditions and the state prediction sequence to obtain the control variable sequence of the vehicle within the prediction time domain; Sending the control variable sequence to the vehicle-mounted communication module.

7. The vehicle-to-air interconnection sensing method based on model predictive control according to claim 6, characterized in that, The state space equation is: x k+1 = Ax k + Bu k + ω k ; where x k+1 is the state vector at time k + 1; A is the system matrix; x k is the state vector at time k, B is the input matrix; u k is the control input vector; ω k is the process noise vector.

8. The vehicle-to-air interconnection sensing method based on model predictive control according to claim 7, characterized in that, The objective function is: Among them, J is the objective function; N is the prediction horizon; is the transposed matrix of the state prediction at time k for the future time k+i; Q is the state weight matrix; x k+i|k is the state prediction at time k for the future time k+i; is the transposed matrix of the control variable prediction at time k for the future time k+i; R is the control input weight matrix; u k+i|k is the control variable prediction at time k for the future time k+i; is the transposed matrix of the state prediction at time k for the future time k+N.

9. The vehicle-to-air interconnection sensing method based on model predictive control according to claim 8, characterized in that The constraint conditions are: u1≤u≤u2; where, u1 is the lower limit of the control variable; u is the control variable; u2 is the upper limit of the control variable.

10. A vehicle, characterized in that, The vehicle applies a vehicle-mounted ground-air interconnected sensing device based on model predictive control according to any one of claims 1-5; The vehicle interacts with the driver, and when an auxiliary driving start instruction is obtained, the vehicle-mounted ground-air interconnected sensing device based on model predictive control is started, and the vehicle-mounted sensor measurement data is sent to the airborne communication module using the vehicle-mounted communication module; The vehicle-mounted communication module receives a sequence of control variables; the sequence of control variables is obtained by a vehicle-mounted ground-air interconnection sensing device based on model predictive control updating the parameters of the state space equation based on the external vehicle environment image and the measurement data of vehicle-mounted sensors, determining a state prediction sequence of the vehicle within a prediction time domain by using the updated state space equation, and solving an objective function based on constraint conditions and the state prediction sequence. The vehicle-mounted display screen displays the sequence of control variables.

Images