Driving control method, remote control terminal, and vehicle

Abstract

This application relates to a driving control method, a remote control terminal, and a vehicle. The method is applied to the remote control terminal. The remote control terminal monitors network latency parameters between itself and the vehicle; receives a scene complexity score of the vehicle's current environment from the vehicle; determines the vehicle control mode based on the scene complexity score and network latency parameters, and controls the vehicle according to the vehicle control mode. This application determines the vehicle control mode based on the scene complexity score and network latency parameters, and controls the vehicle according to the vehicle control mode. This solves the problem that vehicles lack an adaptive mechanism for network latency fluctuations, making it impossible to guarantee control stability and safety under high latency conditions, and improves network latency adaptability.

Description

Technical Field

[0001] This application relates to the field of vehicles, and more particularly to a driving control method, a remote control terminal, and a vehicle. Background Technology

[0002] With the rapid development of autonomous driving technology, autonomous vehicles have been put into commercial trial operation in many cities. To ensure driving safety, autonomous vehicles need to be equipped with remote safety operators who can remotely monitor and take over the vehicle when necessary. However, existing remote control technologies face the challenge of network latency fluctuations in practical applications. The end-to-end latency of 4G / 5G networks often fluctuates between 50-300ms. When a vehicle is traveling at 60km / h, a 100ms latency means that the vehicle has traveled approximately 1.67 meters, which could lead to serious safety hazards in emergency situations. Existing technologies lack adaptive mechanisms for network latency fluctuations and cannot guarantee the stability and safety of control under high latency conditions. Summary of the Invention

[0003] This application provides a driving control method, a remote control terminal, and a vehicle, aiming to solve the problem that vehicles lack an adaptive mechanism for network latency fluctuations, making it impossible to guarantee the stability and safety of control under high latency conditions.

[0004] In a first aspect, this application provides a driving control method applied to a remote control terminal, wherein the remote control terminal and the vehicle are connected via network communication. The method includes: monitoring network latency parameters between the remote control terminal and the vehicle; receiving a scene complexity score of the current environment of the vehicle sent by the vehicle; determining a vehicle control mode based on the scene complexity score and the network latency parameters; and controlling the vehicle according to the vehicle control mode.

[0005] The above technical solution determines the vehicle control mode based on the scenario complexity score and network latency parameters, and controls the vehicle according to the vehicle control mode. This solves the problem that the vehicle lacks an adaptive mechanism for network latency fluctuations and cannot guarantee the stability and safety of control under high latency conditions, thus improving the network latency adaptability.

[0006] In some embodiments of this application, the network latency parameter includes the average round-trip latency. Determining the vehicle control mode based on the scenario complexity score and the network latency parameter includes: determining a first dynamic latency threshold and a second dynamic latency threshold based on a baseline latency threshold, a preset dynamic adjustment coefficient, and the scenario complexity score; comparing the average round-trip latency with the first dynamic latency threshold and the second dynamic latency threshold to determine the range of the average round-trip latency; if the average round-trip latency is less than the first dynamic latency threshold, determining the vehicle control mode as a direct control mode; if the average round-trip latency is greater than or equal to the first dynamic latency threshold and less than the second dynamic latency threshold; if the average round-trip latency is greater than or equal to the second dynamic latency threshold, determining the vehicle control mode as a supervised autonomous mode.

[0007] The above technical solution determines a first dynamic delay threshold and a second dynamic delay threshold based on a baseline delay threshold, a preset dynamic adjustment coefficient, and the scenario complexity score. The average round-trip delay is compared with the first and second dynamic delay thresholds, and the vehicle control mode is determined based on the range of the average round-trip delay. This dynamic delay threshold is dynamically adjusted according to scenario complexity, ensuring that the vehicle system switches to a safe mode earlier in complex scenarios (such as heavy traffic or high-speed driving), reducing risk; and that in simple scenarios, the vehicle system can tolerate higher latency, providing longer direct control time and improving driving safety.

[0008] In some embodiments of this application, the network latency parameter includes the standard deviation of round-trip time (RTD). Controlling the vehicle according to the vehicle control mode includes: if the vehicle control mode is determined to be a direct control mode, and if the network stability is normal based on the RTD standard deviation, sending the operator's remote control command to the vehicle terminal, controlling the vehicle terminal to execute the driving task according to the remote control command; if the vehicle control mode is determined to be a predictive control mode, receiving first trajectory information sent by the vehicle and predicting second trajectory information of the vehicle, fusing the first trajectory information and the second trajectory information to obtain a predictive control command, sending the predictive control command to the vehicle, and controlling the vehicle to execute the driving task according to the predictive control command; if the vehicle control mode is determined to be a supervised autonomous mode, activating the vehicle's autonomous driving system and providing driving suggestions.

[0009] The above technical solution automatically selects direct control mode, predictive control mode or supervised autonomous mode for vehicle control based on the average round-trip time delay in different delay intervals.

[0010] In some embodiments of this application, the step of fusing the first trajectory information and the second trajectory information to obtain a predictive control command includes: determining a control weight based on the time and attenuation coefficient of the received second trajectory information; and fusing the control weight, the first trajectory information, and the second trajectory information to obtain the predictive control command.

[0011] In the above technical solution, the vehicle performs local optimal path prediction to obtain the first trajectory information, the remote control terminal performs global optimal path planning to obtain the second trajectory information, and the first trajectory information and the second trajectory information are fused and calculated to obtain predictive control commands, and the vehicle executes the calculation to obtain predictive control commands. In this way, stable control can be maintained even when the vehicle communication has a high latency.

[0012] In some embodiments of this application, after controlling the vehicle according to the vehicle control mode, the method further includes: if it is determined that the vehicle starts a smooth switching control mode based on the monitored network latency parameters, determining the current vehicle control mode, and determining a transition time window based on the vehicle speed and scene complexity score; timing after starting the smooth switching control mode, and determining whether the timing time is within the transition time window; if the timing time is within the transition time window, updating the current control weight of the predictive control mode according to a preset weight update function; receiving first trajectory information sent by the vehicle and predicting second trajectory information of the vehicle, and fusing the first trajectory information and the second trajectory information according to the updated current control weight to obtain an updated predictive control command, and controlling the vehicle to perform a driving task according to the updated predictive control command; if the timing time is not within the transition time window, determining that the vehicle switches to the predictive control mode, and controlling the vehicle to perform a driving task according to the predictive control mode.

[0013] The above technical solution determines the smooth switching control mode of the vehicle based on the monitored network latency parameters, and controls the vehicle to perform driving tasks in the smooth switching control mode. In this way, by adopting a progressive control weight adjustment strategy for the vehicle, the vehicle smoothly transitions to a low-latency control mode, thus avoiding vehicle vibration caused by sudden changes in control commands, providing passenger comfort, and reducing the safety risks caused by mode switching.

[0014] In some embodiments of this application, the network latency parameter includes the standard deviation of the round-trip latency, and controlling the vehicle according to the vehicle control mode includes: when the vehicle control mode is determined to be a direct control mode, if the stability of the network is abnormal based on the standard deviation of the round-trip latency, receiving first trajectory information sent by the vehicle and predicting second trajectory information of the vehicle; fusing the first trajectory information and the second trajectory information to obtain a predictive control command, sending the predictive control command to the vehicle, and controlling the vehicle to execute the driving task according to the predictive control command.

[0015] When the above technical solution determines that the stability of the network is abnormal based on the standard deviation of the round-trip delay, it receives the first trajectory information sent by the vehicle and the second trajectory information of the predicted vehicle; it fuses the first trajectory information and the second trajectory information to obtain a predictive control command, sends the predictive control command to the vehicle, and controls the vehicle to execute the driving task according to the predictive control command, thereby maintaining stable control of the vehicle.

[0016] In some embodiments of this application, the method further includes: monitoring the available bandwidth of the network, allocating the available bandwidth according to the available bandwidth, obtaining a data transmission strategy, and controlling the vehicle to transmit sensor data according to the data transmission strategy.

[0017] The above technical solution monitors the available bandwidth of the network, allocates the available bandwidth according to the available bandwidth, obtains a data transmission strategy, and controls the vehicle to transmit sensor data according to the data transmission strategy, thereby reducing the bandwidth requirement. Secondly, this application provides a driving control method applied to a vehicle, wherein the vehicle and a remote control terminal are connected via network communication. The method includes: acquiring vehicle-side data; determining a scene complexity score of the current environment of the vehicle based on the vehicle-side data; acquiring sensor data of the vehicle, using a model predictive control model to determine the current position, attitude, and speed of the vehicle, predicting the trajectory of the vehicle under preset constraints in each preset prediction time window to obtain first trajectory information; and sending the scene complexity score and the first trajectory information to the remote control terminal, so that the remote control terminal determines the vehicle control mode of the vehicle.

[0018] Thirdly, this application provides a remote control terminal, which includes a memory and a processor: the memory is used to store program instructions; the processor is used to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the remote control terminal performs the above-mentioned driving control method.

[0019] Fourthly, this application provides a vehicle, the vehicle including a memory and a processor: the memory is used to store program instructions; the processor is used to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the vehicle performs the above-described driving control method.

[0020] Furthermore, the technical effects brought about by the second to fourth aspects can be found in the descriptions of the methods in the above-mentioned method section, and will not be repeated here. Attached Figure Description

[0021] Figure 1 This is an application environment diagram for some embodiments of the driving control method provided in this application.

[0022] Figure 2 A flowchart illustrating a driving control method provided in some embodiments of this application.

[0023] Figure 3 This is a flowchart illustrating a method for determining a vehicle control mode and controlling a vehicle according to that mode in some embodiments of this application.

[0024] Figure 4 This is a schematic diagram of the control weight function provided in some embodiments of this application.

[0025] Figure 5 This is a timing diagram illustrating the determination of predictive control commands under predictive control mode in some embodiments of this application.

[0026] Figure 6 This is a flowchart illustrating a method for controlling a vehicle to perform driving tasks by activating a smooth switching control mode in some embodiments of this application.

[0027] Figure 7 This is a schematic diagram illustrating the transmission of sensor data provided in some embodiments of this application.

[0028] Figure 8 This is a schematic diagram illustrating the generation of semantic data provided in some embodiments of this application.

[0029] Figure 9 This is an architecture diagram of a driving control system provided in some embodiments of this application.

[0030] Figure 10 The diagram shows the structural features of a vehicle provided in some embodiments of this application.

[0031] Figure 11 This is a schematic diagram of the structure of a remote control terminal provided in some embodiments of this application. Detailed Implementation

[0032] To make the technical problems, technical solutions, and beneficial effects solved by this application clearer, the following detailed description is provided in conjunction with embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0033] With the rapid development of autonomous driving technology, autonomous vehicles have been put into commercial trial operation in many cities. To ensure driving safety, autonomous vehicles need to be equipped with remote safety operators who can remotely monitor and take over the vehicle when necessary. However, existing remote control technologies face the challenge of network latency fluctuations in practical applications. The end-to-end latency of 4G / 5G networks often fluctuates between 50-300ms. When a vehicle is traveling at 60km / h, a 100ms latency means that the vehicle has traveled approximately 1.67 meters, which could lead to serious safety hazards in emergency situations. Existing technologies lack adaptive mechanisms for network latency fluctuations and cannot guarantee the stability and safety of control under high latency conditions.

[0034] To address the aforementioned technical problems, this application provides a driving control method. (Reference) Figure 1 The diagram illustrates an application environment for a driving control method provided in some embodiments of this application. The method is applied to a vehicle 10 and a remote control terminal 20. The vehicle 10 is connected to the remote control terminal 20 via a network 30, and the remote control terminal 20 can control the driving behavior of the vehicle 10. In some embodiments of this application, the network 30 can be a 4G, 5G, or 6G network. In some embodiments of this application, the remote control terminal 20 can be, but is not limited to, devices such as computer equipment and servers.

[0035] In some embodiments of this application, vehicle 10 is able to send vehicle-side data to remote control terminal 20. Remote control terminal 20 determines the scene complexity score of the vehicle's current environment based on the received vehicle-side data, monitors the network latency parameters between remote control terminal 20 and vehicle 10, and determines the vehicle control mode based on the scene complexity score and network latency parameters. It then controls vehicle 10 according to the vehicle control mode. This solves the problem that vehicles lack an adaptive mechanism for network latency fluctuations, making it impossible to guarantee control stability and security under high latency conditions.

[0036] The following combination Figure 1 This application provides a detailed description of the driving control method. (See reference) Figure 2 The diagram shown is a flowchart of a driving control method provided in some embodiments of this application. The method is applied to... Figure 2The example method includes one or more steps, but does not constitute a limitation of this application. Furthermore, the order of the steps in the method is merely illustrative and may be changed. Additional steps may be added or steps may be removed without departing from the disclosure of this application. The method includes the following steps.

[0037] Step S201: Monitor the network latency parameters between the remote control terminal and the vehicle.

[0038] In some embodiments of this application, the remote control terminal 20 monitors network latency parameters between the remote control terminal 20 and the vehicle 10 in real time. In some real-time examples of this application, the network latency parameters include one or more of transmission latency, propagation latency, processing latency, and queuing latency. In some embodiments of this application, the network latency parameter can be characterized as round-trip time (RTT). RTT represents the total time elapsed from when the remote control terminal 20 sends a data packet to the vehicle 10 to when it receives a response data packet from the vehicle 10. In some embodiments of this application, the remote control terminal 20 can send a probe data packet to the vehicle 10 and record the sending time. After receiving the probe data packet, the vehicle 10 replies with a response packet. The remote control terminal 20 records the receiving time when it receives the response data packet. The remote control terminal 20 determines the round-trip time based on the receiving time and the sending time.

[0039] In some embodiments of this application, network latency parameters may include the mean and standard deviation of round-trip latency. In some embodiments of this application, the remote control terminal 20 may send a preset number of data packets to the vehicle 10, calculate the corresponding round-trip latency for each data packet according to the method described above, and calculate the mean and standard deviation of the round-trip latency based on the preset number of round-trip latency.

[0040] In some embodiments of this application, the remote control terminal 20 may also use an Autoregressive Integrated Moving Average (ARIMA) model or a Long Short-Term Memory (LSTM) network, and historical round-trip delays to predict the delay trend of the vehicle 10 in a future preset time period.

[0041] Step S202: Receive the scene complexity score of the current environment of the vehicle sent by the vehicle.

[0042] In some embodiments of this application, vehicle-side data can be data analyzed by the autonomous driving software in vehicle 10 based on the vehicle's sensor data. For example, the autonomous driving software in vehicle 10 may determine the number of obstacles, the number of vehicles per unit area, and the average distance between vehicles based on collected image data of the environment surrounding vehicle 10; determine the vehicle speed based on collected vehicle speed data; and determine sensor confidence and sensor data integrity based on sensor data. In some embodiments of this application, vehicle-side data includes, but is not limited to, the number of obstacles, the number of vehicles per unit area, the average distance between vehicles, vehicle speed, sensor confidence, and sensor data integrity.

[0043] In some embodiments of this application, determining the scene complexity score of the current environment of the vehicle based on vehicle-side data includes: determining the scene complexity score of the current environment of the vehicle 10 based on the number of obstacles, the number of vehicles per unit area, the average distance between vehicles, vehicle speed, sensor confidence, and sensor data integrity. In some embodiments of this application, the vehicle 10 normalizes the number of obstacles and vehicle speed; determines the traffic density score based on the number of vehicles per unit area and the average distance between vehicles; determines the sensor data quality score based on the sensor confidence and sensor data integrity; and performs a weighted summation of the normalized obstacle number, vehicle speed, traffic density score, and sensor data quality score, and determines the scene complexity score based on the weighted summation result.

[0044] In some embodiments of this application, vehicle 10 obtains a normalized number of obstacles by calculating the ratio of the number of obstacles to a preset number threshold. Vehicle 10 obtains a normalized speed by calculating the ratio of its speed to a speed threshold. In some embodiments of this application, vehicle 10 scores its traffic density based on the number of vehicles per unit area and the average distance between vehicles, resulting in a higher traffic density score for both the number of vehicles and the average distance between vehicles. In some embodiments of this application, vehicle 10 scores its sensor data quality based on sensor confidence and sensor data integrity, resulting in a higher sensor data quality score for both the higher sensor confidence and the higher the sensor data integrity. In some embodiments of this application, the traffic density score and sensor data quality score range from 0 to 1. In some embodiments of this application, a higher scene complexity score indicates a more complex scene.

[0045] In some embodiments of this application, vehicle 10 is configured according to formula C=α1 S1+α2 S2+α3 S3+α4 S4 calculates the scene complexity score, where C is the scene complexity score, S1 is the number of obstacles after normalization, S2 is the vehicle speed after normalization, S3 is the traffic density score, S4 is the sensor data quality score, and α1, α2, α3, and α4 are the corresponding weight coefficients. α1, α2, α3, and α4 can be set and adjusted according to actual needs. For example, α1 and α2 can be set to 0.3, and α3 and α4 can be set to 0.2.

[0046] Step S203: Determine the vehicle control mode based on the scene complexity score and network latency parameters, and control the vehicle according to the vehicle control mode.

[0047] In some embodiments of this application, the remote control terminal 20 obtains the control mode based on the mean and standard deviation of the round-trip delay and the scene complexity score using a decision matrix. In some embodiments of this application, the decision matrix is ​​a regularized, structured lookup table. The remote control terminal 20 uses the decision matrix to determine the control mode corresponding to the mean, standard deviation, and scene complexity score of the round-trip delay. In some embodiments of this application, the control modes include a direct control mode, a predictive control mode, and a supervised autonomous mode.

[0048] refer to Figure 3 The diagram shown is a flowchart illustrating a method for determining a vehicle control mode and controlling the vehicle according to that mode in some embodiments of this application. The method includes the following steps.

[0049] Step S301: Obtain the mean and standard deviation of round-trip time delay and the scenario complexity score.

[0050] Step S302: Determine the first dynamic delay threshold and the second dynamic delay threshold based on the baseline delay threshold, the preset dynamic adjustment coefficient, and the scene complexity score.

[0051] refer to Figure 3 The mean round-trip time delay is marked as The standard deviation of round-trip time delay is denoted as In some embodiments of this application, the remote control terminal 20 uses the formula T1=T1_base-k1 C calculates the first dynamic delay threshold, where T1 represents the first dynamic delay threshold, k1 represents the first preset dynamic adjustment coefficient, T1_base represents the first baseline delay threshold, and C is the scene complexity score. In some embodiments of this application, the remote control terminal 20 calculates the threshold according to the formula T2=T2_base-k2. C calculates the first dynamic delay threshold, where T2 represents the second dynamic delay threshold, k2 represents the second preset dynamic adjustment coefficient, and T2_base represents the second baseline delay threshold. In some embodiments of this application, the first baseline delay threshold, the second baseline delay threshold, the first preset dynamic adjustment coefficient, and the second preset dynamic adjustment coefficient can be set as needed. For example, the first baseline delay threshold can be set to 50ms, the second dynamic delay threshold can be set to 200ms, the first preset dynamic adjustment coefficient can be set to 30ms, and the second preset dynamic adjustment coefficient can be set to 50ms. Thus, when the scene complexity score is 0 (the vehicle is in a simple scene), T1 can be calculated to be 50ms and T2 to be 200ms; when the scene complexity score is 1 (the vehicle is in a complex scene), T1 can be calculated to be 20ms and T2 to be 150ms.

[0052] Step S303: Compare the average round-trip time with the first dynamic delay threshold and the second dynamic delay threshold. If the average round-trip time is less than the first dynamic delay threshold, proceed to step S304. If the average round-trip time is greater than or equal to the first dynamic delay threshold but less than the second dynamic delay threshold, proceed to step S307. If the average round-trip time is greater than or equal to the second dynamic delay threshold, proceed to step S311.

[0053] Step S304: Determine the vehicle control mode as direct control mode.

[0054] Step S305: Determine whether the network stability is abnormal based on the standard deviation of the round-trip delay.

[0055] In some embodiments of this application, if the standard deviation of the round-trip delay is less than a standard deviation threshold, the stability of network 30 is determined to be normal; if the standard deviation of the round-trip delay is greater than or equal to the standard deviation threshold, the stability of network 30 is determined to be abnormal. If the stability of network 30 is normal, step S306 is executed; if the stability of network 30 is abnormal, step S308 is executed.

[0056] Step S306: Send the operator's remote control command to the vehicle and control the vehicle to perform driving tasks according to the remote control command.

[0057] In some embodiments of this application, when the vehicle control mode is determined to be direct control mode and the network 30 is stable, the operator inputs remote control commands through the remote control terminal 20 and sends the remote control commands to the vehicle 10. The vehicle 10 executes the corresponding driving task according to the received control commands.

[0058] Step S307: Determine the vehicle control mode as predictive control mode.

[0059] Step S308: Receive the first trajectory information sent by the vehicle and the second trajectory information of the predicted vehicle, and fuse the first trajectory information and the second trajectory information to obtain the predictive control command.

[0060] In some embodiments of this application, the Model Predictive Control (MPC) model of vehicle 10 receives data from sensors such as Global Positioning System (GPS), Inertial Measurement Unit (IMU), camera, and radar to determine the current position, attitude, and speed of vehicle 10. Within each preset prediction time window, it predicts the trajectory of vehicle 10 under preset constraints to obtain first trajectory information. In some embodiments of this application, the preset prediction time window and preset constraints can be set according to actual needs. For example, preset constraints may include vehicle dynamics constraints (such as acceleration and steering angle range), road boundary constraints, and obstacle avoidance constraints. In some embodiments of this application, the prediction time window can be 2 seconds. In some embodiments of this application, the first trajectory information can be updated every first preset frequency. The first preset frequency is 10 Hz.

[0061] In some embodiments of this application, the first trajectory information includes a dynamic state vector sequence of the current attitude of the vehicle 10. The dynamic state vector sequence includes, but is not limited to, a position sequence, a velocity sequence, and an acceleration sequence. The above sequences together describe the expected motion path and motion state of the vehicle 10 in the near future.

[0062] In some embodiments of this application, the remote control terminal 20 analyzes data transmitted from sensors such as cameras and radar of the vehicle 10 and obtains scene semantic understanding results. In some embodiments of this application, the remote control terminal 20 analyzes data transmitted from sensors such as cameras and radar of the vehicle 10 and obtains scene semantic understanding results: identifying key elements, including at least one of road boundaries, lane lines, other vehicles, pedestrians, and traffic signs; determining the relative position distance and movement trend of the key elements and the vehicle 10; and obtaining semantic understanding results based on the key elements and their relative position distance and movement trend. In some embodiments of this application, the semantic understanding results include whether the lane currently occupied by the vehicle 10 is a drivable area, whether the road ahead is a curve or a straight road, and the status of traffic lights.

[0063] In some embodiments of this application, the remote control terminal 20 analyzes the vehicle's path planning for a future preset time period based on the scene semantic understanding results and the steering wheel input data, accelerator input data, and brake input data input by the remote operator, thus obtaining second trajectory information. In some embodiments of this application, the second trajectory information includes the expected position, speed, acceleration, and heading angle of the vehicle 10 at each moment within the future preset time period. In some embodiments of this application, the preset time period can be set as needed, for example, from 5 seconds to 10 seconds. In some embodiments of this application, the second trajectory information can be updated every second preset frequency. The second preset frequency is 1-2 Hz.

[0064] In some embodiments of this application, fusing the first trajectory information and the second trajectory information to obtain a predictive control command includes: fusing the first trajectory information and the second trajectory information according to a time series fusion algorithm to obtain a predictive control command. In some embodiments of this application, a control weight is determined based on the time and attenuation coefficient of the received second trajectory information; the predictive control command is obtained by fusing the control weight, the first trajectory information, and the second trajectory information. In some embodiments of this application, the predictive control command is obtained according to the formula τfinal(t)=w(t). τlocal(t)+(1-w(t)) τremote(t) is used to calculate the predictive control command, where w(t) is calculated according to the formula w(t) = exp(-λ·(t-t0)), where exp represents an exponential function with the natural constant as the base, τlocal(t) represents the first trajectory information, τremote(t) represents the second trajectory information, τfinal(t) represents the predictive control command, t represents the current time, λ represents the attenuation coefficient ranging from 0.5 to 1, and t0 represents the time when the second trajectory information is sent or received. In some embodiments of this application, w(t) is the control weight function of the predictive control mode. The control weight function w(t) indicates that as time progresses, the second trajectory information becomes outdated, and the confidence in the new locally predicted first trajectory information should be gradually increased. When t = t0, the control weight function w(t) = 1, and the locally predicted first trajectory information is fully trusted; as time increases, the control weight function w(t) gradually decreases until new second trajectory information is received.

[0065] refer to Figure 4The diagram shown illustrates the control weight function provided in some embodiments of this application. In a two-dimensional coordinate system of time and control weights, the vertical axis of the control weight function w(t) represents the control weight, ranging from 0 to 1.0, and the horizontal axis represents time t, ranging from 0 to 4 seconds. The curve characteristics of the control weight function w(t) in the two-dimensional coordinate system include: when t = 0 seconds, w(t) is 1, indicating complete trust in the first trajectory information predicted by vehicle 10; when λ = 0.5 and t = 1 second, w(t) is 0.1; and when λ = 0.5 and t = 4 seconds, w(t) is 0.14, indicating trust in the second trajectory information.

[0066] refer to Figure 5 The diagram illustrates the timing of determining predictive control commands in predictive control mode in some embodiments of this application. The second trajectory information predicted by the remote control terminal 20 includes τremote1 and τremote2. The generation time of the second trajectory information τremote1 is 0 to 1000 ms, and the preset prediction time period is 5 to 10 s. If the network latency is 200 ms, for example, the average round-trip latency of network 20 is 200 ms, the first trajectory information obtained by predicting the trajectory of vehicle 10 at 1200 ms, 1400 ms, 1600 ms, and 1800 ms are τlocal1, τlocal2, τlocal3, and τlocal4, respectively. Within 0 to 2000 ms, these are sequentially determined according to the formula τfinal(t) = w(t). τlocal(t)+(1-w(t)) τremote(t) fuses the generated second trajectory information with each first trajectory information to obtain the corresponding predictive control command, and then executes the predictive control command. 1000ms after generating the second trajectory information τremote1, the second trajectory information can be updated to generate second trajectory information τremote2. The generation time of second trajectory information τremote2 is 2000 to 3000ms. The subsequent calculation of predictive control commands using second trajectory information τremote2 is similar to the calculation of predictive control commands using second trajectory information τremote1, and will not be described in detail here.

[0067] In this embodiment, vehicle 10 performs high-frequency (e.g., every 10Hz) local optimal path prediction in the short term (e.g., 2s) to obtain first trajectory information, and remote control terminal 20 performs low-frequency (e.g., 1-2Hz) global optimal path planning in the long term (e.g., 5 to 10s) to obtain second trajectory information. The first trajectory information and the second trajectory information are fused and calculated to obtain predictive control commands, and the vehicle is made to execute the calculation to obtain predictive control commands. In this way, it can be ensured that vehicle 10 can still maintain stable control with a delay of 200ms.

[0068] Step S309: Send the predictive control command to the vehicle and control the vehicle to perform driving tasks according to the predictive control command.

[0069] In some embodiments of this application, the vehicle 10 controls the steering, throttle, or braking of the vehicle 10 according to predictive control commands. Step S310: Continuously monitor the network latency parameters between the remote control terminal and the vehicle.

[0070] Step S311: Determine the vehicle control mode as supervised autonomous mode.

[0071] Step S312: Activate the vehicle's autonomous driving system and provide driving suggestions.

[0072] In some embodiments of this application, sensors such as LiDAR, cameras, and millimeter-wave radar can be used to collect information in real time, including vehicle position, the status of surrounding vehicles / obstacles, lane lines, and traffic signs. This information is then fused with high-precision map data. Based on the fused data, a preset algorithm model (such as a rule engine or machine learning model) is used to evaluate the safety, efficiency, and compliance of the current driving scenario, generating potential driving strategies. These strategies are then translated into driving suggestions that the user can understand and are displayed via voice, dashboard icons, or text / animation prompts on the central control screen. For example, the vehicle's central control screen may display "It is recommended to change lanes to the left to maintain the optimal speed," while simultaneously providing a voice prompt: "A slow vehicle has been detected ahead; it is recommended to change lanes to the left lane." Another example is the voice prompt: "It is recommended to stop."

[0073] Step S313: Determine whether the vehicle should start the smooth switching control mode based on the monitored network latency parameters.

[0074] In some embodiments of this application, if it is determined, based on monitored network latency parameters, that vehicle 10 needs to switch from the current vehicle control mode to predictive control mode, a smooth switching control mode is initiated; if it is determined, vehicle 10 does not need to switch from the current vehicle control mode to predictive control mode, the smooth switching control mode is not initiated. For example, if the current vehicle control mode of vehicle 10 is direct control mode, and the average round-trip latency is detected to change from a range less than a first dynamic latency threshold to a range greater than or equal to the first dynamic latency threshold but less than a second dynamic latency threshold, it is determined that vehicle 10 needs to switch from direct control mode to predictive control mode, and the smooth switching control mode is initiated. For example, if the current vehicle control mode of vehicle 10 is supervised autonomous mode, and the average round-trip latency is detected to change from a range greater than a second dynamic latency threshold to a range greater than or equal to the first dynamic latency threshold but less than the second dynamic latency threshold, it is determined that vehicle 10 needs to switch from supervised autonomous mode to predictive control mode, and the smooth switching control mode is initiated.

[0075] In some embodiments of this application, if it is determined, based on the monitored network latency parameters, that vehicle 10 needs to switch from the current predictive control mode to the direct control mode or the supervised autonomous mode, the smooth switching control mode is activated; if vehicle 10 does not need to switch from the current predictive control mode to the direct control mode or the supervised autonomous mode, the smooth switching control mode is not activated.

[0076] In some embodiments of this application, if it is determined that the smooth switching control mode will not be started, step S314 is executed; if it is determined that the smooth switching control mode will not be started, step S315 is executed.

[0077] Step S314: Control the vehicle to perform driving tasks according to the vehicle's current vehicle control mode.

[0078] Step S315: Activate the smooth switching control mode to control the vehicle to perform driving tasks.

[0079] refer to Figure 6 The diagram shown is a flowchart of a method for initiating a smooth switching control mode to control a vehicle to perform driving tasks in some embodiments of this application. The method is applied to a remote control terminal 20. The method includes the following steps.

[0080] Step S601: Determine the current vehicle control mode and determine the transition time window based on the vehicle speed and scene complexity score.

[0081] In some embodiments of this application, the remote control terminal 20 determines the transition time window based on the vehicle speed and scene complexity score. In some embodiments of this application, the remote control terminal 20 calculates the transition time window using the formula Ttrans = Ttrans_base × (1 + 0.5 × C) × (V / Vmax), where Ttrans represents the transition time window, Ttrans_base represents the initial value of the transition time window (which can be a fixed value, such as 2 seconds), C represents the scene complexity score, V represents the vehicle speed, and Vmax represents the maximum speed of the vehicle 20. In some embodiments of this application, the transition time window ranges from 1 second to 3 seconds.

[0082] Step S602: Determine the current control weights of the vehicle in predictive control mode.

[0083] Step S603: After starting the smooth switching control mode, timing is performed to determine whether the timing time is within the transition time window. In some embodiments of this application, if the timing time is within the transition time window, step S604 is executed; if the timing time exceeds the transition time window, step S609 is executed.

[0084] Step S604: Update the current control weights of the predictive control mode according to the preset weight update function.

[0085] In some embodiments of this application, the preset weight update function is a linear transition function. The remote control terminal 20 updates the control weight of the predictive control mode according to the formula w_new(t)=w_old+(w_target-w_old)×(t / Ttrans), where w_new(t) represents the control weight, w_old is the current control weight, t is the time, w_target is the target control weight, the target control weight is a fixed value that can be set according to actual needs, for example, 0.5, and Ttrans represents the transition time window.

[0086] In some embodiments of this application, the preset weight update function is a nonlinear transition function. The remote control terminal 20 updates the control weight of the predictive control mode according to the formula w_new(t)=w_old+(w_target-w_old)×(1 / (1+exp(-k×(t-Ttrans / 2)))), where w_new(t) represents the control weight, w_old is the current control weight, t is time, w_target is the target control weight, and Ttrans represents the transition time window.

[0087] Step S605: Receive the first trajectory information and the predicted second trajectory information sent by the vehicle, and perform fusion calculation on the first trajectory information and the second trajectory information according to the updated current control weights to obtain the updated predictive control command, and control the vehicle to perform driving tasks according to the updated predictive control command.

[0088] In some embodiments of this application, the specific implementation of fusing the first trajectory information and the second trajectory information to obtain the predictive control command according to the control weight can be referred to the implementation of step S308, and will not be described similarly here.

[0089] Step S606: Monitor the vehicle's dynamic response indicators.

[0090] In some embodiments of this application, the vehicle's dynamic response indicators include, but are not limited to, one or more of lateral acceleration, longitudinal acceleration, and steering wheel angle change rate.

[0091] Step S607: Determine whether the vehicle is in a safe driving state based on the dynamic response index.

[0092] This application uses lateral acceleration and longitudinal acceleration as examples of dynamic response indicators for illustration. In some embodiments of this application, if the lateral acceleration of vehicle 10 is greater than a first acceleration threshold and the longitudinal acceleration of vehicle 10 is greater than a second acceleration threshold, it is determined that the vehicle is not in a safe driving state; if the lateral acceleration of vehicle 10 is less than or equal to the first acceleration threshold, or the longitudinal acceleration of vehicle 10 is less than or equal to the second acceleration threshold, or the lateral acceleration of vehicle 10 is less than or equal to the first acceleration threshold and the longitudinal acceleration of vehicle 10 is less than or equal to the second acceleration threshold, it is determined that the vehicle is in a safe driving state. In some embodiments of this application, if the vehicle is not in a safe driving state, step S608 is executed; if the vehicle is in a safe driving state, step S603 is executed.

[0093] Step S608: Extend the transition time window or control the vehicle to perform a driving task according to the current vehicle control mode. After step S608 is completed, proceed to step S603.

[0094] Step S609: Determine that the vehicle has switched to predictive control mode, and control the vehicle to perform driving tasks according to the predictive control mode.

[0095] The specific implementation details of controlling the vehicle to perform driving tasks according to the predictive control mode in this embodiment can be found in step S308, and will not be described similarly here.

[0096] In this embodiment, when it is determined that the vehicle switches from direct control mode or supervised autonomous mode to predictive control mode, or from predictive control mode to direct control mode or supervised autonomous mode, a smooth switching control mode is activated to control the vehicle to perform driving tasks. In this way, by adopting a progressive control weight adjustment strategy for the vehicle 10, the vehicle smoothly transitions to a low-latency control mode, thus avoiding vehicle vibration caused by sudden changes in control commands, providing passenger comfort, and reducing safety risks caused by mode switching.

[0097] The method described in this application can adjust the first dynamic delay threshold and the second dynamic delay threshold according to the scenario complexity score, and determine the vehicle control mode based on the adjusted dynamic delay threshold. This allows the vehicle system to switch to the safety mode earlier in complex scenarios (such as dense traffic or high-speed driving) to reduce risks. In simple scenarios, the vehicle system can tolerate higher latency and provide a longer direct control time.

[0098] In some embodiments of this application, the method further includes: during the process of the vehicle 10 transmitting sensor data collected by the vehicle 10 to the remote control terminal 20, the remote control terminal 20 monitors the available bandwidth of the network 30, allocates the transmission bandwidth of the network 30 according to the available bandwidth, obtains a data transmission strategy, and controls the vehicle 10 to transmit the sensor data according to the data transmission strategy. (See reference...) Figure 7 The diagram shown illustrates the transmission of sensor data according to some embodiments of this application. In some embodiments of this application, the sensor data collected by vehicle 10 includes camera data. In some embodiments of this application, depending on the position of the cameras installed on vehicle 10, the cameras of vehicle 10 include a front camera, a left-side camera, a right-side camera, and a rear camera. The sensor data collected by vehicle 10 includes front camera data collected by the front camera, left-side camera data collected by the left-side camera, right-side camera data collected by the right-side camera, and rear camera data collected by the rear camera.

[0099] In some embodiments of this application, vehicle 10 dynamically adjusts the compression rate of sensor data encoding based on real-time feedback of current network quality (such as latency and jitter), compresses the sensor data according to the adjusted compression rate, and sends the compressed sensor data to remote control terminal 20 via network 30. Remote control terminal 20 monitors network transmission indicators. These indicators include, but are not limited to, at least one of network latency parameters, bandwidth utilization, packet loss rate, and video stuttering. Remote control terminal 20 measures the available bandwidth based on these network transmission indicators. Remote control terminal 20 determines whether the data transmission strategy for sensor data needs adjustment based on bandwidth utilization, packet loss rate, and video stuttering. For example, if the packet loss rate is greater than a preset packet loss rate threshold, or video stuttering occurs, it is determined that the data transmission strategy needs adjustment; if the packet loss rate is less than the preset packet loss rate threshold, or no video stuttering occurs, it is determined that the data transmission strategy does not need adjustment, and vehicle 10 transmits data according to the current data transmission strategy. If it is determined that the data transmission strategy needs adjustment, remote control terminal 20 determines the scene requirement information based on the importance of the video data. In some embodiments of this application, determining the scene requirement information includes: assigning a first weight value to the bandwidth required for vehicle 10 to transmit front camera data; assigning a second weight value to the bandwidth required to transmit left-side and right-side camera data; and assigning a third weight value to the bandwidth required to transmit rear camera data. In some embodiments of this application, the first weight value is greater than the second weight value, and the second weight value is greater than the third weight value. The first, second, and third weight values ​​can be set according to actual needs. In some embodiments of this application, if there are obstacles in the corresponding camera data, the bandwidth weight of the corresponding camera data is doubled.

[0100] The remote control terminal 20 determines the bandwidth range of the available bandwidth. If the available bandwidth is within the first bandwidth range, a high-bandwidth data transmission strategy is determined based on the scenario requirements and the available bandwidth. If the available bandwidth is within the second bandwidth range, a medium-bandwidth data transmission strategy is determined based on the scenario requirements and the available bandwidth. If the available bandwidth is within the third bandwidth range, a low-bandwidth data transmission strategy is determined based on the scenario requirements and the available bandwidth. In some embodiments of this application, the first bandwidth range, the second bandwidth range, and the third bandwidth range can be set according to actual needs. For example, the first bandwidth range can be set to a range greater than 10 Mbps, the second bandwidth range can be set to a range of 5 Mbps to 10 Mbps, and the third bandwidth range can be set to a range less than 5 Mbps.

[0101] In some embodiments of this application, the high-bandwidth data transmission strategy can be: transmitting front camera data at a resolution of 1080p and a bitrate of 4Mbps; transmitting left and right side camera data at a resolution of 720p and a bitrate of 2Mbps; transmitting rear camera data at a resolution of 720p and a bitrate of 1.5Mbps; instructing the vehicle to generate semantic data from sensor data and transmit the semantic data at 200kb / frame. The medium-bandwidth data transmission strategy can be: transmitting front camera data at a resolution of 720p and a bitrate of 2Mbps; transmitting left and right side camera data at a resolution of 480p and a bitrate of 0.8Mbps; transmitting rear camera data at a resolution of 480p and a bitrate of 0.5Mbps; instructing the vehicle to generate semantic data from sensor data and transmit the semantic data at 150kb / frame. The low-bandwidth data transmission strategy can be: transmitting front camera data at a resolution of 480p and a bitrate of 1Mbps; transmitting only keyframes of other camera data; instructing the vehicle to generate semantic data from sensor data and transmit the semantic data at 100kb / frame.

[0102] In some embodiments of this application, the vehicle 10 generates semantic data from sensor data by: extracting the bounding boxes of target objects in the sensor data using a preset detection model; identifying obstacles in the sensor data and predicting the movement trajectories of the obstacles; projecting the 3D point cloud data of the target objects into a bird's-eye view; and planning the vehicle's path. In some embodiments of this application, after generating semantic data, the vehicle 10 can encode the semantic data and send it to the remote control terminal 20.

[0103] refer to Figure 8The diagram illustrates the generation of semantic data according to some embodiments of this application. Vehicle 10 inputs sensor data through the data input layer of the semantic generation model. The semantic processing layer of the semantic generation model processes the input data and outputs the processed semantic data through the data output layer. The output data can be used for data benchmarking, determining network transmission parameters, and remote end reconstruction. In some embodiments of this application, the sensor data includes image data collected by six cameras, 64-line point cloud data collected by LiDAR, target information collected by millimeter-wave radar, and attitude information collected by GPS and IMU. The semantic processing layer uses a preset detection model to extract the bounding boxes of the target objects in the sensor data to obtain target detection results; identifies obstacles in the sensor data and predicts the movement trajectory of the obstacles; projects the 3D point cloud data of the target objects into a bird's-eye view; and plans the vehicle's path. The output layer outputs the target detection results, movement trajectory, bird's-eye view, and vehicle path.

[0104] In this embodiment, the raw data is converted into semantic information, and the bitrate of each camera data is dynamically allocated according to the importance of the scene requirement information. When bandwidth is limited, the integrity of semantic information is given priority. This reduces the bandwidth requirement for transmitting sensor data from 20Mbps to 2-5Mbps (a reduction of 80%), enabling the system to operate under poor network conditions and expanding the application scope.

[0105] In some embodiments of this application, the method further includes: monitoring the performance indicators of the remote control terminal 20, and adjusting the first dynamic delay threshold, the second dynamic delay threshold, and the attenuation coefficient according to the performance indicators and the target value of the performance indicators.

[0106] In some embodiments of this application, the performance indicators of the remote control terminal 20 include at least one of control latency, trajectory tracking error, acceleration change rate, manual takeover frequency, and mode switching frequency. Control latency represents the end-to-end time from operator input on the remote control terminal 20 to response from the vehicle 10, with a target value of less than 300ms. Trajectory tracking error represents the deviation between the actual trajectory of the vehicle 10 and the trajectory planned by the remote control terminal 20, with a target value of less than 0.3m for lateral error and less than 0.5m for longitudinal error. Acceleration change rate represents the rate of acceleration change of the vehicle 10 during its movement, with a target value of less than 2m / s³ for lateral acceleration change. Manual takeover frequency represents the number of times the operator takes over control of the vehicle 10 per unit time, with a target value of less than 1 time / hour. Mode switching frequency represents the frequency at which the vehicle 10 switches vehicle control modes per unit time, with a target value of <3 times / 10 minutes.

[0107] In some embodiments of this application, the remote control terminal 20 adjusts the first dynamic delay threshold, the second dynamic delay threshold, and the attenuation coefficient based on control delay, trajectory tracking error, acceleration change rate, manual takeover frequency, mode switching number, and corresponding target values ​​using reinforcement learning or online learning methods.

[0108] refer to Figure 9 The diagram shown is an architecture diagram of a driving control system provided in some embodiments of this application. The driving control system 100 includes a vehicle-side module 30, a remote module 40, a communication module 50, and a decision-making and scheduling module 60. The vehicle-side module 30 is disposed in the vehicle 10 and includes an onboard communication unit 301, a multi-sensor fusion unit 302, a complexity assessment unit 303, a local trajectory prediction unit 304, and a control execution unit 305.

[0109] In some embodiments of this application, the remote module 40 and the decision-making and scheduling module 60 are disposed in the remote control module 20. The remote module 40 includes an operation input unit 401, a path planning unit 402, a display unit 403, and a monitoring and management unit 404. The communication module 50 includes a semantic generation unit 501, an encoding unit 502, a network channel 503, and a latency detection unit 504. The semantic generation unit 501 and the encoding unit 502 are disposed in the vehicle 10. The network channel 503 is disposed between the vehicle 10 and the remote control terminal 20. The latency detection unit 504 is disposed in the remote control terminal 20. The decision-making and scheduling module 60 includes a control mode decision unit 601, a trajectory fusion unit 602, a mode switching management unit 603, and a performance monitoring unit 604.

[0110] In some embodiments of this application, vehicle 10 is communicatively connected to remote control terminal 20 via vehicle communication unit 301. Multi-sensor fusion unit 302 is used to collect sensor data, such as at least one of camera data, radar point cloud data, and GPS positioning data. Complexity assessment unit 303 determines a scene complexity score based on the sensor data collected by multi-sensor fusion unit 302. Local trajectory prediction unit 304 uses the sensor data to determine first trajectory information.

[0111] The vehicle communication unit 301 sends sensor data to the semantic generation unit 501. The semantic generation unit 501 generates semantic data based on the sensor data. The encoding unit 502 encodes the semantic data, scene complexity score, and first trajectory information, and transmits the encoded data to the remote control terminal 20 via the network channel 503. The delay detection unit 504 monitors the network delay parameters between the remote control terminal and the vehicle and sends the network delay parameters to the control mode decision unit 601, wherein the network delay parameters include the mean and standard deviation of the round-trip delay. The control mode decision unit 601 obtains the scene complexity score from the network channel 503.

[0112] The operation input unit 401 is used to acquire input information from the operator at the remote control terminal 20. For example, the input information may include at least one of steering wheel input data, accelerator input data, and brake input data. The path planning unit 402 analyzes the path planning of the vehicle for a future preset time period based on at least one of the steering wheel input data, accelerator input data, and brake input data, obtains second trajectory information, and transmits the second trajectory information to the control mode decision unit 601 through the network channel 503.

[0113] The control mode decision unit 601 determines the vehicle control mode based on scene complexity scoring and network latency parameters. The trajectory fusion unit 602 obtains first trajectory information from the network channel 503 and second trajectory information from the path planning unit 402. When the control mode decision unit 601 determines that the vehicle 10 is in predictive control mode, the trajectory fusion unit 602 mechanically fuses the first and second trajectory information to generate predictive control commands and sends the predictive control commands to the vehicle 10. The control execution unit 305 of the vehicle 10 controls the vehicle 10 according to the predictive control commands, such as controlling the vehicle's steering, throttle, or brakes. The mode switching management unit 603 sends a mode switching command to the vehicle 10 when the control mode decision unit 601 determines that the vehicle 10 needs to switch modes. The control execution unit 305 of the vehicle 10 performs mode switching according to the mode switching command. After the control mode decision unit 601 sends predictive control commands to control the vehicle 10, the performance detection unit 604 monitors the performance indicators of the remote control terminal 20. The monitoring and management unit 404 adjusts the first dynamic delay threshold, the second dynamic delay threshold, and the attenuation coefficient based on the performance indicators monitored by the performance detection unit 604 and the target values ​​of the performance indicators. In some embodiments of this application, the display unit 403 is also used to display sensor data or semantic data obtained from the network channel 503 for presentation to the operator.

[0114] It should be noted that the driving control method provided in the above embodiments is only an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above. For a detailed implementation of the above driving control system 100, please refer to... Figure 2-8 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0115] The functional units and modules in the above embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of the embodiments of this application.

[0116] refer to Figure 10 The diagram shown is a structural schematic of a vehicle provided in some embodiments of this application. In one embodiment of this application, the vehicle 600 includes, but is not limited to, a storage device 601, a processing device 602, and a computer program, such as a driving control program, stored in the storage device 601 and executable on the processing device 602.

[0117] In one embodiment of this application, the vehicle 600 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored computer-readable instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0118] The vehicle 600 may include network devices and / or user equipment. The network devices include, but are not limited to, a single network electronic device, a group of electronic devices consisting of multiple network electronic devices, or a cloud based on cloud computing consisting of a large number of hosts or network electronic devices.

[0119] The network in which the vehicle 600 is located includes, but is not limited to: the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0120] Those skilled in the art will understand that the schematic diagram is merely an example of vehicle 600 and does not constitute a limitation on vehicle 600. It may include more or fewer components than shown, or combine certain components, or different components. For example, vehicle 600 may also include input / output devices, network access devices, buses, etc.

[0121] The processing device 602 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processing device 602 is the computing core and control center of the vehicle 600, connecting various parts of the vehicle 600 through various interfaces and lines, and acquiring the vehicle 600's operating system and installed applications, program code, etc.

[0122] The processing device 602 acquires the operating system and various installed applications of the vehicle 600. The processing device 602 acquires the applications to implement the steps in the above-described driving control method embodiments.

[0123] For example, the computer program may be divided into one or more modules / units, which are stored in the storage device 601 and retrieved by the processing device 602 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the retrieval process of the computer program in the vehicle 600.

[0124] The storage device 601 can be used to store the computer programs and / or modules. The processing device 602 implements various functions of the vehicle 600 by running or retrieving the computer programs and / or modules stored in the storage device 601 and by calling the data stored in the storage device 601. The storage device 601 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the vehicle 600, etc. In addition, the storage device 601 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0125] The storage device 601 can be an external storage device and / or an internal storage device of the vehicle 600. Further, the storage device 601 can be a physical storage device, such as a memory module, a TF card (Trans-flash Card), etc.

[0126] If the modules / units integrated in the vehicle 600 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when the computer program is acquired by a processor, it can implement the steps of the various method embodiments described above.

[0127] The computer program includes computer program code, which may be in the form of source code, object code, accessible file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, and read-only memory (ROM).

[0128] Specifically, the specific implementation method of the above instructions by the processing device 602 can be found in [reference needed]. Figure 2 , Figure 3 , Figure 6 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0129] refer to Figure 11The diagram shown is a structural schematic of a remote control terminal provided in some embodiments of this application. In one embodiment of this application, the remote control terminal 700 includes, but is not limited to, a storage device 701, a processing device 702, and a computer program, such as a driving control program, stored in the storage device 701 and executable on the processing device 702.

[0130] In one embodiment of this application, the remote control terminal 700 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored computer-readable instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits, programmable gate arrays, digital signal processors, embedded devices, etc.

[0131] The remote control terminal 700 may include network devices and / or user devices. The network devices include, but are not limited to, a single network electronic device, a group of multiple network electronic devices, or a cloud based on cloud computing consisting of a large number of hosts or network electronic devices.

[0132] The network where the remote control terminal 700 is located includes, but is not limited to: the Internet, wide area network, metropolitan area network, local area network, virtual private network, etc.

[0133] Those skilled in the art will understand that the schematic diagram is merely an example of the remote control terminal 700 and does not constitute a limitation on the remote control terminal 700. It may include more or fewer components than shown in the diagram, or combine certain components, or different components. For example, the remote control terminal 700 may also include input / output devices, network access devices, buses, etc.

[0134] The processing device 702 can be a central computing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processing device 702 is the computing core and control center of the remote control terminal 700, connecting various parts of the remote control terminal 700 via various interfaces and lines, and acquiring the operating system of the remote control terminal 700, as well as various installed applications and program code.

[0135] The processing device 702 acquires the operating system and various installed applications of the remote control terminal 700. The processing device 702 acquires the applications to implement the steps in the above-described driving control method embodiment.

[0136] For example, the computer program may be divided into one or more modules / units, which are stored in the storage device 701 and retrieved by the processing device 702 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the process of retrieving the computer program from the remote control terminal 700.

[0137] The storage device 701 can be used to store the computer program and / or module. The processing device 702 implements various functions of the remote control terminal 700 by running or acquiring the computer program and / or module stored in the storage device 701 and calling the data stored in the storage device 701.

[0138] If the modules / units integrated in the remote control terminal 700 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when the computer program is acquired by a processor, it can implement the steps of the various method embodiments described above.

[0139] The computer program includes computer program code, which may be in the form of source code, object code, accessible file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, or read-only memory.

[0140] Specifically, the specific implementation method of the above instructions by the processing device 702 can be found in [reference needed]. Figure 2 , Figure 3 , Figure 6 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0142] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0144] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0145] In this application, "multiple" refers to two or more.

[0146] In this application, unless otherwise expressly defined, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0147] The terms “first,” “second,” “third,” “fourth,” etc., in this application (if present) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0148] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, in this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0149] Unless otherwise specified, all steps in this application may be performed sequentially or randomly. For example, if the method includes steps A and B, it means that the method may include steps A and B performed sequentially, or it may include steps B and A performed sequentially. For example, if the method may also include step C, it means that step C may be added to the method in any order. For example, the method may include steps A, B, and C, or it may include steps A, C, and B, or it may include steps C, A, and B, etc.

[0150] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A driving control method, applied to a remote control terminal, wherein the remote control terminal is connected to the vehicle via network communication, characterized in that, The method includes: Monitor the network latency parameters between the remote control terminal and the vehicle; Receive the scene complexity score of the current environment of the vehicle sent by the vehicle; The vehicle control mode is determined based on the scenario complexity score and the network latency parameters, and the vehicle is controlled according to the vehicle control mode.

2. The driving control method as described in claim 1, characterized in that, The network latency parameter includes the average round-trip latency, and determining the vehicle control mode based on the scenario complexity score and the network latency parameter includes: The first dynamic delay threshold and the second dynamic delay threshold are determined based on the baseline delay threshold, the preset dynamic adjustment coefficient, and the scenario complexity score; The average round-trip time is compared with the first dynamic delay threshold and the second dynamic delay threshold; If the average round-trip delay is less than the first dynamic delay threshold, the vehicle control mode is determined to be the direct control mode. If the average round-trip delay is greater than or equal to the first dynamic delay threshold and less than the second dynamic delay threshold, the vehicle control mode is determined to be a predictive control mode. If the average round-trip time delay is greater than or equal to the second dynamic delay threshold, the vehicle control mode is determined to be a supervised autonomous mode.

3. The driving control method as described in claim 2, characterized in that, The network latency parameter includes the standard deviation of round-trip latency, and controlling the vehicle according to the vehicle control mode includes: If the vehicle control mode is determined to be direct control mode, and the network stability is normal based on the standard deviation of the round-trip delay, the operator's remote control command is sent to the vehicle terminal, and the vehicle terminal is controlled to execute the driving task according to the remote control command. If the vehicle control mode is determined to be a predictive control mode, the system receives the first trajectory information sent by the vehicle and the second trajectory information of the predicted vehicle, and performs a fusion calculation on the first trajectory information and the second trajectory information to obtain a predictive control command. The predictive control command is then sent to the vehicle to control the vehicle to execute the driving task according to the predictive control command. If the vehicle control mode is determined to be the supervised autonomous mode, the autonomous driving system of the vehicle is activated, and driving suggestions are provided.

4. The driving control method as described in claim 3, characterized in that, The step of fusing the first trajectory information and the second trajectory information to obtain predictive control commands includes: The control weights are determined based on the time and attenuation coefficient of the received second trajectory information; The predictive control command is obtained by fusing the control weights, the first trajectory information, and the second trajectory information.

5. The driving control method as described in claim 4, characterized in that, After controlling the vehicle according to the vehicle control mode, the method further includes: If the smooth switching control mode for vehicle startup is determined based on the monitored network latency parameters, the current vehicle control mode is determined, and the transition time window is determined based on the vehicle speed and scene complexity score. After starting the smooth switching control mode, a timer is started to determine whether the timing time is within the transition time window. If the timing time is within the transition time window, the current control weight of the predictive control mode is updated according to the preset weight update function; The system receives first trajectory information sent by the vehicle and predicts second trajectory information of the vehicle. It then fuses and calculates the first trajectory information and the second trajectory information according to the updated current control weights to obtain an updated predictive control command. Finally, it controls the vehicle to perform driving tasks according to the updated predictive control command. If the timing period is not within the transition time window, the vehicle is determined to switch to predictive control mode, and the vehicle is controlled to perform driving tasks according to the predictive control mode.

6. The driving control method as described in claim 2, characterized in that, The network latency parameter includes the standard deviation of the return latency, and controlling the vehicle according to the vehicle control mode includes: If the vehicle control mode is determined to be direct control mode, and the network stability is determined to be abnormal based on the standard deviation of the round-trip delay, the first trajectory information sent by the vehicle and the second trajectory information of the predicted vehicle are received. The first trajectory information and the second trajectory information are fused and calculated to obtain a predictive control command. The predictive control command is then sent to the vehicle to control the vehicle to execute the driving task according to the predictive control command.

7. The driving control method as described in claim 1, characterized in that, The method further includes: The available bandwidth of the network is monitored, and the transmission bandwidth of the network is allocated according to the available bandwidth to obtain a data transmission strategy. The vehicle is then controlled to transmit sensor data according to the data transmission strategy.

8. A driving control method applied to a vehicle, wherein the vehicle is connected to a remote control terminal via network communication, characterized in that, The method includes: Acquire vehicle-side data; The scene complexity score of the current environment of the vehicle is determined based on the vehicle-side data; The vehicle's sensor data is acquired, and the current position, attitude, and speed of the vehicle are determined using a model predictive control model. In each preset prediction time window, the trajectory of the vehicle is predicted under preset constraints to obtain the first trajectory information. The scenario complexity score and the first trajectory information are sent to the remote control terminal, so that the remote control terminal can determine the vehicle control mode of the vehicle.

9. A remote control terminal, characterized in that, The remote control terminal includes a memory and a processor: The memory is used to store program instructions; The processor is configured to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the remote control terminal performs the driving control method as described in any one of claims 1 to 7.

10. A vehicle, characterized in that, The vehicle includes a memory and a processor: The memory is used to store program instructions; The processor is configured to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, cause the vehicle to perform the driving control method as described in claim 8.

Images