A vehicle remote interpolation fitting smooth control method and device

By discarding some control point data at the remote control end and interpolating it back at the vehicle end, the problems of network latency and jitter in remote driving are solved, realizing the real-time and precision requirements of remote driving of commercial vehicles and improving the smoothness and fluency of operation.

CN113934196BActive Publication Date: 2024-11-26ZHONGRUI INTELLIGENT TRANSPORTATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111164028.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-30
Publication Date
2024-11-26
Estimated Expiration
2041-09-30

AI Technical Summary

Technical Problem

In existing technologies for remote driving, network communication latency and jitter lead to packet loss of control commands, which cannot meet the real-time and accuracy requirements of remote driving and control of commercial vehicles, and affects the smoothness and continuity of operation.

Method used

The remote control discards some control point data and recovers it through interpolation. The vehicle generates a command sequence and uses interpolation fitting technology to achieve real-time transmission and smoothness of control commands under network jitter. By using packet loss strategy and interpolation to recover control point data, a command sequence for controlling the actuator is generated.

Benefits of technology

It improves the robustness and adaptability of remote control operation, retains operational precision, and enhances the smoothness and fluidity of driving operation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113934196B_ABST
    Figure CN113934196B_ABST
Patent Text Reader

Abstract

The present application belongs to the field of vehicle remote control, and in particular, relates to a method and device for smooth control of remote interpolation fitting of a vehicle. The method includes: the remote control end obtains a control command of a control device, discards part of the control point data of the control command, obtains a control command for discarding part of the control point data, and sends the control command for discarding part of the control point data to the vehicle end; after the vehicle end receives the control command for discarding part of the control point data, it restores the control point data of the control command by interpolation, and generates a command sequence for controlling the execution device. The vehicle remote interpolation fitting smooth control method of the present application improves the robustness and adaptability to the jitter of the network that carries the remote control driving operation; retains the operating accuracy attribute of the original control device, and can improve the accuracy of the remote control operation; improves the smoothness of the remote driving operation effect, and can improve the smooth remote control experience of the driving operator.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of vehicle remote control, and in particular, relates to a method and device for smooth control of vehicle remote interpolation fitting. Background Art

[0002] With the development of autonomous driving and 5G, commercial vehicles, due to their specific scenarios, do not face many constraints such as regulations and can be put into use more quickly. However, considering product experience and safe operation, remote operation driving as a necessary supplement to vehicle operation is a shortcut to realize the unmanned operation of commercial vehicles. In addition to the basic lateral and longitudinal walking control needs, commercial vehicles may also face the control of other business operation actions of the vehicle, such as harvesting by harvesters and digging by excavators. In remote driving, it is necessary to build a communication networking solution that meets the needs of the scenario, carrying the real-time transmission and reception of remote operation commands and possible information transmission of monitoring videos.

[0003] In actual applications, according to the needs of the scene, there are long-distance remote control, generally through 5G / 4G wide area communication; short-distance remote control, generally through Wifi or 1.4G / 2.4G ISM local area communication. Video function needs to be added beyond the line of sight. In actual communication methods, 5G, Wifi, 1.4G and other communications generally have an average end-to-end transmission delay of about 30ms. In actual applications, the issuance cycle of analog single control device commands (such as steering wheels, throttles, handles, etc.) is generally around 10ms. If there are more complex actions, the average command cycle will be shorter. In other words, in remote driving scenarios, the general command acquisition cycle is less than the network communication delay, and the communication channel cannot meet the real-time transmission of each control command, and it will cause channel congestion and packet loss.

[0004] In engineering implementation, the usual practice is to reduce the signal acquisition frequency or increase the acquisition cycle so that the acquisition cycle is greater than the average communication delay. For example, if the acquisition is adjusted every 10ms to 100ms. For analog quantities with low switching and operation accuracy requirements, it can solve practical problems in certain scenarios. As the goal of ensuring the safety of unmanned driving operations and effective control of manual intervention, the existing control schemes cannot meet the needs of remote driving and control of commercial vehicles. The main defects are as follows: increasing the overall network delay, especially when video surveillance is required beyond the line of sight, the driver will feel a sense of operation delay; reducing the remote operation accuracy of vehicles traveling at high speeds or vehicles with operating devices; reducing the smoothness and continuity of operating actions.

[0005] Therefore, it is desired to have a technical solution to overcome or at least alleviate at least one of the above-mentioned defects of the prior art. Summary of the invention

[0006] The purpose of the present application is to provide a method and device for smooth control of remote interpolation fitting of a vehicle to solve at least one problem existing in the prior art.

[0007] The technical solution of this application is:

[0008] A first aspect of the present application provides a vehicle remote interpolation fitting smooth control method, comprising:

[0009] The remote control end obtains the control command of the control device, discards part of the control point data of the control command, obtains the control command discarding part of the control point data, and sends the control command discarding part of the control point data to the vehicle end;

[0010] After receiving the control command that discards part of the control point data, the vehicle end restores the control point data of the control command through interpolation and generates a command sequence for controlling the execution device.

[0011] In at least one embodiment of the present application, the remote control end obtains a control command of the control device, discards part of the control point data of the control command, obtains a control command to discard part of the control point data, and sends the control command to discard part of the control point data to the vehicle end, including:

[0012] The remote control terminal obtains control commands of different control devices in real time and stores them in a classified queue;

[0013] Record and monitor the end-to-end transmission delay, send it through a separate socket link, save and dynamically update the average delay value, and calculate the packet loss strategy integer operator μ based on the average delay value and the sampling period of the control device, where: T is the average delay value, t is the sampling period of the control device, Indicates rounding up to an integer;

[0014] Read the control command of the classification queue, and discard some control point data according to the packet loss strategy integer operator μ, specifically, discard μ-1 from every μ control commands of the classification queue. If μ=1, no discard is required;

[0015] The control commands that discard some control point data are classified and packaged, and sent to the vehicle end through the socket link.

[0016] In at least one embodiment of the present application, the control commands of the control device include real-time operation instructions of the steering wheel, real-time operation instructions of the brake, and real-time operation instructions of the joystick.

[0017] In at least one embodiment of the present application, after the vehicle receives a control command that discards part of the control point data, restoring the control point data of the control command by interpolation, and generating a command sequence for controlling the execution device includes:

[0018] The vehicle side parses the received control command data packet, wherein the control command data packet includes a control command ID and a control command value;

[0019] Obtain the control curve and fitting function of the corresponding control device according to the control command ID;

[0020] Fill the control points discarded by interpolation between the control commands according to the fitting function, and generate a command sequence;

[0021] The generated command sequence is sent to the execution device.

[0022] In at least one embodiment of the present application, the control command and the command sequence are both digital signals, or the control command and the command sequence are both analog signals.

[0023] A second aspect of the present application provides a vehicle remote interpolation fitting smooth control device, comprising:

[0024] The remote control end is used to obtain a control command of the control device, discard part of the control point data of the control command, obtain a control command to discard part of the control point data, and send the control command to discard part of the control point data to the vehicle end;

[0025] The vehicle side is used to receive a control command that discards part of the control point data, restore the control point data of the control command through interpolation, and generate a command sequence for controlling the execution device.

[0026] In at least one embodiment of the present application, the remote control terminal includes:

[0027] A real-time command receiving queue module is used to obtain control commands from different control devices in real time and store them in a classified queue;

[0028] The network QoS analysis module is used to record and monitor the end-to-end transmission delay, send it through a separate socket link, save and dynamically update the delay average value, and calculate the packet loss strategy integer operator μ based on the delay average value and the sampling period of the control device, where: T is the average delay value, t is the sampling period of the control device, Indicates rounding up to an integer;

[0029] The DROPOUT processing module is used to read the control commands of the classification queue and discard some control point data according to the packet loss strategy integer operator μ, specifically, μ-1 is discarded from every μ control commands of the classification queue. If μ=1, no discard is required.

[0030] The classification and packaging module is used to classify and package the control commands that discard some control point data, and send them to the vehicle end through the socket link.

[0031] In at least one embodiment of the present application, the control commands of the control device include real-time operation instructions of the steering wheel, real-time operation instructions of the brake, and real-time operation instructions of the joystick.

[0032] In at least one embodiment of the present application, the vehicle end includes:

[0033] A classification unpacking receiving module is used to parse a received control command data packet, wherein the control command data packet includes a control command ID and a control command value;

[0034] A control curve interpolation module is used to obtain a control curve and a fitting function of a corresponding control device according to a control command ID;

[0035] A command sequence generation module, used for filling control points discarded by interpolation between control commands according to a fitting function, and generating a command sequence;

[0036] The execution device command sending module is used to send the generated command sequence to the execution device.

[0037] In at least one embodiment of the present application, the control command and the command sequence are both digital signals, or the control command and the command sequence are both analog signals.

[0038] The invention has at least the following beneficial technical effects:

[0039] The vehicle remote interpolation fitting smooth control method of the present application improves the robustness and adaptability to network jitter that carries remote control driving operations; retains the operating accuracy attributes of the original control device, and can improve the accuracy of remote control operations; improves the smoothness of remote driving operation effects, and can improve the driving operator's smooth remote control experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a block diagram of a vehicle remote interpolation fitting smooth control method according to one embodiment of the present application;

[0041] Figure 2 It is a flow chart of a vehicle remote interpolation fitting smooth control method according to an embodiment of the present application;

[0042] Figure 3 It is a schematic diagram of a vehicle remote interpolation fitting smooth control device according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical scheme and advantages of the implementation of this application clearer, the technical scheme in the embodiment of this application will be described in more detail below in conjunction with the drawings in the embodiment of this application. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of this application, not all of them. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain this application, and should not be construed as limitations on this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The embodiments of this application are described in detail below in conjunction with the drawings.

[0044] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the scope of protection of the present application.

[0045] The following is combined with Figures 1 to 3 This application is described in further detail.

[0046] A first aspect of the present application provides a vehicle remote interpolation fitting smooth control method, comprising:

[0047] The remote control end obtains the control command of the control device, discards part of the control point data of the control command, obtains the control command discarding part of the control point data, and sends the control command discarding part of the control point data to the vehicle end;

[0048] After receiving the control command that discards part of the control point data, the vehicle end restores the control point data of the control command through interpolation and generates a command sequence for controlling the execution device.

[0049] In a preferred embodiment of the present application, the process in which the remote control terminal obtains the control command of the control device, discards part of the control point data of the control command, obtains the control command for discarding part of the control point data, and sends the control command for discarding part of the control point data to the vehicle terminal can be divided into four stages: real-time command receiving queue, network QoS analysis management, command DROPOUT, and classification and grouping packet sending, specifically including:

[0050] The remote control terminal obtains control commands of different control devices in real time and stores them in a classified queue;

[0051] Record and monitor the end-to-end transmission delay, send it through a separate socket link, save and dynamically update the average delay value, and calculate the packet loss strategy integer operator μ based on the average delay value and the sampling period of the control device, where: T is the average delay value, t is the sampling period of the control device, Indicates rounding up to an integer;

[0052] Read the control command of the classification queue, and discard some control point data according to the packet loss strategy integer operator μ, specifically, discard μ-1 from every μ control commands of the classification queue. If μ=1, no discard is required;

[0053] The control commands that discard some control point data are classified and packaged, and sent to the vehicle end through the socket link.

[0054] In the vehicle remote interpolation fitting smooth control method of the present application, the control device may include a variety of different control devices, for example, a control device for controlling the vehicle steering wheel, a control device for controlling the vehicle brakes, and a control device for controlling the vehicle joystick. In this embodiment, the control commands of the control device include real-time operation instructions of the steering wheel, real-time operation instructions of the brakes, and real-time operation instructions of the joystick. When the control commands are stored in the classification queue, the command sending frequency of the native control device is not changed.

[0055] The vehicle remote interpolation fitting smooth control method of the present application has jitter in the wireless network transmission quality due to factors such as environmental interference, and it is necessary to record and monitor the end-to-end transmission delay. In order to reduce the impact on the control command transmission, it is preferably detected once every 1s and sent through a separate socket link. The reference delay average value is saved and dynamically updated, and the packet loss strategy integer operator μ is calculated using the reference delay average value and the sampling period of the specific vehicle control device.

[0056] In the vehicle remote interpolation fitting smooth control method of the present application, the end-to-end transmission delay refers to the time taken from sending a data packet from the remote control end to receiving the data packet at the vehicle end, which can be statistically updated in seconds, which is T milliseconds. The sampling period of the control device refers to the period of a control instruction issued by a control device such as a steering wheel, accelerator pedal, etc., which is t milliseconds. In a preferred embodiment of the present application, the packet loss strategy integer operator μ = [T / t], and the operator μ takes an integer (such as 2.7 or 2.1 calculated to be 3) to reduce network congestion. The packet loss strategy takes one instruction from the classification queue for every u control commands according to the operator, and discards μ-1. If μ = 1, no discard is required.

[0057] In one embodiment of the present application, for example, a 5G application network system is built, the end-to-end delay average value is 35ms, and the collection period of the control device is 10ms, then μ=[35 / 10]=4, and one out of every four control commands in the classification queue is sent, and the other three are discarded, and the instructions that need to be sent in the classification queue are processed in sequence.

[0058] In a preferred embodiment of the present application, after the vehicle receives a control command that discards part of the control point data, the process of restoring the control point data of the control command by interpolation and generating a command sequence for controlling the execution device can be divided into four stages: classified unpacking reception, control curve interpolation model, command sequence generation, and control device command sending, specifically including:

[0059] The vehicle side parses the received control command data packet, which includes a control command ID and a control command value;

[0060] Obtain the control curve and fitting function of the corresponding control device according to the control command ID;

[0061] Fill the control points discarded by interpolation between the control commands according to the fitting function, and generate a command sequence;

[0062] The generated command sequence is sent to the execution device.

[0063] The vehicle remote interpolation fitting smooth control method of the present application can obtain the control curve of the specific control device through the manufacturer, and each type of control device corresponds to a fitting function. In addition, according to the generated various command sequences, CAN instructions are sent to the vehicle controller VCU or the corresponding execution device MCU. They may be hung on the same bus or not, and can be processed according to the actual situation.

[0064] In one embodiment of the present application, the control command and the command sequence are both digital signals. In another embodiment of the present application, the control command and the command sequence are both analog signals.

[0065] Based on the above-mentioned vehicle remote interpolation fitting smooth control method, the second aspect of the present application provides a vehicle remote interpolation fitting smooth control device, including:

[0066] The remote control end is used to obtain a control command of the control device, discard part of the control point data of the control command, obtain a control command to discard part of the control point data, and send the control command to discard part of the control point data to the vehicle end;

[0067] The vehicle side is used to receive a control command that discards part of the control point data, restore the control point data of the control command through interpolation, and generate a command sequence for controlling the execution device.

[0068] The vehicle remote interpolation fitting smooth control device of the present application has a remote control end deployed in a remote control end controller, which is responsible for processing control commands received from the control device, and a vehicle end deployed in a vehicle end controller, which is responsible for generating a command sequence for controlling the execution device. By using the interpolation fitting technology, some control commands are actively abandoned at the remote control end to reduce the control frames that actually need to be transmitted, and the control command data points are restored by using the interpolation method at the vehicle end to achieve the goal of remote smoothness control.

[0069] In the preferred embodiment of the present application, the remote control terminal includes a real-time command receiving queue module, a network QoS analysis module, a DROPOUT module and a classification and grouping module, wherein:

[0070] A real-time command receiving queue module is used to obtain control commands from different control devices in real time and store them in a classified queue;

[0071] The network QoS analysis module is used to record and monitor the end-to-end transmission delay, send it through a separate socket link, save and dynamically update the delay average value, and calculate the packet loss strategy integer operator μ based on the delay average value and the sampling period of the control device, where: T is the average delay value, t is the sampling period of the control device, Indicates rounding up to an integer;

[0072] The DROPOUT processing module is used to read the control commands of the classification queue and discard some control point data according to the packet loss strategy integer operator μ, specifically, μ-1 is discarded from every μ control commands of the classification queue. If μ=1, no discard is required.

[0073] The classification and packaging module is used to classify and package the control commands that discard some control point data, and send them to the vehicle end through the socket link.

[0074] The vehicle remote interpolation fitting smooth control device of the present application may include a variety of different control devices, for example, a control device for controlling the vehicle steering wheel, a control device for controlling the vehicle brakes, and a control device for controlling the vehicle joystick. In this embodiment, the control commands of the control device include real-time operation instructions of the steering wheel, real-time operation instructions of the brakes, and real-time operation instructions of the joystick. When the control commands are stored in the classification queue, the command sending frequency of the native control device is not changed.

[0075] In a preferred embodiment of the present application, the vehicle end includes a classification unpacking receiving module, a control curve interpolation module, a command sequence generating module, and an execution device command sending module, wherein:

[0076] A classification unpacking receiving module is used to parse a received control command data packet, wherein the control command data packet includes a control command ID and a control command value;

[0077] A control curve interpolation module is used to obtain a control curve and a fitting function of a corresponding control device according to a control command ID;

[0078] A command sequence generation module, used for filling control points discarded by interpolation between control commands according to a fitting function, and generating a command sequence;

[0079] The execution device command sending module is used to send the generated command sequence to the execution device.

[0080] In one embodiment of the present application, the control command output by the remote control device and the command sequence for driving the vehicle-side actuator are both CAN digital signals. In another embodiment of the present application, the control command output by the remote control device and the command sequence for driving the vehicle-side actuator are both analog signals.

[0081] The vehicle remote interpolation fitting smooth control method and device of the present application are implemented by actively discarding the real-time control frame of the remote control end and the strategy and technology of restoring the generated control frame on the vehicle end. When the network jitter does not exceed the threshold, it can realize both the real-time transmission of control instructions and the smoothness and precision of control; the DROPOUT strategy of the control command frame can be calculated in real time according to the network quality to reduce the risk of network congestion; the vehicle end fits the curve function according to the control performance of the native control device, and the control sequence frame generated by interpolation has a high degree of matching with the locally generated control frame, retaining the experience consistency of the original device; the control command discarding package, grouping package and interpolation scheme are relatively flexible, and some operation network resources that are not sensitive to accuracy can be released for operations that require higher control accuracy; it has strong adaptability to the nativeness of the vehicle's native control system, and there is no need to make any changes to the software and hardware outside the remote control system.

[0082] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A vehicle remote interpolation fitting smooth control method, characterized in that: include: The remote control end obtains the control command of the control device, discards part of the control point data of the control command, obtains the control command to discard part of the control point data, and sends the control command to discard part of the control point data to the vehicle end; specifically: The remote control terminal obtains control commands of different control devices in real time and stores them in a classified queue; Record and monitor the end-to-end transmission delay, send it through a separate socket link, save and dynamically update the average delay value, and calculate the packet loss strategy integer operator μ based on the average delay value and the sampling period of the control device, where: T is the average delay value, t is the sampling period of the control device, Indicates rounding up to an integer; Read the control command of the classification queue, and discard some control point data according to the packet loss strategy integer operator μ, specifically, discard μ-1 from every μ control commands of the classification queue. If μ=1, no discard is required; Classify and package the control commands that discard some control point data, and send them to the vehicle end through the socket link; After receiving the control command that discards part of the control point data, the vehicle end restores the control point data of the control command through interpolation and generates a command sequence for controlling the execution device.

2. The vehicle remote interpolation fitting smooth control method according to claim 1, characterized in that: The control commands of the control device include real-time operation instructions of the steering wheel, real-time operation instructions of the brake and real-time operation instructions of the joystick.

3. The vehicle remote interpolation fitting smooth control method according to claim 2 is characterized in that: After the vehicle end receives the control command for discarding part of the control point data, restoring the control point data of the control command by interpolation and generating a command sequence for controlling the execution device includes: The vehicle side parses the received control command data packet, wherein the control command data packet includes a control command ID and a control command value; Obtain the control curve and fitting function of the corresponding control device according to the control command ID; Fill the control points discarded by interpolation between the control commands according to the fitting function, and generate a command sequence; The generated command sequence is sent to the execution device.

4. The vehicle remote interpolation fitting smooth control method according to claim 3 is characterized in that: The control command and the command sequence are both digital signals, or the control command and the command sequence are both analog signals.

5. A vehicle remote interpolation fitting smooth control device, characterized in that: include: The remote control end is used to obtain a control command of the control device, discard part of the control point data of the control command, obtain a control command to discard part of the control point data, and send the control command to discard part of the control point data to the vehicle end; The remote control terminal comprises: A real-time command receiving queue module is used to obtain control commands from different control devices in real time and store them in a classified queue; The network QoS analysis module is used to record and monitor the end-to-end transmission delay, send it through a separate socket link, save and dynamically update the delay average value, and calculate the packet loss strategy integer operator μ based on the delay average value and the sampling period of the control device, where: T is the average delay value, t is the sampling period of the control device, Indicates rounding up to an integer; The DROPOUT processing module is used to read the control commands of the classification queue and discard some control point data according to the packet loss strategy integer operator μ, specifically, μ-1 is discarded from every μ control commands of the classification queue. If μ=1, no discard is required. The classification and packaging module is used to classify and package the control commands that discard some control point data, and send them to the vehicle end through the socket link; The vehicle side is used to receive a control command that discards part of the control point data, restore the control point data of the control command by interpolation, and generate a command sequence for controlling the execution device.

6. The vehicle remote interpolation fitting smooth control device according to claim 5, characterized in that: The control commands of the control device include real-time operation instructions of the steering wheel, real-time operation instructions of the brake and real-time operation instructions of the joystick.

7. The vehicle remote interpolation fitting smooth control device according to claim 6, characterized in that: The vehicle end includes: A classification unpacking receiving module is used to parse a received control command data packet, wherein the control command data packet includes a control command ID and a control command value; A control curve interpolation module is used to obtain a control curve and a fitting function of a corresponding control device according to a control command ID; A command sequence generation module, used for filling control points discarded by interpolation between control commands according to a fitting function, and generating a command sequence; The execution device command sending module is used to send the generated command sequence to the execution device.

8. The vehicle remote interpolation fitting smooth control device according to claim 7, characterized in that: The control command and the command sequence are both digital signals, or the control command and the command sequence are both analog signals.