Vehicle remote control driving man-machine interaction method and system

Abstract

The invention relates to the technical field of vehicle remote control driving, and discloses a vehicle remote control driving man-machine interaction method and system, and the system comprises a multi-terminal interaction interface module (1), a remote control interaction interface module (2), and a plurality of function execution submodules. The multi-terminal interaction interface module (1) establishes and verifies communication connection between a mobile terminal and a vehicle. The remote control interaction interface module (2) responds to the signal and presents an integrated remote control driving main interface; and the plurality of function sub-modules comprise steering control, vehicle speed control, vehicle environment, gear information, instrument information and function setting interactive interface modules, are in communication connection with the main interface and are respectively and correspondingly provided with function interactive interfaces. By optimizing interface layout and interaction logic, a vehicle remote control driving solution which is visual in operation, accurate in control and high in safety is provided, and reliability of remote control and user experience are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle intelligent control technology, and discloses a human-machine interaction method and system for remote vehicle driving, particularly the design of a human-machine interface in a vehicle remote control system. Background Technology

[0002] Remote driving technology is an important branch of autonomous driving technology. It refers to a technical solution that uses a remote control terminal (physical handle, mobile terminal, etc.) to transmit control signals through a communication link to remotely control the core operations of a vehicle, such as steering, acceleration, and braking, without the operator having to directly touch the vehicle's control mechanisms (such as steering wheel, accelerator, brake pedal, etc.).

[0003] Remote driving technology enables precise remote control of vehicles through remote control terminals. Its core value lies in overcoming spatial limitations, allowing operators to safely control vehicles in dangerous or inaccessible scenarios (such as narrow parking spaces or high-risk areas), significantly improving operational safety and flexibility under special conditions. This technology integrates intelligent connectivity with high-precision interactive control, expanding the boundaries of vehicle applications through real-time video and an intuitive interface while ensuring safety. It provides key technical support for scenarios such as remote takeover and intelligent parking, representing an important practical path for the development of intelligent and connected vehicles.

[0004] With the rapid development of intelligent connected vehicle technology, the application scenarios of vehicle remote driving technology are becoming increasingly widespread. Traditional vehicle remote driving solutions mainly rely on dedicated remote control hardware or basic control software based on mobile terminals. These existing technologies still have the following shortcomings: insufficient operational precision, making it difficult to adapt to low-speed, precise scenarios; poor demand interaction adaptation, relying on dedicated hardware or having a rough interface that is prone to accidental touches and lacks feedback; unstable communication links, resulting in insufficient reliability in complex environments; and limited scenario adaptation, with poor cross-scenario applicability.

[0005] Chinese Patent Publication No. CN120704303A, entitled "A Vehicle Remote Control Driving System and Method," proposes a system comprising a cloud remote control platform and an in-vehicle control terminal. The cloud remote control platform sends a communication connection request to the in-vehicle control terminal after the target user's authentication is successful. The in-vehicle control terminal authenticates the cloud remote control platform and establishes an encrypted communication connection. The cloud remote control platform generates a remote control mode activation command, a remote driving interface, and driving control commands, and sends the driving control commands to the in-vehicle control terminal. The in-vehicle control terminal decrypts the encrypted driving control commands and performs driving control on the target vehicle. This solution constructs a two-tier architecture of "cloud remote control platform + in-vehicle control terminal," establishing an encrypted communication connection through two-way authentication. The cloud platform generates the remote driving interface and sends control commands, which are then decrypted and executed by the in-vehicle terminal. The core solution addresses the issues of "remote communication security and basic remote control command transmission." However, this solution relies heavily on the cloud platform for remote control and does not provide any alternative communication links or control modes. The cloud platform becomes the sole core of the system, simultaneously handling multiple responsibilities such as user interface rendering, driving control logic processing, and encrypted data transmission. A server failure or network outage could render the entire remote control function completely ineffective. Furthermore, command transmission requires traversing a "terminal-cloud-vehicle" path, increasing communication latency. In addition, the solution only vaguely mentions that the cloud platform generates the remote driving interface and driving commands, without specifying the control logic for core operations such as steering, driving, and braking. This results in insufficient intuitiveness and a potential lack of a local interface and the ability to process complex commands.

[0006] Chinese patent publication number CN115712255A, entitled "A Remote Control Driving System for Human-Machine Co-driving," describes a method that achieves secure parsing and execution of remote control commands through command interaction between a control terminal and an onboard terminal, fusion of environmental perception data, and reduced risks caused by errors in judgment by a single operator. This method achieves remote control driving through human-machine co-driving, avoiding the data delays and safety issues inherent in traditional remote control driving where the operator is solely responsible. The operator only needs to issue driving mode control commands, which are received by the onboard autonomous driving system and executed in accordance with the actual environment, thereby improving the safety of remote control driving. However, its technical focus is on the instruction parsing and environment adaptation of the vehicle-side autonomous driving module, without involving the refinement of mobile terminal touch interaction; the control-side instructions are "general descriptions", such as the "steering" instruction does not include precise parameters such as steering angle and speed, and must rely on the vehicle-side autonomous driving module to calculate based on the environment, which may have two core problems: in low-speed and precise scenarios (such as reversing into a parking space), the vehicle-side calculation is prone to deviation due to the complexity of the environment; the operator cannot directly control the execution accuracy through touch actions, thus "losing the real-time controllability of human-computer interaction".

[0007] Chinese patent publication number CN119225346A, entitled "A Remote Driving Method and System," focuses on remote safety assistance for unmanned vehicles in mining scenarios. Specifically, the unmanned vehicle acquires surrounding environmental information through sensing devices such as LiDAR and cameras; divides the surrounding area into 8 to 10 sub-spaces according to a "first specified rule"; identifies the nearest target object (including distance) within each sub-space and sends it to the remote driving terminal; the terminal displays the target object information on the interface according to a "second specified rule" (e.g., displaying the sub-space border and target distance around the vehicle identifier), assisting the driver in judging the environment. This core solution addresses the problem of insufficient environmental information in remote driving in mining scenarios. However, this solution revolves entirely around "environmental information acquisition – sub-space division – target object display," without mentioning any touch interaction design for "remote control command input," lacking both touch action rules and command quantification generation logic. Furthermore, in this solution, the "remote driving terminal" is only used as an "environmental information display" rather than a "precise operation input carrier," which may not be able to solve the core problem of how the driver performs steering, acceleration, and other operations based on the displayed information; its safety protection relies solely on the "environmental information display" and has no "end-side operation safety" design.

[0008] Chinese patent publication number CN116229695A, entitled "A Bluetooth-based Vehicle Remote Control Driving Method and System," focuses on basic vehicle remote control via Bluetooth communication. Its specific architecture and functions are as follows: A two-level link is constructed between a mobile terminal and an in-vehicle Bluetooth control module. Low-latency communication is achieved through Bluetooth, solving the problem of "remote control failure when the network signal is poor." The operation involves setting "forward / backward / left turn / right turn / one-click return to center" buttons on the mobile terminal interface. Pressing the button executes the corresponding action, and releasing the button stops the movement. For steering, a long press of the button is required to achieve a fixed angle of rotation. If an obstacle is detected during command execution, Bluetooth connection is lost, or the operation page is switched to the background, the vehicle stops and remains stationary. Pressing the button while stationary allows for steering wheel return to center. This solution enables basic Bluetooth remote control functionality and solves the problem of "remote control connectivity in poor network scenarios." However, considering only short-range Bluetooth communication technology, its effective control distance is typically less than 100 meters, and the signal is easily interfered with by physical obstacles, resulting in poor connection stability. More importantly, the lack of communication link redundancy means that it cannot switch to a backup channel when the main communication is interrupted, making it difficult to guarantee reliability in complex electromagnetic environments. Furthermore, the solution lacks a command quantification generation mechanism (such as precise control of steering angle and acceleration intensity), and its "remote driving" can only achieve basic actions like "forward / backward / rough steering," lacking intuitive visual feedback and intelligent guidance. This may not meet the remote driving needs of some complex scenarios, and its core limitations lie in its crude operation, rudimentary interaction, and poor adaptability.

[0009] In summary, existing limitations restrict the application of remote-controlled driving in complex scenarios. An integrated control method is needed to achieve "precise steering control, timely interactive feedback, stable communication links, and strong scenario adaptability" to overcome the shortcomings of existing technologies. Summary of the Invention

[0010] To achieve the above objectives, the present invention provides a human-machine interaction method and system for remote vehicle driving, adopting the following technical solution:

[0011] A vehicle remote driving human-machine interaction method and system includes: a multi-terminal interaction interface module (1), a remote control interaction interface module (2), a steering control interaction interface module (3), a vehicle speed control interaction interface module (4), a vehicle environment interaction interface module (5), a gear information interaction interface module (6), an instrument information interaction interface module (7), and a function setting interaction interface module (8).

[0012] Furthermore, the multi-terminal interactive interface module (1) is configured to establish a communication connection between the mobile terminal and the vehicle. After the connection is successful, it sends a connection ready signal to the remote control interactive interface module (2). The remote control interactive interface module (2) receives the connection ready signal and loads and presents the remote control interactive interface on the mobile terminal.

[0013] The steering control interactive interface module (3), vehicle speed control interactive interface module (4), vehicle environment interactive interface module (5), gear information interactive interface module (6), instrument information interactive interface module (7) and function setting interactive interface module (8) establish a communication connection with the remote control interactive interface module (2), and set corresponding steering control interactive interface (21), vehicle speed control interactive interface (22), vehicle environment interactive interface (23), gear information interactive interface (24), instrument information interactive interface (25) and function setting interactive interface (26) on the remote control interactive interface. Each interactive interface is spatially partitioned and functionally complementary and coordinated.

[0014] Furthermore, the system adopts a three-tier architecture consisting of a communication connection layer, an interface presentation layer, and a function execution layer. Each layer interacts with data through standardized interfaces. The system optimizes the spatial layout of the interactive interfaces corresponding to the eight functional modules based on the priority of remote driving tasks. It achieves intelligent linkage by relying on an event-driven mechanism and constructs a closed-loop control link of "command input - status feedback - safety monitoring" through a unified data bus, thus fully realizing the human-machine interaction function of remote vehicle driving.

[0015] Furthermore, the multi-terminal interactive interface module (1) includes a multi-terminal interactive interface (11), a communication connection unit (12), and a permission allocation unit (13).

[0016] The multi-terminal interactive interface (11) presents the available vehicle connection method options to the user, receives the user's selection input, and calls the communication connection unit (12) to initiate a connection;

[0017] The communication connection unit (12) responds to the call of the vehicle connection interface (11) to establish and maintain a data transmission channel with the vehicle, the data transmission channel including at least one of USB wired connection, Wi-Fi direct connection and cloud remote connection;

[0018] The permission allocation unit (13) identifies the first terminal that successfully establishes a control connection with the vehicle as the main control terminal and grants it complete vehicle driving control permissions; it identifies the subsequent connected terminals as monitoring terminals and restricts their permissions to vehicle status monitoring and operation of non-driving related on-board equipment.

[0019] Furthermore, the remote control interactive interface module (2) provides a steering control interactive interface (21), a vehicle speed control interactive interface (22), a vehicle environment interactive interface (23), a gear information interactive interface (24), an instrument information interactive interface (25), and a function setting interactive interface (26) that are spatially isolated from each other.

[0020] The six interactive interfaces are optimized and arranged based on the driving task logic. The steering control interactive interface (21) and the vehicle speed control interactive interface (22) are located on the left and right sides of the interface, respectively, which is convenient for two-hand operation. The vehicle environment interactive interface (23) is set in the central area of ​​the interface to display the image of the vehicle's surrounding environment. The instrument information interactive interface (25) and the gear information interactive interface (24) are located in the upper area of ​​the interface, which is convenient for the driver to quickly obtain vehicle status information. After entering the function setting interactive interface (26), it is superimposed on the environment interactive interface (23) to provide the system parameter configuration entry.

[0021] Furthermore, the steering control interactive interface module (3) receives the user's steering operation input through the steering control interactive interface (21) and generates vehicle steering control commands based on the rotation angle and direction of the operation trajectory;

[0022] A dual mapping relationship is established in the steering control interface (21): First, at the operation level, a first mapping relationship is established between the cumulative rotation angle of the user's operation trajectory and the steering wheel angle of the vehicle, so as to accurately convert the touch operation into steering wheel control command; Second, at the vehicle execution level, a second mapping relationship is established between the steering wheel angle and the actual rotation angle of the vehicle wheels to ensure the accurate execution of the control command.

[0023] The steering control interface (21) synchronously displays the current steering wheel angle value (31), wheel angle value (32) and dynamic wheel angle diagram (33), wherein the steering wheel angle value is displayed in numerical form and the wheel angle diagram is dynamically displayed through a rotatable wheel icon.

[0024] A horn trigger area (34) is set in the steering control interface (21), which is displayed in the form of a horn trigger circle and generates a vehicle horn trigger signal in response to user touch operation;

[0025] Furthermore, the steering control interface module (3) provides steering return-to-center function, including autonomous return-to-center (35) and automatic return-to-center (36).

[0026] Autonomous return to center (35): When an operation input opposite to the current steering direction is detected in the steering control interface (21), the wheel rotation angle is controlled to decrease to the return to center state according to the rotation angle of the reverse operation.

[0027] Automatic return to center (36): When the user's finger leaves the steering control interface (21), the wheel angle is automatically returned to center at a preset rate, and visual feedback information is updated synchronously during the return to center process.

[0028] Furthermore, the vehicle speed control interactive interface module (4) receives the user's operation input through the vehicle speed control interactive interface (22) and converts the user's interactive operation into vehicle acceleration or braking control commands.

[0029] The acceleration and braking status (41) is displayed in a dynamic and visual manner in the vehicle speed control interactive interface (22). The visual attributes of the graphic elements change with the control intensity and are supplemented by a digital percentage display. Specifically, the control intensity is intuitively displayed through the progress bar length and color gradient (such as from green to red) and supplemented by a precise digital percentage display, providing users with intuitive speed control feedback.

[0030] The vehicle speed control interface (22) includes an acceleration start area (42), a turn signal control area (43), a braking status indication area (44), and a power and voltage display area (45).

[0031] Furthermore, the vehicle speed control interactive interface module (4) supports single-pedal mode and double-pedal mode, which can be selected in the function setting interactive interface (26);

[0032] In single-pedal mode, the user’s linear sliding in the first direction (such as sliding down) in the vehicle speed control interface (22) is mapped to an acceleration command, and the sliding in the opposite direction (such as sliding up) is mapped to a braking command, thereby achieving efficient energy recovery and vehicle control.

[0033] In dual-pedal mode, the user needs to first trigger the acceleration start area (42) to activate the acceleration function, and then control the acceleration by sliding down in the vehicle speed control interface (22); while sliding up in any position in the vehicle speed control interface (22) triggers the braking command, maintaining the traditional driving operation habits.

[0034] Furthermore, the vehicle environment interaction interface module (5) includes an environment display unit (51) and a real-time transmission unit (52).

[0035] The environment display unit (51) adopts multi-channel video stream synchronous processing technology, which is responsible for receiving and processing video data from the vehicle-side surround view camera system, and performing high-quality rendering display in the vehicle environment interaction interface (23). During the display process, the system supports perspective switching and screen zooming functions, and users can view partial close-ups of the panoramic view according to driving needs;

[0036] The real-time transmission unit (52) establishes a dedicated video data transmission channel and adopts an adaptive bitrate control mechanism to ensure the real-time performance of video transmission. This unit continuously monitors the network link quality and dynamically adjusts video encoding parameters (including frame rate, resolution, and compression ratio) to control the transmission delay within a preset threshold.

[0037] Furthermore, the gear information interaction interface module (6) provides gear switching function through the gear information interaction interface (24);

[0038] The gear information interaction interface (24) adopts a graphical design, clearly displaying four gear positions: Parking (P), Reverse (R), Neutral (N), and Drive (D). Each gear position is distinguished by unique visual identifiers, including but not limited to color coding (such as red for P, yellow for R, green for N, and blue for D), icon shape changes, and text prompts. When a user selects a specific gear position, that gear position is selected and highlighted by a circular identifier, while other gear position identifiers will dim accordingly, providing intuitive feedback on the current gear position status.

[0039] The gear information interaction interface module (6) monitors the vehicle speed signal and the braking system status signal in real time. The safety interlock logic is set to activate the gear switching function only when both conditions are met: "vehicle speed is zero" and "braking system is in effective braking state".

[0040] Furthermore, the instrument information interaction interface module (7) provides an integrated vehicle control panel through the instrument information interaction interface (25) to realize centralized control of vehicle equipment, including mode switching operation of vehicle lighting group (71), and start / stop control and operation parameter adjustment of air conditioning system (72), wiper system (73) and window defrosting system (74);

[0041] Lighting control system (71): provides working mode switching function for lighting equipment such as headlights, fog lights, and hazard lights, and supports centralized control of multiple lighting modes such as automatic, low beam, and high beam;

[0042] Air conditioning system (72): It realizes precise control of air conditioning parameters such as temperature setting, air volume adjustment, and air outlet mode, and supports remote pre-adjustment of the in-vehicle environment;

[0043] Wiper system (73): Provides multiple wiper working modes such as intermittent, low speed, and high speed;

[0044] Window defogging system (74): integrates front and rear window defogging functions and supports one-button quick defogging operation mode;

[0045] The instrument information interaction interface module (7) integrates and displays key vehicle operating status parameters within the instrument information interaction interface (25), provides intuitive vehicle speed feedback in digital form, and clearly displays the activation status of intelligent driving mode through icon color changes.

[0046] Furthermore, the function setting interactive interface module (8) provides system parameter configuration function through the function setting interactive interface (26). This module realizes multi-level and categorized system parameter configuration function through the function setting interactive interface (26), providing complete setting management support for the vehicle remote control driving system.

[0047] The vehicle self-check and diagnostic function (81) automatically performs a comprehensive vehicle self-check process when the system starts. This function performs status checks on key components of the vehicle, such as the power system, braking system, and steering system, through a multi-level diagnostic protocol and generates a diagnostic report. When a fault is detected, the system generates a detailed query interface containing fault codes and fault descriptions;

[0048] The safe exit function (82) first verifies whether the current state of the vehicle meets the safe exit conditions when responding to the user's exit operation, including multiple safety indicators such as the vehicle speed being zero, the gear being in parking gear, and the electronic parking brake being activated. Only when all verification conditions are met will the system safely terminate the current remote driving session to ensure the safety of the exit process;

[0049] The air conditioning parameter adjustment function (83) provides comprehensive control of the air conditioning system and supports centralized management of air volume level (stepless adjustment from 1 to 8 levels), target temperature (precise control from 16 to 30℃) and internal and external circulation modes;

[0050] The driving trajectory management function (84) realizes the full life cycle management of vehicle driving trajectory and supports real-time recording, storage, playback and export functions;

[0051] The connection mode switching function (85) supports intelligent switching between near-end remote control mode (based on USB connection and Wi-Fi Direct) and far-end remote control mode (based on mobile network);

[0052] The driving mode configuration function (86) provides flexible configuration of driving mode. Users can select the working mode of the vehicle speed control interaction interface module (4) as single pedal mode or double pedal mode through an intuitive interface. The system will automatically optimize the interface layout and control logic according to different modes.

[0053] The user account management function (87) provides complete support for account information maintenance, including functions such as modifying user personal information and updating login credentials, to ensure the security and personalization of system use.

[0054] In summary, the beneficial effects of the present invention are as follows:

[0055] Precise interface partitioning reduces the risk of accidental touches. This invention employs a functionally customized interface partitioning design, using a clearly defined interface area division mechanism to clearly divide the driving interaction interface into six core interaction areas. Combined with highlighted boundary prompts, this provides visual guidance and operational isolation. Simultaneously, it precisely defines the touch recognition range of each area, eliminating accidental touches across areas from the source of operation, thus solving the problems of chaotic interface layout and susceptibility to misoperation.

[0056] Precise steering control and intuitive, efficient operation. This invention constructs a three-level mapping mechanism of "touch-slide rotation - steering wheel angle - wheel angle," combined with a spiral visual trajectory generated synchronously during steering, real-time steering angle values, and wheel status icon feedback. This achieves precise matching between steering actions and visual feedback, meeting the needs of high-precision scenarios such as low-speed parking and parallel parking, significantly improving control accuracy and operational efficiency.

[0057] Flexible mode adaptation and comprehensive scenario coverage. This invention supports flexible switching between single / dual pedal modes, adapting to the operating habits of both traditional driving and electric vehicle users, and improving coverage of different driving scenarios. In addition, the single-pedal mode of this invention has a dedicated gliding logic (the vehicle glides at a constant speed after the finger leaves the screen), avoiding sudden drops in speed, reducing the frequency of continuous operation, and balancing adaptability to different scenarios with driving comfort.

[0058] Collaborative control enhances safety and reliability. This invention reduces user focus switching and improves operational response speed through a linkage feedback mechanism across various interfaces. Simultaneously, it establishes a safety interlock logic for unlocking gears under the dual conditions of "vehicle speed zero + effective braking," fundamentally eliminating the risk of accidental gear shifting while driving. This dual safety guarantee of "status feedback + hardware constraints" significantly improves the safety and reliability of the remote-controlled driving system. Attached Figure Description

[0059] The accompanying drawings are provided to further illustrate the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention.

[0060] Figure 1 This is a schematic diagram of the structure of a vehicle remote control driving human-machine interaction system disclosed in an embodiment of the present invention;

[0061] Figure 2 This is a schematic diagram of a multi-terminal interactive interface of a vehicle remote control driving human-machine interaction system disclosed in an embodiment of the present invention.

[0062] Figure 3 This is a schematic diagram of the overall layout of the remote control interaction interface of a vehicle remote control driving human-machine interaction system disclosed in an embodiment of the present invention;

[0063] Figure 4 This is a schematic diagram of the function setting interaction interface of a vehicle remote driving human-machine interaction system disclosed in an embodiment of the present invention;

[0064] Figure 5 This is a schematic diagram of the vehicle environment interaction interface of a vehicle remote control driving human-machine interaction system disclosed in an embodiment of the present invention.

[0065] Table 1: Reference Table for the Meaning of Serial Numbers Serial Number definition Serial Number definition 1111213 Multi-terminal interactive interface module, multi-terminal interactive interface communication connection unit, permission allocation unit 44142434445 Vehicle speed control interface module: acceleration and braking status, acceleration start area, turn signal control area, braking status indicator area, battery and voltage display area. 2212223242526 Remote control interactive interface module: Steering control interactive interface, Vehicle speed control interactive interface, Vehicle environment interactive interface, Gear information interactive interface, Instrument information interactive interface, Function settings interactive interface 551526771727374 Vehicle environment interaction interface module, environment display unit, real-time transmission unit, gear information interaction interface module, instrument information interaction interface module, vehicle lighting assembly, air conditioning system, wiper system, and window defroster system. 3313233343536 Steering control interface module: Steering wheel angle value, wheel angle value, dynamic wheel angle diagram, horn trigger area, automatic return to center. 881828384858687 Function settings interface modules: vehicle self-check and diagnostics, safe exit function, air conditioning parameter adjustment function, driving trajectory management function, connection mode switching function, drive mode configuration function, and user account management function. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of the present invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. The following embodiments are only used to explain the present invention and are not intended to limit the scope of protection of the present invention. Furthermore, technical features not explicitly defined in this embodiment can be implemented by existing technology, and the same or similar technical features in different embodiments can be referenced from each other.

[0067] Example 1: Implementation of Human-Machine Interaction for Near-End Remote Control Driving

[0068] like Figure 1 As shown, Embodiment 1 of the present invention provides a vehicle remote driving human-machine interaction system, specifically including:

[0069] Multi-terminal interactive interface module (1)

[0070] Channel selection and connection establishment: User access such as Figure 2The multi-terminal interactive interface (11) of the multi-terminal interactive interface module (1) shown allows users to select either a USB wired connection or a Wi-Fi wireless direct connection channel based on the near-end scenario. If a USB connection is selected, the user connects the mobile terminal to the vehicle's OBD interface via a data cable, and the remote control system automatically identifies the vehicle model (e.g., LAIE1) and initiates a connection request. If a Wi-Fi connection is selected, the user connects to the vehicle's dedicated Wi-Fi (SSID: LAIE1) in the mobile terminal system settings, and the remote control system automatically initiates a connection request after detecting a successful Wi-Fi connection.

[0071] Permission verification: After the connection request is initiated, the multi-terminal interactive interface (11) pops up the permission authentication interface. The user enters the preset account (such as the mobile phone number bound to the vehicle) and password (including uppercase and lowercase English letters and special characters). The verification information is transmitted to the permission allocation unit (13) through the communication connection unit (12). The permission allocation unit (13) compares the user information stored locally in the vehicle. After the verification is successful, the vehicle control permission is activated.

[0072] Permission allocation: If the mobile terminal is the first terminal to establish a control connection with the vehicle, the permission allocation unit (13) sets it as the master terminal and grants it full driving control permissions (such as steering, acceleration, braking, and gear switching); subsequent mobile terminals (such as other users' mobile phones) are set as monitoring terminals and only obtain vehicle status monitoring permissions (such as viewing vehicle speed, battery level, and fault information) and non-driving related equipment operation permissions (such as adjusting air conditioning, lights, and windshield wipers), and cannot perform core driving operations such as steering, acceleration, and braking.

[0073] Remote control interactive interface module (2)

[0074] like Figure 3 As shown, the remote control interaction interface module (2) adapts the aspect ratio of the remote control interaction interface rendered on the mobile terminal to the mobile phone screen (default 19:9), and is divided into independent functional interfaces according to the spatial layout:

[0075] The steering control interface (21) is located on the left side of the remote control interface. It is 7.5 units high and 6 units wide. From top to bottom, it is set up with the wheel angle display area, steering wheel angle display area, steering operation area, horn trigger circle and intelligent driving controller status display area. The steering wheel angle display area displays the angle in the form of "left / right + degree". The wheel angle display area displays the corresponding wheel steering angle in the form of "left / right + degree", ranging from -40° to +40°, and is visualized synchronously with the vehicle chassis wheel icon.

[0076] The vehicle speed control interface (22) is located on the right side of the remote control interface. Its size is the same as that of the steering control interface (21). The interface includes an acceleration and braking status display progress bar, an acceleration start button (specific start area), a turn signal trigger area, a brake signal light, a power display area, and a low voltage display area. The progress bar indicates acceleration intensity in blue and braking intensity in red, and displays percentage numbers (0% to 100%) simultaneously.

[0077] The vehicle environment interaction interface (23) is located in the middle of the remote control interaction interface. It is 9 units high and 7 units wide. It is used to render and display the surround view image captured by the vehicle's surrounding cameras and processed by the image stitching algorithm in real time. It also controls the end-to-end delay of the video stream from the vehicle end to the mobile end to be below the preset threshold of 200ms, so as to ensure the real-time performance of the environmental visual feedback during remote driving.

[0078] The gear information interaction interface (24) is usually located in a prominent position in the middle of the remote control interaction interface. It includes parking gear (P), reverse gear (R), neutral gear (N) and drive gear (D). Each gear button is distinguished by a different color (parking gear red, neutral gear green, drive gear blue, reverse gear orange). A bright concentric circle outline is drawn around the currently engaged active gear button as a selection mark.

[0079] The instrument information interaction interface (25) is located on the upper part of the remote control interaction interface, including the lighting operation area (high beam, low beam, rear fog light, hazard lights), self-test status display area, start / stop button, other electrical appliance buttons (air conditioner, wiper, defroster), and setting button.

[0080] Function setting interaction interface (26): When the user touches the gear-shaped setting button in the upper right corner of the remote control interaction interface, the system overlays the function setting interaction interface (26) on the vehicle environment interaction interface (23).

[0081] Steering control interactive interface module (3)

[0082] The steering control interface module (3) implements steering control in the following ways:

[0083] Steering command generation: The user performs a single finger continuous arc sliding (spiral operation) from any point in the steering control interaction interface (21) to simulate steering wheel rotation; the steering control interaction interface (21) collects the cumulative rotation angle of the finger sliding trajectory in real time and establishes the first mapping relationship of "cumulative rotation angle - steering wheel angle" (mapping ratio 1:1, that is, 1° finger rotation corresponds to 1° steering wheel angle), and the steering wheel angle is limited to the range of 540° to 540° from the left to the right; at the same time, based on the vehicle's preset steering transmission ratio (13.5:1), the second mapping relationship of "steering wheel angle - wheel angle" is established, that is, for every 13.5° increase in steering wheel angle, the wheel angle increases by 1°, ensuring that the wheel angle does not exceed the mechanical limit of 40°.

[0084] Visual feedback function: During the steering operation, a spiral trajectory consistent with the sliding direction is generated in real time in the interface. The trajectory growth rate is synchronized with the finger sliding speed. When the finger rotates more than 1.5 times in the same direction, the trajectory no longer grows. The center of the spiral synchronously displays the current steering wheel angle value (31). The wheel angle (32) display area synchronously provides feedback on the angle value and steering direction with numbers and icons.

[0085] Steering return function: Autonomous return (35) Based on the user's steering and sliding in the opposite direction (drawing a circle in the opposite direction), the steering control interface module (3) controls the wheel angle to decrease at a ratio of 1:1 according to the cumulative angle of the reverse sliding, and the spiral trajectory shortens back synchronously until the wheel returns to center (the turning angle is 0°); Automatic return (36) Based on the user's finger leaving the steering control interface (21), the wheel angle is controlled to decrease automatically at a preset rate of 10° / s, and the spiral trajectory shortens and disappears synchronously at this rate until the wheel returns to center, avoiding vehicle deviation caused by failure to return to center.

[0086] Vehicle speed control interactive interface module (4)

[0087] The vehicle speed control interface module (4) implements drive and braking control in the following ways:

[0088] Mode selection: The vehicle speed control interactive interface module (4) supports switching between single-pedal mode and double-pedal mode. Users can select the operation mode in the function setting interactive interface (26).

[0089] Dual-pedal mode: After pressing the virtual button in the acceleration start area (42) of the vehicle speed control interface (22), the user slides down (first direction) to generate an acceleration command. The sliding distance is positively correlated with the throttle opening percentage (the maximum sliding distance is 100% of the throttle opening). The interface displays the acceleration intensity with a blue trajectory and progress bar. The user slides up (second direction) at any position in the drive and brake control interface (22) to generate a braking command. The sliding distance is positively correlated with the braking force percentage (the maximum sliding distance is 100% of the braking force). The interface displays the braking intensity with a red trajectory and progress bar.

[0090] One-pedal mode: The user swipes down to generate an acceleration command (blue trajectory) and swipes up to generate a braking command (red trajectory). When the finger leaves the screen, the trajectory disappears, the vehicle enters coasting mode, and both the throttle opening and braking force are reduced to zero.

[0091] Dynamic status indication (44): During acceleration / braking operations, the interface displays the control intensity in real time with a dynamic progress bar (blue / red). The length of the progress bar and the internal numerical percentage are updated synchronously (such as "throttle 30%" and "brake 60%"). Users can intuitively obtain the current control status.

[0092] Vehicle power and voltage display (45): Real-time display of vehicle power and low voltage values.

[0093] Gear shift information interaction interface module (6)

[0094] The gear information interaction interface module (6) can perform gear shifting operations and has safety interlock logic, as follows:

[0095] When the vehicle speed is detected to be 0 km / h and the braking status signal is valid (braking force percentage ≥ 30%), the gear position button on the gear position information interaction interface (24) is unlocked, and the user can switch between parking gear, neutral gear, forward gear and reverse gear. When switching, the APP will issue a voice prompt (such as "the vehicle has entered forward gear").

[0096] When the vehicle speed signal is detected to be greater than 0 km / h, or the braking status signal is invalid (braking force percentage < 30%), the gear shift buttons in the shift operation area are locked, preventing shifting operations and avoiding safety risks caused by accidental gear shifting while driving.

[0097] Instrument information interaction interface module (7)

[0098] like Figure 3 As shown, the instrument information interaction interface (25) is integrated into the remote control interaction interface. This interaction interface serves as the information aggregation control center and adopts a layered layout design, integrating three major functional modules: vehicle equipment control, key parameter display, and vehicle self-test.

[0099] The vehicle-mounted equipment centralized control function is provided by the instrument information interaction interface module (7) in the upper area of ​​the instrument information interaction interface (25), which provides a centralized control panel for the vehicle-mounted equipment. This panel adopts a grouped icon layout, including:

[0100] The vehicle lighting assembly (71) control provides independent on / off buttons for low beam headlights, high beam headlights, fog lights, and hazard warning lights. Users can switch the light assembly status by tapping the corresponding icon, and the interface provides real-time visual feedback through changes in icon color and brightness (e.g., bright green when on and gray when off).

[0101] The environmental control system includes an air conditioning system (72), a wiper system (73), and a window defroster system (74), integrating an air conditioning switch, a temperature and airflow adjustment slider, a wiper switch, and front and rear window defroster switches. Users can complete the settings by sliding the slider or pressing the button, and the current setting value (such as a set temperature of 24°C) is displayed next to the controls in real time.

[0102] The key operating status parameter display function displays key vehicle operating parameters with high priority in the central core area of ​​the instrument information interaction interface (25):

[0103] Real-time speed display: The current speed is displayed in a large font (e.g., "30 km / h").

[0104] Intelligent driving status indicator: An intelligent driving status indicator is located next to the vehicle speed display area. When the intelligent driving function is activated, the indicator turns blue and displays the words "Intelligent Driving"; when deactivated, it displays a gray "standby" state.

[0105] Function settings interactive interface module (8)

[0106] like Figure 4 As shown, the function setting interface (26) is integrated into the remote control interface to realize comprehensive parameter configuration and safety management of the remote driving system. This interface, as the system control center, adopts a layout combining a category navigation bar and a content area, providing the following configuration functions:

[0107] Vehicle self-check and diagnostic functions (81)

[0108] Self-test initiated: When the user enters remote driving mode, the system automatically performs a vehicle self-test. During the self-test, the interface displays a "System self-test in progress..." message and a progress bar. After the self-test is successful, all status icons turn green.

[0109] Fault Diagnosis: If a fault is detected, the corresponding self-test icon will turn red and flash. After the user clicks the icon, the interface will jump to the details page, displaying the fault and a brief description (such as "ABS anti-lock braking system fault").

[0110] When the user clicks the "Exit Remote Driving" option, the function setting interface module (8) does not execute immediately, but first initiates a safety verification process. The system detects the vehicle's current status parameters in real time, including whether the vehicle speed is 0, whether the gear is in parking gear (P gear), and whether the electronic parking brake is activated. Only when all safety conditions are met will a confirmation dialog box pop up on the interface. After the user confirms a second time, the system safely terminates the remote control session and closes the driving interface.

[0111] The air conditioning parameter adjustment function (83) allows users to make fine-tuned settings for the air conditioning system. The interface provides an intuitive virtual control panel, including:

[0112] Airflow level setting: The airflow level can be adjusted via a horizontal slider, and the current airflow level is displayed in real time.

[0113] Target temperature adjustment: Adjusted via the horizontal slider, with an adjustment range of 16℃-30℃.

[0114] Loop Mode Switch: Provides a visual switch for "inner loop" and "outer loop" modes, with the currently active mode highlighted.

[0115] The driving trajectory management function (84) allows users to turn the trajectory recording function on or off via a switch button. When the function is enabled, the system continuously records data such as the vehicle's geographical location, speed, and timestamps, and generates a visual trajectory route map. Users can replay historical trajectories at any time or choose to export data files.

[0116] The connection mode switching function (85) provides two remote control modes: "near-end remote control" and "far-end remote control". When the user switches modes, the system automatically detects the current network environment: if switching from far-end to near-end, it will automatically search for vehicle Wi-Fi and USB signals; if switching from near-end to far-end, it will verify the availability of cloud services. During the switching process, the interface displays connection status prompts to ensure the reliability and transparency of mode conversion.

[0117] The drive mode configuration function (86) allows users to set the working mode of the vehicle speed control interface module (4) and provides a switch for switching between "single pedal mode" and "double pedal mode".

[0118] The user account management function (87) provides identity management functions, as follows:

[0119] Personal information maintenance: You can modify basic information such as name and contact information.

[0120] Login credentials update: Supports changing the vehicle login password, but requires verification using the original password to ensure security.

[0121] Example 2: Cloud-based remote control driving human-machine interaction implementation method

[0122] This second embodiment, based on the first embodiment, is applicable to scenarios where the user and vehicle are far apart (such as remote dispatching and unmanned operations). It adopts a cloud-based remote connection channel based on mobile networks, combined with real-time video transmission and multi-terminal collaboration, to achieve remote visual remote control driving. Specifically, it includes the following:

[0123] Communication connection and permission allocation process

[0124] Channel selection and connection establishment: The user enters the multi-terminal interactive interface (11) of the multi-terminal interactive interface module (1), selects the cloud connection channel in the multi-terminal interactive interface (11) according to the remote scenario, and the APP automatically sends a connection request to the cloud server. The cloud server queries the list of currently online vehicles (such as LAIE1, LAIE2). After the user selects the target vehicle, a connection request is initiated. The vehicle responds to the request and a cloud communication link is established.

[0125] Permission verification: After the connection request is initiated, the multi-terminal interactive interface (11) pops up the permission authentication interface. The user enters the preset account (such as the mobile phone number bound to the vehicle) and password (including uppercase and lowercase English letters and special characters). The verification information is transmitted to the permission allocation unit (13) through the communication connection unit (12). The permission allocation unit (13) compares the user information stored locally in the vehicle. After the verification is successful, the vehicle control permission is activated.

[0126] Permission allocation: If the mobile terminal is the first terminal to establish a control connection with the vehicle, the permission allocation unit (13) sets it as the master terminal and grants it full driving control permissions (such as steering, acceleration, braking, and gear switching); subsequent mobile terminals (such as other users' mobile phones) are set as monitoring terminals and only obtain vehicle status monitoring permissions (such as viewing vehicle speed, battery level, and fault information) and non-driving related equipment operation permissions (such as adjusting air conditioning, lights, and windshield wipers), and cannot perform core driving operations such as steering, acceleration, and braking.

[0127] Remote control interactive interface module (2)

[0128] like Figure 5 As shown, the remote control interaction interface in the cloud-based remote driving mode is based on Example 1, but with the vehicle environment interaction interface (23) activated, as follows:

[0129] The vehicle environment interaction interface (23) in the middle of the interface is set as a 360° environment image display interface (9 units high and 7 units wide), which displays the environment image collected by the vehicle's panoramic camera in real time and supports touch zoom to view details; the layout and functions of the steering control interaction interface (21), vehicle speed control interaction interface (22), gear information interaction interface (24), instrument information interaction interface (25) and function setting interaction interface (26) are the same as in Example 1.

[0130] Vehicle motion control implementation

[0131] The operation logic, mapping relationship, and return mechanism of the steering control interactive interface module (3) and the vehicle speed control interactive interface module (4) are consistent with those of Example 1, ensuring the uniformity of near-end and far-end driving operations; the difference is that in the cloud remote control driving mode, the environmental image and vehicle status data are displayed synchronously, and the user can adjust the steering, acceleration, and braking operations based on the image feedback to achieve precise remote control.

[0132] Track recording and line-following driving

[0133] Driving trajectory management function (84): When the user turns on the “Track Recording” function in the function setting interface (26) (the button turns green), the cloud server stores the vehicle’s position coordinates, driving speed, steering angle and other data in real time during the vehicle’s driving process, forming a driving trajectory; after the user finishes driving, the trajectory recording function is turned off and the trajectory data is stored in the cloud server for subsequent retrieval.

[0134] Tracking Driving Mode: After activating the intelligent driving mode, the user can select "Tracking Driving Mode". The interface displays the stored driving trajectory (such as Route 1, Route 2, Route 3). The user selects the target route and clicks "Start Driving". The vehicle drives automatically according to the stored trajectory data. Steering, acceleration and braking are automatically controlled by the system. During the driving process, the user can exit Tracking Driving Mode at any time and switch to manual control.

[0135] Communication link guarantee mechanism

[0136] Connection stability guarantee mechanism: The real-time transmission unit (52) continuously monitors the latency and packet loss rate of the communication link between the vehicle and the mobile terminal. When the link quality parameters are detected to exceed the safety threshold, the system first pops up an advanced warning prompt on the remote control interface of the mobile terminal, and then simultaneously sends a braking command to the vehicle speed control unit to automatically trigger the vehicle braking system, so that the vehicle decelerates to a complete stop, and finally automatically switches to the parking gear (P gear) to avoid the risk of loss of control caused by link interruption.

[0137] The two embodiments of the present invention cover two typical remote driving scenarios, namely near-end and far-end. Through multi-channel communication connection, precise motion control logic, perfect safety interlock mechanism and flexible mode management, the present invention solves the problems of complex operation, low safety and poor scenario adaptability of existing vehicle remote driving.

[0138] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A human-machine interaction method and system for remote vehicle driving, characterized in that: It includes a multi-terminal interactive interface module (1), a remote control interactive interface module (2), a steering control interactive interface module (3), a vehicle speed control interactive interface module (4), a vehicle environment interactive interface module (5), a gear information interactive interface module (6), an instrument information interactive interface module (7), and a function setting interactive interface module (8). The multi-terminal interactive interface module (1) is configured to establish and manage the communication connection between the mobile terminal and the vehicle through the multi-terminal interactive interface (11), and send a connection ready signal to the remote control interactive interface module (2) after the connection is successful. The remote control interactive interface module (2) receives the connection ready signal and loads and presents the remote control interactive interface on the mobile terminal. The steering control interactive interface module (3), vehicle speed control interactive interface module (4), vehicle environment interactive interface module (5), gear information interactive interface module (6), instrument information interactive interface module (7) and function setting interactive interface module (8) establish a communication connection with the remote control interactive interface module (2), and set corresponding steering control interactive interface (21), vehicle speed control interactive interface (22), vehicle environment interactive interface (23), gear information interactive interface (24), instrument information interactive interface (25) and function setting interactive interface (26) on the remote control interactive interface. Each interactive interface is spatially partitioned and functionally complementary and coordinated. The system adopts a modular layered architecture that includes a communication connection layer, an interface presentation layer, and a function execution layer. It uses standardized interfaces to achieve data interaction, optimizes the spatial layout of the interactive interfaces corresponding to the eight functional modules based on the priority of remote driving tasks, and realizes the collaborative response and state synchronization between modules through an event-driven mechanism. Then, it constructs a closed-loop control link from "command input" to "state feedback" and then to "safety monitoring" through a unified data bus, thereby fully realizing the human-machine interaction function of remote vehicle driving.

2. The remote control interactive interface module (2) according to claim 1, characterized in that: The remote control interactive interface module (2) provides a steering control interactive interface (21), a vehicle speed control interactive interface (22), a vehicle environment interactive interface (23), a gear information interactive interface (24), an instrument information interactive interface (25), and a function setting interactive interface (26) that are spatially isolated from each other. Each interactive interface can independently respond to the user's touch operation commands within that interface, and the operation responses between the interfaces do not interfere with each other.

3. The steering control interactive interface module (3) according to claim 1, characterized in that: The steering control interface (21) receives the user's steering operation input and generates vehicle steering control commands based on the rotation angle and direction of the operation trajectory. Establish a first mapping relationship between the cumulative rotation angle of the operation trajectory and the steering wheel angle of the vehicle, and establish a second mapping relationship between the steering wheel angle and the actual rotation angle of the vehicle wheels; The steering control interface (21) synchronously displays the current steering wheel angle value (31), wheel angle value (32) and dynamic wheel angle diagram (33), wherein the steering wheel angle value is displayed in numerical form and the wheel angle diagram is dynamically displayed through a rotatable wheel icon. A horn trigger area (34) is set in the steering control interface (21) to generate a vehicle horn trigger signal in response to user touch operation; The steering control interface module (3) provides steering return function, including autonomous return (35) and automatic return (36). Autonomous return to center (35): In response to the detection of an operation input opposite to the current steering direction in the steering control interface (21), the wheel rotation angle is controlled to decrease to the return to center state according to the rotation angle of the reverse operation; Automatic return to center (36): In response to detecting that the user's finger leaves the steering control interface (21), the wheel angle is controlled to automatically return to center at a preset rate, and visual feedback information is updated synchronously during the return to center process.

4. The vehicle speed control interactive interface module (4) according to claim 1, characterized in that: The system receives user input through the vehicle speed control interface (22) and converts the input into vehicle acceleration or braking control commands. The acceleration and braking status (41) is displayed in a dynamic and visual manner within the vehicle speed control interface (22), wherein the visual attributes of the graphic elements change with the control intensity and are supplemented by a digital percentage display; The vehicle speed control interface (22) includes an acceleration start area (42), a turn signal control area (43), a braking status indication area (44), and a power and voltage display area (45). The vehicle speed control interactive interface module (4) supports single-pedal mode and dual-pedal mode; In the single-pedal mode, the linear sliding of the user in the vehicle speed control interface (22) along the first direction is mapped to the vehicle acceleration command, and the linear sliding along the second direction opposite to the first direction is mapped to the vehicle braking command. In the dual-pedal mode, a linear slide along the first direction in response to the user's operation of the acceleration start button is mapped to a vehicle acceleration command, and a linear slide along the second direction in any area of ​​the vehicle speed control interface (22) is mapped to a vehicle braking command.

5. The vehicle environment interaction interface module (5) according to claim 1, characterized in that: It includes an environmental display unit (51) and a real-time transmission unit (52); The environment display unit (51) is configured to receive the video stream transmitted from the vehicle end and synchronously render and display the vehicle end surround view image within the vehicle environment interaction interface (23). The real-time transmission unit (52) is configured to manage the video data transmission link and control the delay of the video stream from the vehicle end to the mobile end to be below a reasonable preset threshold.

6. The gear information interaction interface module (6) according to claim 1, characterized in that: The gear information interaction interface (24) provides a gear switching function, allowing users to select and switch between parking gear (P), reverse gear (R), neutral gear (N) and drive gear (D), and distinguish and display the currently active gear status through different visual indicators; The gear information interaction interface module (6) includes safety interlock logic, which monitors the vehicle's driving speed signal and braking system status signal in real time. When the vehicle speed signal is detected to be zero and the braking status signal is valid, the gear switching function is unlocked, allowing the user to perform gear change operations.

7. The instrument information interaction interface module (7) according to claim 1, characterized in that: The instrument information interaction interface (25) provides an integrated vehicle control panel, enabling centralized control of vehicle equipment, including mode switching of the vehicle lighting group (71), and start / stop control and operation parameter adjustment of the air conditioning system (72), wiper system (73), and window defrosting system (74). The instrument information interaction interface (25) integrates and displays key vehicle operating status parameters, including the numerical display of real-time vehicle speed and the activation status indication of intelligent driving mode.

8. The function setting interactive interface module (8) according to claim 1, characterized in that: The system parameter configuration function is provided through the interactive interface (26) of the aforementioned function settings; The vehicle self-check and diagnostic function (81) automatically executes the whole vehicle self-check process when the system starts, and provides a corresponding fault query interface and fault description information when a fault is detected; The safe exit function (82) terminates the current remote driving session after responding to the user's operation and verifying that the vehicle is in a safe state; The air conditioning parameter adjustment function (83) provides comprehensive control over the air volume level, target temperature and internal / external circulation mode of the air conditioning system; Driving trajectory management function (84), including enabling or disabling real-time recording and storage of vehicle driving trajectory; Connection mode switching function (85) supports selection and switching between near-end remote control mode and far-end remote control mode; The drive mode configuration function (86) is used to set the working mode of the vehicle speed control interface module (4) to single-pedal mode or dual-pedal mode. The user account management function (87) supports the modification and updating of user personal information and login credentials.

Images