Bus remote driving control method and system

Abstract

The invention provides a control method and system for remote driving of a bus, and is applied to the field of data processing. According to the method, the full-process original data containing multiple camera pictures, control instructions and the like are obtained, and operations such as picture calibration and instruction standardization are completed during analysis. And then adding a collaborative regulation factor through adaptive optimization, and encoding and embedding an identifier to generate to-be-executed data. Then, transmission and execution priorities are judged, scheduling is deployed after queue data are formed, and stable operation data are generated. And finally, resource allocation is optimized, a parallel processing result is obtained, and stability and high efficiency of remote driving are guaranteed in the whole process.

Description

Technical Field

[0001] This invention relates to the field of data processing, and in particular to a control method and system for remote bus driving. Background Technology

[0002] For the needs of large public transportation vehicles such as buses, existing technologies still have significant shortcomings, mainly in the following aspects: Insufficient adaptability of multi-source data processing: The data involved in remote bus driving, such as camera footage, vehicle status, and control commands, are heterogeneous. Existing parsing methods are mostly designed for single types of data and lack a collaborative adaptation mechanism for the entire process. For example, camera angle calibration and network routing configuration are performed independently without considering the timing of data processing. This makes it difficult for the parsed data to directly support a coherent driving process, easily leading to information gaps.

[0003] Low data transmission and execution efficiency: The existing system does not prioritize processed data or embed process identifiers, which can easily lead to excessive bandwidth consumption and network congestion when multiple types of data are transmitted synchronously. Especially in scenarios where multiple buses are remotely controlled simultaneously, the increased latency in data transmission of control commands and status feedback results in delayed driver response and affects operational stability.

[0004] Lack of end-to-end collaborative scheduling capability: Existing technologies lack a dynamic resource allocation mechanism for the entire process from data parsing to execution feedback, do not introduce process coordination adjustment factors to adapt and optimize each link, and do not establish an end-to-end monitoring and dynamic coordination mechanism covering data processing, transmission, and execution. When network fluctuations or equipment malfunctions occur, the data flow path cannot be quickly adjusted, which can easily lead to system interruption.

[0005] Poor integration between security authentication and data adaptation: Security steps such as SOCKET direct connection parameter configuration and cloud server authentication lack deep collaboration with subsequent data parsing and transmission processes. The existing verification mechanism only focuses on the legality of the connection and does not optimize the routing configuration according to the needs of driving scenarios. This results in a mismatch between the data transmission path after security authentication and the real-time driving requirements, further increasing response latency.

[0006] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore includes information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] According to one aspect of this application, a control method for remote bus driving is provided, comprising: acquiring raw data of the entire remote driving process and parsing the raw data to generate parsed remote driving information, wherein the raw data of the entire remote driving process includes multi-channel camera image acquisition data, control command data, vehicle status data, and network connection data, including the vehicle's unique license plate number, cloud server authentication information, driver's cab IP address, and SOCKET direct connection transmission parameters; the parsing operation includes camera image angle calibration and encoding preprocessing, control command format standardization parsing, real-time extraction of vehicle status parameters, network connection authentication information verification, and SOCKET direct connection routing configuration; and performing full-process adaptation optimization on the parsed remote driving information. The process involves: acquiring the target data stream supporting the complete remote driving workflow, adding workflow coordination adjustment factors to generate optimized data; encoding workflow modules and embedding full-link identifiers into the optimized data to generate execution data with workflow-specific identifiers; performing full-process transmission and execution priority determination on the execution data with workflow-specific identifiers to generate sub-process priority queue data; deploying and coordinating the sub-process priority queue data to generate stable operating data supporting the complete remote driving workflow; monitoring and dynamically coordinating the stable operating data to generate a continuous and controllable data stream covering the entire remote driving workflow; and allocating resources and optimizing the efficiency of the continuous and controllable data stream to generate parallel processing results.

[0008] Another aspect of this application discloses a control device for remote bus driving, characterized by comprising: an acquisition module for acquiring raw data of the entire remote driving process and parsing the raw data to generate parsed remote driving information; a processing module for performing full-process adaptation and optimization processing on the parsed remote driving information, acquiring a target data stream supporting the complete remote driving workflow, adding a process coordination adjustment factor, and generating optimized data; performing process module encoding and full-link identifier embedding processing on the optimized data to generate execution data with process-specific identifiers; performing full-process transmission and execution priority determination processing on the execution data with process-specific identifiers to generate sub-process priority queue data; performing full-process deployment and collaborative scheduling processing on the sub-process priority queue data to generate stable operating data supporting the complete remote driving workflow; performing full-process monitoring and dynamic output coordination processing on the stable operating data to generate a continuous and controllable data stream covering the entire remote driving process; and performing resource allocation and efficiency optimization processing on the continuous and controllable data stream to generate parallel processing results.

[0009] According to another aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a second processor, implements the aforementioned control method for remote bus driving.

[0010] This application provides a control method and system for remote bus driving. It acquires raw data from the entire process, including images from multiple cameras and control commands, and processes this data to generate standardized information. This information is then adapted, optimized, and supplemented with a collaborative adjustment factor to generate optimized data. Next, identifiers are embedded to obtain data to be executed, and transmission and execution priorities are determined to form a queue of data. Subsequently, stable operational data is generated through deployment and scheduling, and a continuous and controllable data stream is monitored and coordinated. Finally, resource allocation is optimized to generate parallel processing results. By prioritizing viewpoints, optimizing transmission through direct SOCKET connections, and implementing tiered scheduling based on functional urgency and safety risks, along with end-to-end anomaly monitoring and dynamic adjustments, multi-source data collaboration and efficient resource utilization are achieved.

[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0012] Figure 1 This invention provides a flowchart illustrating a control method for remote bus driving according to an embodiment of this application. Figure 2 A schematic diagram of the structure of a control device for remote driving of a bus provided in one embodiment of this application is shown. Detailed Implementation

[0013] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0014] The following is combined with Figure 1 This application describes a control method for remote bus driving according to an exemplary embodiment.

[0015] S101: Obtain the raw data of the entire remote driving process and parse and process the raw data of the entire remote driving process to generate parsed remote driving information.

[0016] In one implementation, multiple dedicated cameras at the front, rear, left, and right of the bus are used to capture environmental images in real time. Preferably, the camera data needs to obtain specific parameters of four cameras (front-view camera, left / right-view camera, and rear-view camera, their respective layout positions, horizontal angles, and viewing angles) to ensure that environmental details are clearly discernible.

[0017] The control command data includes bidirectional commands issued by the driver's cockpit and responded to by the vehicle during remote driving. Driver-issued commands include commands to switch between remote / automatic / manual modes (e.g., "remote mode activated" command), brake / accelerator quantization commands (brake amount quantized from 0-1000, e.g., "brake amount 500" corresponds to 50% braking force), and emergency stop commands (triggered when network latency exceeds 500ms or radar detection distance is <50cm). Vehicle-response commands include command reception confirmation commands (e.g., "remote mode switching command received") and execution result feedback commands (e.g., "brake amount 500 executed, current vehicle speed reduced to 30km / h"). The command transmission format is uniformly JSON to ensure accurate and unambiguous command interaction.

[0018] Vehicle status data is collected in real time from bus chassis sensors and onboard industrial control computer, including core operating parameters such as dynamic driving parameters (vehicle speed, throttle opening, braking status, gear position), hardware load parameters (industrial control computer CPU utilization, memory usage, disk storage, CPU temperature), sensor status parameters (GPS positioning data, millimeter-wave radar distance data, ultrasonic radar distance data, lidar working status), and energy status parameters (battery charge, power consumption rate, charging status). The data collection frequency is differentiated according to the importance of the parameters: dynamic driving parameters and sensor status parameters are collected every 100ms, and hardware load parameters and energy status parameters are collected every 500ms to ensure real-time reflection of the vehicle's operating status.

[0019] Network connection data covers key configuration and status data in the vehicle-cloud-warehouse connection process, including vehicle identification data (unique vehicle number, such as "CN65**27"), cloud server authentication data (cloud server access key and authentication certificate pre-stored in the vehicle), cockpit connection configuration data (cockpit IP address, such as "192.168.1.100"), and SOCKET direct connection transmission parameters (transmission bandwidth threshold, latency monitoring frequency, packet loss rate threshold, and direct connection routing configuration information). The data is obtained in real time through interaction between the vehicle-mounted CPE (4G / 5G communication module) and the cloud server to ensure accurate and effective network connection configuration.

[0020] The above four types of raw data are parsed and processed according to the principles of "standardization, precision, and usability" to generate remote driving information with a clear structure and uniform format. The specific parsing operations are as follows: For camera image angle calibration and encoding preprocessing, for the raw images collected by the four cameras, the installation deviation is first corrected by a preset angle calibration algorithm (e.g., the left-view camera has a 5° offset due to vehicle body obstruction, and the horizontal offset of the image is corrected to 0° by the algorithm) to ensure that the image view is consistent with the actual road conditions; then encoding preprocessing is performed, and the image is compressed using the H.265 encoding format, while retaining the high definition of key areas of the image (such as lane lines and traffic light areas in the front view). The bit rate of the encoded image is controlled at 2-4Mbps to adapt to the bandwidth requirements of WebRTC direct connection transmission and avoid excessive transmission latency.

[0021] For standardized parsing of control command formats, the original control commands undergo format verification and content parsing. Commands are encoded in byte format. First, the command header identifier (e.g., 0Xxxxx) is verified, and a checksum is used to eliminate commands with format errors or those that have been tampered with. Then, the core information of the command is extracted according to preset parsing rules, parsed as "Driver's cabin issues braking command, such as: braking amount 500, command generation time October 11, 2025, 14:23". Simultaneously, the parsed command is bound to the vehicle's unique license plate number to prevent mismatches between commands and other vehicles.

[0022] Vehicle status parameters are extracted in real time, and core parameters are filtered and standardized from the raw vehicle status data. For dynamic driving parameters, the vehicle speed unit is unified to km / h, and the braking status is converted to a "braked / not braked" Boolean value. For hardware load parameters, normal thresholds are set (CPU utilization < 80%, memory utilization < 90%, CPU temperature < 45℃), and parameters exceeding the thresholds are marked as "to be monitored". For sensor status parameters, GPS latitude and longitude coordinates (e.g., "30.1234°N, 120.5678°E") and the distance to the vehicle ahead detected by millimeter-wave radar (e.g., "60m") are extracted, and invalid data (e.g., "NA" data when LiDAR is not enabled) are removed to generate a structured vehicle status information table.

[0023] For network connection authentication information verification and SOCKET direct connection route configuration, first verify the matching of the vehicle's unique license plate number with the cloud server's authentication information (e.g., the license plate number "CN65**27" must match the vehicle information pre-stored in the cloud server, and the authentication key must be verified through MD5 encryption). After successful verification, the cloud server pushes the driver's cabin IP address to the vehicle. Then, configure the route based on the SOCKET direct connection transmission parameters, disable the intermediate server forwarding function, and directly establish a public network direct connection channel between the vehicle and the driver's cabin. Configure the direct connection bandwidth adaptive threshold (automatically reduce the frame rate to 25fps when the bandwidth is <2Mbps) and the latency monitoring frequency (check network latency every 100ms) to ensure a stable and low-latency network connection. Finally, generate network connection resolution information containing "license plate number - authentication result - driver's cabin IP - direct connection route configuration".

[0024] S102, perform full-process adaptation and optimization processing on the parsed remote driving information, obtain the target data stream that supports the complete remote driving workflow, add process coordination adjustment factors, and generate optimized data.

[0025] In one implementation, the parsed multi-camera image data is optimized for viewpoint priority and transmission adaptation to generate an image data stream adapted to the remote driving scenario. Specifically, the front-view, left / right-view, and rear-view image data are prioritized in the order of front-view > left / right-view > rear-view. Based on real-time network bandwidth and latency monitoring data, the resolution and frame rate of images in areas with insufficient bandwidth or excessive latency are dynamically adjusted, while also adapting to the WebRTC direct transmission protocol. For the parsed front-view, left / right-view, and rear-view camera image data, priorities are set according to the principle of "core driving viewpoint priority" and adapted to transmission requirements to generate an image data stream adapted to the remote driving scenario. The front-view image data is set to the highest priority (priority 1), the left / right view to the medium priority (priority 2), and the rear view to the low priority (priority 3). For example, the images of lane lines and traffic lights directly ahead, captured by the front-view camera (located inside the vehicle, with a horizontal angle of 45° and a viewing angle of 150°), are the core reference for remote driving control and are given priority for transmission; the images of pedestrians on both sides of the vehicle and vehicles in adjacent lanes, captured by the left / right-view cameras (located outside the vehicle, with a horizontal angle of 55° and a viewing angle of 130°), are secondary observation perspectives; and the images of following distance behind, captured by the rear-view camera (located outside the vehicle, with a horizontal angle of 45° and a viewing angle of 130°), have the lowest priority.

[0026] By monitoring network bandwidth and latency data in real time through the vehicle-mounted CPE, the screen resolution and frame rate are dynamically adjusted when bandwidth is <2Mbps or latency is >300ms. For example, in areas with weak signal (such as tunnels), when bandwidth drops to 1.5Mbps, the front-view screen resolution is reduced from 1080P to 720P while maintaining a frame rate of 30fps; the left / right views are reduced from 720P to 480P while maintaining a frame rate of 25fps; and the rear view is reduced from 480P to 360P while maintaining a frame rate of 20fps. When the network recovers (bandwidth ≥5Mbps, latency <100ms), the settings automatically return to the initial parameters. Simultaneously, the screen data is encapsulated in a WebRTC protocol-compatible format to support real-time streaming and avoid latency accumulation caused by intermediate server forwarding.

[0027] The execution logic and transmission path of the video data stream adapted for remote driving scenarios are optimized to generate a closed-loop command data stream. Specifically, the closed-loop logic of remote / automatic / manual driving mode switching commands, brake / accelerator quantification commands, and emergency stop commands is optimized according to the command generation-driver cabin issuance-vehicle reception and verification-execution feedback sequence. Commands are bound to the target vehicle based on the vehicle's unique license plate identifier. The command transmission path is optimized to rely on direct transmission between the vehicle and driver cabin via a SOCKET direct connection channel. Based on the adapted video data stream, the interaction logic and transmission path of control commands are optimized to generate a closed-loop, traceable command data stream. Three core command types are processed according to the "command generation-issuance-verification-feedback" closed-loop logic. Taking the remote / automatic mode switching command as an example, when the autonomous driving system encounters a road obstacle that it cannot handle, the vehicle chassis generates a "request remote takeover" command. After receiving the command, the driver confirms it and generates a "remote mode activation" command (command format: {vehicle number: CN65**27, command type: mode switching, target mode: remote, generation time: 202510111530}), which is then sent to the vehicle via the SOCKET direct connection channel. After receiving the command, the vehicle's industrial control computer verifies its validity, executes the mode switch, and generates a feedback command of "switched to remote mode, current vehicle speed 30km / h," which is then sent back to the driver's cabin, forming a closed loop. For brake / accelerator quantization commands, the braking amount and accelerator amount are quantized from 0 to 1000 (e.g., "brake amount 500" corresponds to 50% braking force), and the actual braking effect is fed back after execution. When an emergency stop command is triggered (e.g., network latency exceeds 500ms), the command priority is automatically upgraded to the highest level, and it is sent directly without driver confirmation, and the stop status is forcibly fed back after execution. Commands are bound to target vehicles based on their unique vehicle identification numbers (e.g., CN65**27) to prevent accidental command transmission. Command transmission relies on a direct SOCKET connection, disabling intermediate server forwarding and establishing a direct public network data transmission channel between the driver's cab (IP: 192.168.1.100) and the vehicle. For example, after an emergency stop command is issued from the driver's cab, it is transmitted directly to the vehicle's industrial control computer via the direct connection, with transmission time controlled within 100ms, far lower than the 500ms+ latency of the traditional "vehicle-server-cabin" model.

[0028] The parsed vehicle status data undergoes monitoring priority and anomaly filtering optimization to generate a monitoring data stream focusing on core states. From the parsed vehicle status data, core parameters are filtered and sorted by monitoring importance to generate a monitoring data stream focusing on critical states. "Dynamic driving parameters + hardware load parameters + key sensor status" are set as high-priority monitoring items, while "energy status parameters + non-core sensor status" are set as low-priority. For example, high-priority parameters include vehicle speed (reflecting real-time driving safety), industrial control computer CPU utilization (affecting instruction processing efficiency), millimeter-wave radar distance data (assessing collision risk), and GPS positioning data (confirming vehicle location), collected every 100ms and uploaded in real time; low-priority parameters include battery power consumption rate and LiDAR operating status (not necessary for remote driving), collected every 500ms and actively uploaded only in case of anomalies.

[0029] Set abnormal thresholds for parameters, remove invalid data, and mark abnormal items. For example, the normal threshold for CPU utilization is set to <80%. When the CPU utilization is detected to reach 85%, it is marked as "Warning Abnormal" and a "Suggest checking background programs" prompt is added. The normal threshold for millimeter-wave radar distance is set to >50cm. When the distance drops to 30cm, it is marked as "Severe Abnormal" and a pop-up reminder is triggered in the cockpit. For "NA" data when the LiDAR is not enabled and invalid coordinates when the GPS signal is lost, they are directly filtered to avoid consuming transmission resources.

[0030] The parsed network connection data is optimized for connection establishment efficiency and direct connection adaptation to generate the target network data stream. Specifically, the connection establishment logic is optimized based on the following process: the cockpit initiates a control request, sends the vehicle number, the cloud server verifies the vehicle number's validity, the cloud server obtains the cockpit IP and pushes the cockpit IP address to the vehicle, and the vehicle and cockpit connect directly via SOCKET. SOCKET direct connection routing parameters are configured to directly establish a public network data transmission channel between the vehicle and the cockpit. The network connection status is marked in real time. For the parsed vehicle identity, cloud server authentication, cockpit configuration, and other network data, the connection establishment process and direct connection parameters are optimized to generate a stable target network data stream. The connection establishment is simplified using a three-step process: "vehicle-cloud authentication-cockpit direct connection." For example, after the vehicle starts, it automatically sends a request containing a unique vehicle number (CN65**27) and a pre-stored authentication key to the cloud server. The cloud server verifies whether the vehicle number is in the controlled vehicle database and whether the key matches. After successful verification, when the cockpit initiates control of the vehicle, the cloud server pushes the cockpit IP address (192.168.1.100) to the vehicle. After receiving the IP, the vehicle configures a direct connection route based on the SOCKET protocol and establishes a direct public network connection with the cockpit. The number of retry attempts for connection timeout is set to 3 (with a 2-second interval between each attempt) to ensure rapid connection establishment.

[0031] Configure core SOCKET direct connection parameters, including transmission bandwidth threshold (2-8Mbps), latency monitoring frequency (every 100ms), and packet loss rate threshold (<5%), and mark the network connection status in real time. For example, after a successful direct connection, mark it as "online" and simultaneously record the current bandwidth (4Mbps) and latency (80ms); when network fluctuations cause the latency to rise to 400ms, mark it as "unstable connection"; when the network is interrupted, mark it as "offline" and trigger the vehicle's local emergency stopping plan.

[0032] The optimized multi-type data is then integrated and processed with adjustment factors to generate optimized data. This involves integrating the video, command, vehicle status, and network connection data streams according to the entire remote driving process timeline. Process coordination adjustment factors are added, including network latency grading thresholds, dynamic camera view adjustment parameters, and hardware anomaly response levels. The optimized video, command, vehicle status, and network connection data streams are then integrated in timeline, and adjustment factors are added to generate unified optimized data. Multi-type data are associated according to the entire remote driving process timeline of "network connection establishment → video transmission → command interaction → status monitoring." For example, after successful network connection (time T0), video transmission begins (T0+1s), the driver's cabin issues commands based on the video (T0+5s), the vehicle executes the commands and provides status feedback (T0+5.1s), and the monitoring data stream synchronizes the command execution effect in real time (T0+5.2s), ensuring that data flows orderly according to the driving process and avoiding timeline errors.

[0033] Three types of adjustment factors are incorporated to achieve dynamic adaptation. First, network latency grading thresholds (warning: 300-500ms, emergency: >500ms): when latency reaches the warning threshold, the screen resolution is reduced. Second, dynamic adjustment parameters for camera view (forward gear: 70% front view, reverse gear: 70% rear view): when the vehicle switches to reverse gear, the rear view image is automatically prioritized. Third, hardware anomaly response levels (minor: CPU <85%, severe: CPU >90%): minor anomalies are only indicated, while severe anomalies prohibit remote driving, ensuring the system can flexibly adjust according to actual operating conditions.

[0034] S103, perform process module coding and full-link identifier embedding on the optimized data to generate execution data with process-specific identifiers.

[0035] In one implementation, optimized multi-camera image data streams are input into a viewpoint priority encoding module. The image data is then format-standardized using preset encoding rules to generate image encoding data compatible with WebRTC direct transmission. Optimized front-view, left / right-view, and rear-view image data streams are input into the viewpoint priority encoding module, and format standardization is performed according to preset rules to generate image encoding data compatible with WebRTC direct transmission. The encoding rules include a binding logic for four elements: "viewpoint identifier - resolution - frame rate - transmission protocol." The viewpoint identifier uses a 2-bit binary code (front view: 01, left view: 10, right view: 11, rear view: 00). Resolution and frame rate are matched with fixed parameters according to viewpoint priority (front view defaults to 1080P / 30fps, left / right view 720P / 25fps, rear view 480P / 20fps). Simultaneously, it encapsulates the SDP (Session Description Protocol) information required by the WebRTC protocol, including the encoding format (H.265), transmission bandwidth adaptation range (2-8Mbps), and NACK (Negative Acknowledgment) retransmission mechanism configuration.

[0036] Taking the front-view video data stream as an example, the module first reads the original video parameters (resolution 1080P, frame rate 30fps, view type "front"), automatically adds the view identifier "01", encapsulates SDP information (v=0\r\no = - 12345 1 IN IP4192.168.1.100\r\ns = WebRTC Video Stream\r\nm = video 5004 UDP / TLS / RTP / SAVPF96\r\nc =IN IP4 192.168.1.100\r\na=rtpmap:96 H265 / 90000\r\na =fmtp:96profile-level-id = 42E01F;level-asymmetry-allowed = 1; packetization-mode =1), and finally generates standardized video encoding data (format: [view identifier][SDP information][video compression data]), ensuring that the WebRTC direct connection channel can directly recognize and decode the transmission.

[0037] The optimized closed-loop command data stream is input into the command binding encoding module. Based on the vehicle's unique license plate number, the command data is associated and encoded to generate command encoded data with a unique binding identifier. The optimized closed-loop command data streams for remote / automatic mode switching, brake / accelerator quantization, and emergency stop are also input into the command binding encoding module. Based on the vehicle's unique license plate number, association encoding is performed to generate command encoded data with a unique binding identifier. The encoding module first extracts the command type (e.g., "remote mode activation," "brake quantity 500") and generation timestamp from the command data stream. Then, it encrypts the vehicle's unique license plate number (e.g., "CN65**27") using MD5 to generate a 16-bit license plate identifier. The command is encapsulated in the format "license plate identifier - command type code - timestamp - command parameters - checksum," where the command type code uses a 1-digit decimal (mode switching: 1, brake / accelerator: 2, emergency stop: 3). The checksum is calculated using the CRC32 algorithm to ensure command integrity.

[0038] Taking "Vehicle CN65**27 receives remote mode activation command" as an example, the module first encrypts the vehicle number "CN65**27" into "8A3F5C7E9D1B2C4D", the command type "mode switch" corresponding code "1", the timestamp "20251011163000", the command parameter "target mode: remote", and the checksum "0x12F8", and finally generates command encoded data (format: 8A3F5C7E9D1B2C4D-1-20251011163000-target mode: remote-0x12F8), ensuring that the command is uniquely bound to the target vehicle and avoiding accidental transmission to other vehicles.

[0039] The optimized core status monitoring data stream is input into the anomaly level coding module. The monitoring data is categorized and coded according to hardware load and safety risk levels, generating monitoring coded data with warning indicators. The optimized core monitoring data streams, including vehicle dynamics, hardware load, and sensor status, are also input into the anomaly level coding module. These data are then categorized and coded according to hardware load and safety risk levels, generating monitoring coded data with warning indicators. Hardware load levels are divided into three levels (Minor: Load < 80%, Warning: 80%-90%, Severe: > 90%), and safety risk levels are divided into three levels (Low Risk: Normal parameters, Medium Risk: Approaching threshold, High Risk: Exceeding threshold). Each level corresponds to a unique warning indicator (Minor / Low Risk: Green 00, Warning / Medium Risk: Yellow 01, Severe / High Risk: Red 10). Anomaly handling suggestions are also associated (e.g., a "Do Not Remotely Drive" suggestion is linked to the severe risk indicator).

[0040] Taking industrial control computer CPU monitoring data as an example, if the real-time CPU utilization rate is 85% (hardware load "warning" level, safety risk "medium risk"), the module automatically adds a warning flag "01" and associates it with the processing suggestion "prompt the driver to check the background program", and finally generates monitoring code data (format: [parameter type][real-time value][warning flag][processing suggestion], such as "CPU-85%-01-prompt the driver to check the background program"); ​​if the millimeter-wave radar detection distance is 30cm (safety risk "high risk"), then a flag "10" is added and associated with the suggestion "trigger emergency braking command", to ensure that abnormal conditions can be quickly identified and responded to.

[0041] The optimized target network data stream is input into the direct connection status encoding module. Combined with the SOCKET direct connection routing configuration parameters, the network data undergoes transmission adaptation encoding to generate network encoded data with connection status identifiers. The optimized vehicle-cloud authentication, driver's cabin IP, SOCKET direct connection parameters, and other target network data streams are input into the direct connection status encoding module. Combined with the direct connection routing configuration parameters, transmission adaptation encoding is completed to generate network encoded data with connection status identifiers. The encoding module first reads the SOCKET direct connection routing configuration parameters (such as driver's cabin IP "192.168.1.100", direct connection port "5004", route type "public network direct connection", connection status "online / offline / unstable"), encapsulates them in the format "connection status identifier-IP-port-routing parameters", where the connection status identifier uses a 1-character string (online: O, offline: F, unstable: U), and also embeds direct connection timeout retry parameters (such as retry count "3 times", interval "2s").

[0042] Taking a successful vehicle-cabin direct connection as an example, the module reads the connection status "online", the driver's cabin IP "192.168.1.100", the port "5004", and the routing parameter "disable intermediate server forwarding", and generates network encoded data (format: [status identifier][IP:port][routing parameter][retry parameter], such as "O-192.168.1.100:5004-disable intermediate server forwarding-retry 3 times / 2s"). If the network latency rises to 400ms (status "unstable"), the identifier is changed to "U", and the routing parameter "dynamically adjust bandwidth to 4Mbps" is added to ensure that changes in network status can be fed back to the transmission link in real time.

[0043] The above four types of coded data are input into the end-to-end identifier integration module. The identifier mapping rules associate and integrate the unique identifiers of each data type to generate execution data with process-specific identifiers. The above four types of coded data (screen, command, monitoring, and network) are input into the end-to-end identifier integration module. The identifier mapping rules associate the unique identifiers of each data type to generate execution data with process-specific identifiers. The rules define a three-dimensional integration logic of "global timestamp - unique vehicle number - data type code". The global timestamp uses a 13-bit millisecond-level time (e.g., "1740000000000") to ensure time sequence alignment of multiple data types; the unique vehicle number reuses the 16-bit encrypted identifier from the command encoding (e.g., "8A3F5C7E9D1B2C4D") to bind data to the vehicle; the data type code uses a 1-bit binary (screen: 00, command: 01, monitoring: 10, network: 11) for easy process identification.

[0044] The module reads the unique identifiers of four types of coded data (screen: viewpoint identifier "01", command: vehicle number identifier "8A3F5C7E9D1B2C4D", monitoring: warning identifier "01", network: status identifier "O"), generates a global timestamp "17400000000000", integrates them according to the format "[global timestamp][vehicle number identifier][data type code][unique identifier][coded data]", and finally generates the data to be executed (e.g., screen data is 1740000000000 - 8A3F5C7E9D1B2C4D-00-01-[standardized screen coded data]; command data is 1740000000000 - 8A3F5C7E9D1B2C4D-01-1-[command coded data]), realizing full-process data traceability and collaborative scheduling.

[0045] S104 performs full-process transmission and execution priority determination processing on the data to be executed with process-specific identifiers, and generates sub-process priority queue data.

[0046] In one implementation, the data to be executed, bearing a process-specific identifier, is input into a transmission priority determination module. The data is then categorized and sorted according to preset transmission priority rules to generate a preliminary sorted data to be transmitted. The data to be executed (including four types of coded data: screen, command, monitoring, and network) bearing process-specific identifiers is input into the transmission priority determination module, and categorized and sorted according to preset rules to generate a preliminary sorted data to be transmitted. The preset transmission priority rules are set according to a dual dimension of "data type - security relevance." Data with higher security relevance and more critical to driving decisions has a higher transmission priority. Specifically, the priorities are divided into three levels from high to low: Level 1 (Safety Emergency): Emergency stop commands, network offline status data, millimeter-wave radar near-field warning data; Level 2 (Driving Core): Remote / automatic mode switching commands, front-view screen data, dynamic driving data such as vehicle speed / braking amount; Level 3 (Auxiliary Reference): Left / right / rear-view screen data, hardware load data such as CPU / memory, customized task progress data.

[0047] Taking the data to be executed at a certain moment as an example, the module first reads the process-specific identifier of each data (including data type code and security association identifier), identifies "Emergency Stop Command (Level 1)", "Network Online Status Data (Level 3)", "Front View Image Data (Level 2)", "Millimeter Wave Radar 30cm Warning Data (Level 1)" and "Left View Image Data (Level 3)", and sorts them from high to low priority as follows: Emergency Stop Command → Millimeter Wave Radar 30cm Warning Data → Front View Image Data → Network Online Status Data → Left View Image Data, and generates a preliminary sorted list of data to be transmitted.

[0048] The initially sorted data to be transmitted is input into the network status adaptation module. Combined with real-time network bandwidth and latency data, the transmission priority is dynamically adjusted to generate network-adapted data to be transmitted. The module presets network status thresholds (bandwidth ≥ 5Mbps, latency < 100ms is "Excellent"; 2Mbps ≤ bandwidth < 5Mbps, 100ms ≤ latency < 300ms is "Medium"; bandwidth < 2Mbps, latency ≥ 300ms is "Poor"). Data transmission priority is adjusted according to the network status level: When the network is "Excellent," the initial sorting is maintained; when the network is "Medium," the transmission priority of third-level data is reduced (e.g., left / right view image data is delayed); when the network is "Poor," only first-level data is prioritized for transmission, third-level data transmission is paused, and second-level data is further sorted according to "front view image > mode switching command > dynamic driving data."

[0049] If the real-time network status is "medium" (bandwidth 3Mbps, latency 200ms), for the initial sorted data, the module postpones the third-level "left-view image data" and "network online status data" to the end of the queue, while the second-level "front-view image data" remains in the second order. The final adjusted sorting is: emergency stop command → millimeter-wave radar 30cm warning data → front-view image data → left-view image data → network online status data, generating network-adapted data to be transmitted.

[0050] The network-adapted data to be transmitted is input into the execution priority determination module. Based on the urgency of the data function and the level of security risk, an execution priority is set, generating data to be scheduled containing execution priority identifiers. The urgency of the function is divided according to "immediate response requirements" ("high urgency" requires execution within 100ms, "medium urgency" requires execution within 500ms, and "low urgency" has no strict time requirement); the security risk level is divided according to "risk impact range" ("high risk" affects vehicle safety, "medium risk" affects driving experience, and "low risk" has no direct impact). The execution priority identifier uses a combination of "urgency level - risk level" coding (e.g., high urgency - high risk: 01, medium urgency - medium risk: 10, low urgency - low risk: 11), with a smaller code indicating a higher execution priority.

[0051] Taking the "Emergency Stop Command" as an example, its urgency level is "High Urgency" (must be executed within 50ms) and its safety risk level is "High Risk" (failure to execute will result in a collision risk), so the module assigns it the identifier "01". The "Front View Image Data" function has an urgency level of "Medium Urgency" (must be displayed within 300ms) and a safety risk level of "Medium Risk" (delay will affect road condition judgment), so it is assigned the identifier "10". The "Left View Image Data" function has an urgency level of "Low Urgency" (must be displayed within 1000ms) and a safety risk level of "Low Risk" (delay has minimal impact on driving), so it is assigned the identifier "11". Finally, scheduling data containing execution priority identifiers is generated.

[0052] The data to be scheduled, containing execution priority identifiers, is input into the sub-process queue generation module. The module divides the queues according to both the transmission and execution processes, generating sub-process priority queue data. The two-dimensional division logic is as follows: the transmission process is divided into a "vehicle-to-cabin direct transmission queue" (data that needs to be transmitted to the driver's cab in real time) and a "local cache queue" (data temporarily stored in the vehicle's industrial control computer when the network is poor) based on the "data transmission link"; the execution process is divided into a "driver's cab execution queue" (data that requires a driver's cab response, such as instructions and warnings) and a "vehicle local execution queue" (data that needs to be processed autonomously by the vehicle, such as hardware load monitoring) based on the "data processing subject". Each queue is further sorted by execution priority identifiers to ensure that high-priority data is transmitted and executed first.

[0053] Taking the data to be dispatched with identifiers (emergency stop command "01", front view image "10", left view image "11", millimeter-wave radar warning "01") as an example, the module first divides the data according to the transmission process: the emergency stop command, front view image, and millimeter-wave radar warning need to be transmitted to the driver's cabin in real time and are included in the "vehicle-cabin direct connection transmission queue"; the left view image is temporarily stored in the network and is included in the "local cache queue"; then it is divided according to the execution process: the emergency stop command and millimeter-wave radar warning need to be confirmed and executed by the driver's cabin and are included in the "driver's cabin execution queue" (sorted: emergency stop command → millimeter-wave radar warning); the front view image needs to be displayed in the driver's cabin and the left view image needs to be cached locally and are included in the "driver's cabin execution queue (image display sub-queue)" and the "vehicle local execution queue (cache sub-queue)" respectively. Finally, a priority queue data with four sub-queues is formed to ensure that the data of each process is transmitted and executed in an orderly manner.

[0054] S105 performs full-process deployment and collaborative scheduling processing on the priority queue data of the sub-processes to generate stable operating data that supports the complete workflow of remote driving.

[0055] In one implementation, network connection data blocks in the priority queue are directly deployed to generate stable connection data for the vehicle-warehouse direct connection channel. The deployment process follows a sequence: the driver initiates a control request, sends the vehicle number, the cloud server verifies the vehicle number's validity, the cloud server obtains the driver's IP address and pushes the driver's IP address to the vehicle, the vehicle configures a direct connection route based on the SOCKET protocol, and the vehicle and warehouse establish a public network direct connection. The deployment unit is the vehicle number and IP address binding pair, and the number of connection timeout retries is calculated based on SOCKET direct connection parameters. The relevant data blocks for "vehicle-cloud-warehouse" network connection (including the vehicle's unique number, cloud server authentication key, and driver's IP configuration information) in the priority queue are deployed according to a fixed process to establish a stable direct connection channel, generating stable connection data for the vehicle-warehouse direct connection channel.

[0056] The process follows a four-step procedure: "Vehicle-Cloud Authentication → IP Acquisition → Direct Connection Configuration → Channel Establishment." First, after the vehicle's industrial control computer starts, it automatically encapsulates a connection request containing a unique vehicle number (e.g., "CN65**27") and a pre-stored cloud server authentication key (e.g., "8A3F5C7E9D1B2C4D"), and sends it to the cloud server via the vehicle's CPE. Second, upon receiving the request, the cloud server verifies whether the vehicle number is in the preset control vehicle database (e.g., checking if "CN65**27" belongs to a registered vehicle) and whether the authentication key is verified using MD5 encryption. If verification is successful, the vehicle-cloud connection status is stored. Third, the driver's cabin initiates a remote control request for the vehicle. Fourth, after receiving the IP address pushed by the cloud server, the vehicle configures direct connection routing parameters based on the SOCKET protocol, setting the transmission bandwidth threshold (2-8Mbps) and packet loss rate threshold (<5%). Fifth, the vehicle sends a SOCKET direct connection request to the driver's cabin. Upon receiving the request, the driver's cabin verifies the legitimacy of the vehicle number and establishes a public network direct connection channel.

[0057] Combining SOCKET direct connection parameters (such as a default connection timeout of 1000ms and a network latency monitoring value of 200ms), the number of retries is calculated using the formula "Number of retries = (Maximum tolerable connection timeout - Single connection timeout) ÷ Single connection timeout". If the maximum tolerable connection timeout is 3000ms and the single timeout is 1000ms, then the number of retries = (3000-1000) ÷ 1000 = 2 times, ensuring that the connection can still be rebuilt during network fluctuations. The final generated stable connection data includes information such as "Vehicle number - Driver's cabin IP - Direct connection parameters - Connection status (online / offline) - Number of retries".

[0058] The camera image data blocks in the priority queue are processed for view scheduling to generate image output data adapted to the driving scenario. Front view, left / right view, and rear view image data blocks (including encoded image streams and view priority identifiers) in the queue are scheduled, and the image output strategy is dynamically adjusted according to the driving scenario to generate image output data adapted to driving needs. Based on the vehicle's current driving status (gear, speed) and driving scenario requirements, the image display priority and output format are adjusted. When the vehicle is in forward gear (speed > 0 km / h), the front view image is set as the main display image (occupying 70% of the cockpit display area), the left / right view images are set as auxiliary displays (each occupying 15% of the area), and the rear view image is minimized to the corner. When the vehicle switches to reverse gear (speed < 0 km / h), the rear view image priority is automatically increased (occupying 60% of the display area), the front view image is reduced to 20%, and the left / right views remain at 10% each. Simultaneously, combined with ultrasonic radar distance data (e.g., left view radar detection distance < 50cm), a red warning border is added to the edge of the corresponding view image to alert the driver to surrounding obstacles.

[0059] At a certain moment, the vehicle is moving forward at 30km / h. The front view image encoding data is "01-720P-30fps-H.265" (view identifier 01 represents the front view). The radar detection distance of the left view is 40cm. The dispatch module displays the front view image in full screen on the main display in the driver's cabin, adds a red border to the left view image and displays it on the right auxiliary screen, and compresses the right view and rear view images to the bottom status bar. Finally, it generates image output data adapted to the forward driving scenario to ensure that the driver focuses on the core road conditions.

[0060] The system performs execution scheduling on control command data blocks in the priority queue, generating stable control data for closed-loop command execution. It also schedules remote control command data blocks (including mode switching, brake / accelerator quantization, emergency stop, etc.) in the queue, ensuring command execution and feedback according to closed-loop logic, generating stable control data for closed-loop command execution. This follows a closed-loop process of "command issuance → reception and verification → execution feedback → status synchronization". Taking the "Brake Amount 500" command as an example, the first step is that after the driver's cab generates the command, it is encapsulated in the format of "Vehicle Number-Command Type-Parameter-Timestamp" (e.g., "CN65**27-Brake-500-202510111630") and sent to the vehicle through the SOCKET direct connection channel. The second step is that after the vehicle's industrial control computer receives the command, it verifies whether the vehicle number is consistent with the local vehicle number and whether the command format conforms to the JSON standard (e.g., checking whether the "Brake Amount" parameter is within the range of 0-1000). After verification, it sends a "Command Receipt Confirmation" feedback to the driver's cab. The third step is that the vehicle chassis performs the braking operation, converting the brake amount of 500 into 50% braking force, and then collects the current vehicle speed (e.g., reducing from 30km / h to 20km / h). The fourth step is that the vehicle sends the result "Brake execution completed, current vehicle speed 20km / h" back to the driver's cab, and the driver's cab updates the command execution status after receiving it.

[0061] If the instruction execution times out (e.g., no execution feedback is received within 500ms), or the execution result does not match the instruction parameters (e.g., the instruction requires a braking amount of 500, but only 300 is actually executed), the scheduling module automatically triggers instruction retransmission, with the number of retransmissions not exceeding 3, to ensure reliable instruction execution. The final generated stable control data includes "instruction content - execution result - feedback time - abnormal status (none / timeout / mismatch)".

[0062] The system monitors and schedules vehicle status data blocks in the priority queue to generate real-time controllable status monitoring data. It performs real-time scheduling of vehicle status data blocks (including dynamic driving, hardware load, and sensor status data) in the queue, filters core parameters, and synchronizes them to the driver's cabin to generate real-time controllable status monitoring data. The process follows the workflow of "parameter filtering → real-time acquisition → anomaly marking → synchronous display." The first step is to filter core monitoring parameters from the raw vehicle status data. High-priority parameters (vehicle speed, braking status, millimeter-wave radar distance, network latency) are collected every 100ms, while low-priority parameters (CPU temperature, memory usage) are collected every 500ms. The second step is to acquire parameter values ​​in real time through onboard sensors and the industrial control computer, such as vehicle speed 30km / h, CPU temperature 35℃, and millimeter-wave radar distance to the vehicle in front 60m. The third step is to compare the parameters with preset normal thresholds (such as vehicle speed ≤ 60km / h, CPU temperature < 45℃, radar distance > 50cm). Parameters exceeding the threshold are marked as "to be monitored" (e.g., when the radar distance drops to 40cm, it is marked as "medium risk"). The fourth step is to encapsulate the filtered status data in the format of "parameter type - real-time value - status (normal / abnormal)" and synchronize it to the driver's cabin monitoring interface via a direct SOCKET connection.

[0063] At a certain moment, the system collects data showing "vehicle speed 35km / h (normal), CPU temperature 42℃ (normal), millimeter-wave radar distance 45cm (medium risk), network latency 250ms (normal)". The scheduling module marks this set of data as "containing medium risk parameters" and prioritizes its synchronization to the driver's cabin. The radar distance "45cm (medium risk)" is displayed in yellow on the interface, along with a "Pay attention to the distance of the vehicle in front" prompt. The generated status monitoring data can reflect the vehicle's operational health in real time.

[0064] The deployed and scheduled "stable connection data, screen output data, stable control data, and status monitoring data" are then collaboratively integrated according to the time sequence and logic of the entire remote driving process to generate stable operational data that supports the complete remote driving workflow. "Timestamp + vehicle number" is used as the association key to ensure that the time sequence of multiple data types is aligned and bound to the vehicle. The first step is to extract the global timestamp (e.g., 1740000000000ms) and the unique vehicle number (CN65**27) of each data block, and sort them in order of timestamp. The second step is to associate the direct connection status with the image transmission (e.g., in the "direct connection online" state, the front view image and left view warning information in the synchronous image output data) and associate the command execution with the status feedback (e.g., "vehicle speed decrease" in the status data corresponding to "brake command execution completed"). The third step is to remove duplicate or invalid data (e.g., duplicate connection request data after the network connection has been established) and integrate it according to the remote driving process logic of "network status → image data → command data → status data". The fourth step is to generate structured stable operation data, including fields such as "timestamp-vehicle number-direct connection status-image output configuration-command execution record-status monitoring result", to ensure that the data is traceable and collaborative.

[0065] At timestamp 1740000000000ms, the integrated data is “1740000000000-CN65**27-Direct connection online-Front view 720P / Left view warning-Brake amount 500 completed-Vehicle speed 20km / h / Radar distance 45cm”, which fully reflects the operating status of the remote driving system at that moment and provides a basis for subsequent monitoring and optimization.

[0066] S106 performs full-process monitoring and dynamic output coordination processing of stable operating data to generate a continuous and controllable data stream covering the entire remote driving process.

[0067] In one implementation, anomaly identification and tiered monitoring are performed on vehicle status data from stable operation data to generate real-time anomaly warning information. Anomalies are identified and tiered monitoring is performed on core parameters in stable operation data, such as vehicle dynamics (speed, braking status), hardware load (CPU utilization, memory usage), and sensor status (millimeter-wave radar distance, GPS positioning), generating real-time anomaly warning information. The anomaly identification logic is based on preset normal thresholds for each parameter (e.g., vehicle speed ≤ 60 km / h, CPU utilization < 80%, millimeter-wave radar distance > 50 cm, GPS positioning error < 10 m). Abnormal data is identified by comparing parameter values ​​with these thresholds in real time. If a parameter exceeds the threshold, it is further tiered according to the degree of deviation (10%-30% deviation is "minor anomaly," 30%-50% is "moderate anomaly," and > 50% is "serious anomaly"), and the cause of the anomaly is inferred (e.g., high CPU utilization may indicate "too many background programs," and close radar distance may indicate "obstacles ahead").

[0068] At a certain moment, the vehicle status data collected is "vehicle speed 35km / h (normal), CPU utilization 88% (moderately abnormal), millimeter-wave radar distance 40cm (moderately abnormal), GPS positioning error 8m (normal)". The monitoring module first marks the CPU and radar parameters as abnormal, and then generates graded warning information: moderately abnormal CPU utilization (prompt "It is recommended to check the background program and drive with caution"), and moderately abnormal millimeter-wave radar (prompt "The obstacle ahead is 40cm away, pay attention to slowing down"). The warning information is pushed to the driver's cabin control interface in real time in the form of a pop-up window. At the same time, the time of occurrence of the abnormality (202510111635) and the duration are recorded for subsequent traceability.

[0069] The system prioritizes and adapts camera footage from stable operating data to generate a video output stream tailored to driving needs. For front, left / right, and rear view footage from stable operating data, the system dynamically adjusts the priority and display format according to the driving scenario, generating a video output stream adapted to driving requirements. The system adjusts the view priority based on vehicle gear, speed, and driving task (e.g., forward, reverse, obstacle avoidance): In forward gear (speed > 0 km / h), the front view is set as the "main display" (occupying 70% of the cockpit main screen, 720P resolution, 30fps), and the left / right views are set as "auxiliary displays" (each occupying 15%, 480P resolution, 25fps); in reverse gear (speed < 0 km / h), the rear view is upgraded to the "main display" (occupying 60% of the area, 720P resolution), the front view is reduced to 20%, and the left / right views each occupy 10%; if the ultrasonic radar detects that the distance to one side is too close (e.g., left view < 50cm), a flashing red border is added to the edge of the corresponding view to enhance the warning.

[0070] When the vehicle is moving at 20km / h and the ultrasonic radar detects a distance of 45cm from the left, the output stream displays the front view in full screen on the main screen, while the left view is overlaid with a red border and displayed on the right auxiliary screen. The right and rear views are compressed to the status bar at the bottom of the screen, and the frame rate of the left view is maintained at 25fps to ensure real-time performance. This generates an output stream that is adapted to the forward obstacle avoidance scenario, helping the driver focus on the core road conditions and potential risks.

[0071] The system performs execution feedback and closed-loop monitoring on control command data from stable operation data, generating a command execution status stream. It also performs full closed-loop monitoring on the execution process of remote control commands (mode switching, brake / accelerator quantization, emergency stop) from stable operation data, generating a command execution status stream. Monitoring follows the process of "command issuance → vehicle reception → execution feedback → status confirmation." After the driver issues a command, the system tracks the vehicle's reception status in real time (e.g., "command received" or "command format error"). After the vehicle executes the command, the system collects the execution results (e.g., speed change after braking command execution, current vehicle mode after mode switching). If the execution result matches the command target (e.g., "brake amount 500" corresponds to a speed reduction from 30km / h to 20km / h), it marks "execution successful." If inconsistent or timed out (>500ms without feedback), it marks "execution abnormal" and triggers command retransmission (up to 3 times), while simultaneously recording the entire lifecycle data of the command (issuance time, reception time, execution time, feedback time).

[0072] Taking the "Remote Mode Activation" command as an example, the command is issued at 16:30:00. The vehicle responds at 16:30:01 with "Command received." At 16:30:02, the mode switch is executed, and the vehicle reports "Switched to remote mode, current speed 30km / h." At 16:30:03, the driver confirms that the execution result matches the target and marks it as "Execution successful." The final generated command execution status stream includes "Command Type - Issuance Time - Execution Result - Feedback Information," such as "Remote Mode Activation - 20251011163000 - Success - Switched to remote mode, speed 30km / h," ensuring that the entire command execution process is traceable.

[0073] The system performs direct connection status and fault location processing on network connection data in stable operation data, generating a network status monitoring stream. It monitors and locates faults in real-time the network parameters (bandwidth, latency, packet loss rate, connection status) of the vehicle-to-cabin direct connection in stable operation data, generating a network status monitoring stream. The system marks network connection status in real-time as "online," "unstable," or "offline." When bandwidth < 2Mbps or latency > 300ms, it is marked as "unstable"; when the network is interrupted (no data transmission for more than 1000ms), it is marked as "offline." Simultaneously, it locates the cause of faults by detecting SOCKET direct connection routing parameters (such as driver's cabin IP connectivity and port open status) (e.g., "IP cannot be pinged" may indicate "driver's cabin network fault," "port not open" may indicate "routing configuration error"), and associates fault solutions (e.g., "check driver's cabin network cable connection," "reconfigure direct connection route").

[0074] At a certain moment, the network data is "Bandwidth 2.5Mbps, latency 250ms, packet loss rate 3%, connection status: unstable". The monitoring module first marks the connection status as "unstable", then locates the fault as "public network signal fluctuation", and generates a network status monitoring stream: "202510111636-Unstable connection-Bandwidth 2.5Mbps, latency 250ms-It is recommended to observe the signal changes. If it continues to be unstable, the screen resolution can be reduced". At the same time, the network parameter change curve is updated in real time (such as the latency increasing from 200ms to 250ms within 5 minutes) to help the driver predict the network trend.

[0075] Customized task data from stable operation data is processed for task collaboration and progress monitoring, generating a task execution status stream. Customized task data includes task type, task parameters, and associated instructions. Tasks and target vehicles are bound together using a task ID and a unique vehicle number. When task parameters are abnormal, a linked alert is triggered, and route correction suggestions are simultaneously pushed to the driver's cabin. A task execution report is automatically generated upon task completion. Collaborative management and progress monitoring are performed on the task type, parameters, and associated instructions for customized tasks (such as delivery route execution and fixed-point stops) from stable operation data, generating a task execution status stream. Tasks and target vehicles are bound together using a "task ID - unique vehicle number" (e.g., task ID "TS20251011001" is bound to vehicle number "CN65**27"). Task parameters include "task type (delivery), target location (North Station), route (XX Road - XX Road), and associated instructions (automatic announcement upon arrival)." Track task progress in real time (e.g., "30% of the distance has been traveled" or "2km from the target location"). If task parameters are abnormal (e.g., the actual route deviates from the planned route by more than 500m), trigger a linked warning (pushing "Route deviation, it is recommended to correct to XX route"). After the task is completed, automatically generate an execution report (including task duration, mileage traveled, and number of abnormalities).

[0076] The delivery task data for vehicle number CN65**27 is "Task ID: TS20251011001, type: delivery, destination: North Station, current progress: 70%, actual route deviates from the plan by 100m (normal)". The monitoring module generates the task execution status stream: "202510111638-Task TS20251011001 (CN65**27)-Progress 70%-Distance from North Station: 1.5km, route is normal, estimated arrival time: 10 minutes". If the subsequent route deviation reaches 600m, an alert is triggered: "Route deviation 600m, it is recommended to turn left onto XX Road to correct", and the corrected route is pushed to the driver's cabin simultaneously.

[0077] The various types of data after the above monitoring and processing are output coordinated and time-series integrated to generate a continuous and controllable data stream covering the entire remote driving process. The above-mentioned abnormal warning information, screen output stream, command execution status stream, network status monitoring stream, and task execution status stream are linked according to the time sequence and logic of the entire remote driving process, and output coordinated and integrated to generate a continuous and controllable data stream covering the entire process. Using a "global timestamp" as the core association key (e.g., 13-bit millisecond time 1740000000000), data is integrated according to the remote driving process sequence of "network status → screen data → command data → vehicle status → task data": first, the network connection status (online / unstable) is synchronized; then, the screen display configuration under the corresponding status is matched; next, the command execution results and vehicle status feedback are associated; finally, task progress information is supplemented. At the same time, it is ensured that the output format of each data is consistent (e.g., warning information is in JSON format, and the screen stream is H.265 encoded), and the output frequency is adapted to driving needs (vehicle status is output once every 100ms, and the screen stream is output once every 33ms).

[0078] At timestamp 1740000000000ms, the integrated continuous and controllable data stream is as follows: "Timestamp: 17400000000000, Vehicle number: CN65**27, Network status: Unstable (bandwidth 2.5Mbps, latency 250ms), Screen output: Front view main display (720P / 30fps), left view red border, Command execution: Remote mode successfully activated (vehicle speed 30km / h), Vehicle status warning: CPU moderately abnormal, radar moderately abnormal, Task progress: TS20251011001 (70%, route normal)", which fully reflects the multi-dimensional operating status of the remote driving system at this moment, providing coherent data support for driver decision-making and subsequent system optimization.

[0079] S107 performs resource allocation and performance optimization on the continuous and controllable data stream, generating parallel processing results.

[0080] In one implementation, the continuous and controllable data stream (including network status, screen output, command execution, vehicle status, task progress, etc.) covering the entire remote driving process is processed from two aspects: hardware resource allocation and data processing efficiency optimization. Dynamic resource scheduling and parallel computing are used to improve system operating efficiency, ultimately generating parallel processing results. Based on the computational requirements of different types of data in the continuous and controllable data stream (e.g., high GPU resources for screen encoding and high CPU resources for command processing), hardware resources (CPU cores, GPU computing power, memory space, and network bandwidth) are allocated to corresponding data processing tasks according to priority, ensuring that core data occupies resources first.

[0081] Resource allocation is prioritized based on the impact of data on remote driving safety. Emergency safety data (such as emergency stop commands and millimeter-wave radar warnings) has the highest priority, allocated with dedicated CPU cores (e.g., CPU cores 1-2) and a dedicated memory partition (2GB). Driving-related data (such as front-view images and mode switching commands) has the next highest priority, allocated with GPU cores (e.g., GPU cores 0-1) and shared memory (4GB). Auxiliary reference data (such as left / right view images and task progress) has the lowest priority, allocated with the remaining CPU cores (e.g., CPU cores 3-4) and cache memory (1GB). Additionally, 10% of hardware resources are reserved as emergency backup to handle sudden data processing needs (such as screen resolution reduction calculations when network latency spikes).

[0082] At any given moment, the continuous controllable data stream includes "emergency stop command (safety emergency category), front-view image data (driving core category), and left-view image data (auxiliary reference category)". The resource allocation module first allocates CPU core 1 and 2GB of memory to the emergency stop command to ensure zero-latency command processing; then it allocates GPU cores 0-1 and 4GB of memory to the front-view image to ensure smooth image encoding and display; finally, it allocates CPU core 3 and 1GB of cache memory to the left-view image, using low-priority scheduling. After allocation, resource utilization is monitored in real time. If the CPU core 1 utilization exceeds 90%, the processing tasks for auxiliary reference data are temporarily suspended to prioritize the processing of emergency commands and avoid delays caused by resource contention.

[0083] For data in a continuous, controllable data stream that exhibits temporal correlation (e.g., command issuance - execution feedback) or independent parallelism (e.g., image transmission and vehicle status monitoring), a hybrid strategy of "serial processing of correlated data + parallel processing of independent data" is adopted, improving data processing efficiency through multi-threaded scheduling. First, the correlation between data in the data stream is identified. Data with dependencies (e.g., "remote mode activation command → command execution feedback → vehicle status update") is processed sequentially to ensure logical coherence. Independent data without dependencies (e.g., front-view image encoding, network latency monitoring, task progress updates) is allocated to different threads for parallel processing, utilizing the parallel computing power of multi-core CPUs and GPUs to complete calculations synchronously. Simultaneously, thread pool management (e.g., setting the maximum number of threads in the thread pool to 8) avoids resource waste caused by excessive threads, and a dynamic management mechanism of "releasing threads after task completion" is adopted to improve thread reuse.

[0084] In the continuous and controllable data stream, "front-view image encoding" and "network latency monitoring" are independent data, while "braking command → execution feedback → vehicle speed update" are related data. The parallel processing module first starts thread 1 (responsible for front-view H.265 encoding) and thread 2 (responsible for real-time network latency calculation) to perform parallel processing simultaneously; then it starts thread 3 to process the related data serially according to the sequence of "braking command reception → command parsing → vehicle execution → vehicle speed feedback". During processing, the calculation results of thread 1 and thread 2 are synchronized to the main thread in real time, and the execution results of thread 3 are updated to the cockpit interface in sequence. Ultimately, the parallel operation of "image encoding + network monitoring + command processing" is achieved, reducing the data processing time from 500ms for serial processing to 200ms, improving efficiency by 60%.

[0085] By combining real-time operating parameters (such as CPU utilization, GPU computing power utilization, and network bandwidth utilization) in a continuously controllable data stream, the system dynamically adjusts data processing strategies (such as screen resolution and instruction processing frequency) to avoid resource waste or overload and ensure that the system is always in a state of high-efficiency operation. Preset hardware resource utilization thresholds (CPU utilization < 80%, GPU utilization < 70%, bandwidth utilization < 85%) are used to compare the actual utilization with the thresholds in real time: if CPU utilization exceeds 80% (e.g., due to multi-threaded processing), the processing frequency of auxiliary reference data is reduced (e.g., the frame rate of the left-view screen is reduced from 25fps to 20fps); if GPU utilization is less than 50% (e.g., when the amount of screen data is small), the resolution of core screens is increased (e.g., the front-view screen is increased from 720P to 1080P) to fully utilize idle computing power; if network bandwidth utilization exceeds 85%, compression encoding (H.265→H.266) is used for non-core screens (e.g., the rear-view screen) to reduce bandwidth consumption. At the same time, the optimization parameters for different scenarios (such as the screen resolution of urban road scenarios and the instruction processing frequency of high-speed scenarios) are recorded to form an optimization parameter library, which can be directly called in subsequent similar scenarios to reduce redundant calculations.

[0086] When network bandwidth utilization in a continuous, controllable data stream reaches 90% (due to simultaneous transmission of front-view and left / right-view images), the performance optimization module first identifies non-core images as left / right-view images and switches their encoding format from H.265 to H.266, reducing bandwidth usage from 3Mbps to 2Mbps and utilization to 70%. Simultaneously, detecting a GPU utilization of only 40%, the front-view image resolution is increased from 720P to 1080P, improving image clarity without increasing bandwidth burden. After optimization, system image clarity is improved, while network latency is reduced from 250ms to 180ms, achieving dual optimization of resource utilization and driving experience.

[0087] After resource allocation and performance optimization, various data are integrated into parallel processing results according to the principles of "safety first, time sequence alignment." This includes three parts: data processing efficiency indicators, hardware resource usage, and the effectiveness of optimization strategies, providing a basis for subsequent system iterations and troubleshooting. First, the output results of each parallel processing task are summarized (such as emergency command processing time, screen encoding frame rate, and resource utilization rate), and the time sequence is aligned by timestamps (e.g., associating the command processing result at 1740000000000ms with the screen output result). Then, performance optimization indicators are calculated (such as the percentage reduction in data processing time and the percentage increase in resource utilization rate), and the system performance before and after optimization is compared (e.g., command processing time was 100ms before optimization, and 50ms after optimization, a 50% reduction). Finally, a structured parallel processing result is generated, containing fields such as "timestamp-data type-processing time-resource utilization rate-optimization strategy," supporting real-time viewing in the dashboard and traceability in cloud storage.

[0088] The parallel processing result for timestamp 1740000000000ms is as follows: "Timestamp: 17400000000000, Data type: Emergency stop command, Processing time: 50ms, CPU core 1 utilization: 60%, Optimization strategy: None (sufficient resources); Data type: Front view, Processing time: 100ms, GPU cores 0-1 utilization: 65%, Optimization strategy: Resolution increased from 720P to 1080P; Data type: Left view, Processing time: 150ms, CPU core 3 utilization: 40%, Optimization strategy: Encoding format switched from H.265 to H.266." This result is pushed to the cockpit monitoring interface in real time and uploaded to the cloud server simultaneously. If subsequent data processing delays occur, the results can be used to trace whether resource allocation is reasonable and whether optimization strategies are effective, providing data support for system adjustments.

[0089] In one implementation, such as Figure 2 As shown, this application also provides a control device for remote driving of a bus, comprising: The acquisition module 201 is used to acquire the raw data of the entire remote driving process and parse and process the raw data of the entire remote driving process to generate parsed remote driving information; The processing module 202 is used to perform full-process adaptation and optimization processing on the parsed remote driving information, obtain the target data stream supporting the complete remote driving workflow, add process coordination adjustment factors, and generate optimized data; perform process module encoding and full-link identifier embedding processing on the optimized data to generate execution data with process-specific identifiers; perform full-process transmission and execution priority determination processing on the execution data with process-specific identifiers to generate sub-process priority queue data; perform full-process deployment and collaborative scheduling processing on the sub-process priority queue data to generate stable operation data supporting the complete remote driving workflow; perform full-process monitoring and dynamic output coordination processing on the stable operation data to generate a continuous and controllable data stream covering the entire remote driving process; and perform resource allocation and efficiency optimization processing on the continuous and controllable data stream to generate parallel processing results.

[0090] The various embodiments in this application are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for evaluating the control method, electronic device, electronic device, and readable storage medium for remote bus driving are basically similar to the aforementioned bus remote driving control method embodiments, and therefore the description is relatively simple. Relevant parts can be referred to in the descriptions of the aforementioned bus remote driving control method embodiments.

Claims

1. A control method for remote driving of a bus, characterized in that, include: The system acquires raw data from the entire remote driving process and parses and processes this raw data to generate parsed remote driving information. The raw data from the entire remote driving process includes data captured from multiple cameras, control command data, vehicle status data, and network connection data. The parsed remote driving information is adapted and optimized for the entire process to obtain the target data stream that supports the complete remote driving workflow, and a process coordination adjustment factor is added to generate optimized data. The optimized data is processed by encoding process modules and embedding full-link identifiers to generate execution data with process-specific identifiers; Perform full-process transmission and execution priority determination processing on the data to be executed with process-specific identifiers, and generate process-specific priority queue data; The priority queue data of each process is deployed and coordinated throughout the entire process to generate stable operating data that supports the complete workflow of remote driving. The system performs full-process monitoring and dynamic output coordination of stable operating data to generate a continuous and controllable data stream covering the entire remote driving process. Resource allocation and performance optimization are performed on continuous and controllable data streams to generate parallel processing results.

2. The method as described in claim 1, characterized in that, The parsed remote driving information undergoes full-process adaptation and optimization to obtain the target data stream supporting the complete remote driving workflow. A workflow coordination adjustment factor is then added to generate optimized data, including: The parsed multi-camera image data is optimized for viewpoint priority and transmission adaptation to generate an image data stream adapted to remote driving scenarios. The image data of the front view, left / right view, and rear view are sorted in order of priority: front view > left / right view > rear view. Combined with real-time network bandwidth and latency monitoring data, the resolution and frame rate of the image in areas with insufficient bandwidth or high latency are dynamically adjusted, while adapting to the WebRTC direct connection transmission protocol. The execution logic and transmission path of the image data stream adapted to remote driving scenarios are optimized to generate a closed-loop command data stream. Specifically, the closed-loop logic of remote / automatic / manual driving mode switching commands, brake / accelerator quantization commands, and emergency stop commands is optimized according to the command generation - cockpit issuance - vehicle reception and verification - execution feedback. The commands are bound to the target vehicle based on the vehicle's unique vehicle number identifier. The command transmission path is optimized to rely on the vehicle-cockpit direct transmission through the SOCKET direct connection channel. The parsed vehicle status data is optimized by prioritizing monitoring and filtering anomalies to generate a monitoring data stream focused on core statuses. The parsed network connection data is optimized for connection establishment efficiency and direct connection adaptation to generate the target network data stream. Specifically, the connection establishment logic is optimized based on the following process: the cockpit initiates a control request, sends the vehicle number, the cloud server verifies the vehicle number's validity, the cloud server obtains the cockpit IP and pushes the cockpit IP address to the vehicle, and the vehicle and cockpit connect directly via SOCKET. SOCKET direct connection routing parameters are configured to directly establish a public network data transmission channel between the vehicle and the cockpit. The network connection status is marked in real time. The optimized multi-type data are then integrated and processed with adjustment factors to generate optimized data. This process integrates the video, commands, vehicle status, and network connection data streams according to the entire remote driving process timeline. Process coordination adjustment factors are added, including network latency grading thresholds, dynamic adjustment parameters for camera viewpoints, and hardware anomaly response levels.

3. The method as described in claim 1, characterized in that, The optimized data undergoes process module coding and end-to-end identifier embedding to generate execution data with process-specific identifiers, including: The optimized multi-camera image data stream is input into the viewpoint priority encoding module, and the image data is format-standardized through preset encoding rules to generate image encoding data adapted for WebRTC direct transmission. The optimized closed-loop command data stream is input into the command binding encoding module, and the command data is associated and encoded based on the vehicle's unique vehicle number to generate command encoding data with a unique binding identifier; The optimized core status monitoring data stream is input into the anomaly level coding module, which classifies and codes the monitoring data according to hardware load and security risk level, generating monitoring coded data with warning indicators. The optimized target network data stream is input into the direct connection status coding module, and the network data is adapted and encoded for transmission by combining the SOCKET direct connection routing configuration parameters to generate network coded data with connection status identifiers. The above four types of coded data are input into the end-to-end identifier integration module. The unique identifiers of each type of data are associated and integrated through the identifier mapping rules to generate data to be executed with process-specific identifiers.

4. The method as described in claim 1, characterized in that, Perform end-to-end transmission and execution priority determination processing on data to be executed with process-specific identifiers, generating process-specific priority queue data, including: Input the data to be executed with the process-specific identifier into the transmission priority determination module. The data is classified and sorted according to the preset transmission priority rules to generate preliminary sorted data to be transmitted. The initially sorted data to be transmitted is input into the network status adaptation module, and the transmission priority is dynamically adjusted in combination with real-time network bandwidth and latency data to generate the data to be transmitted after network adaptation. The network-adapted data to be transmitted is input into the execution priority determination module, which sets the execution priority based on the urgency of the data function and the level of security risk, and generates data to be scheduled containing execution priority identifiers. The scheduled data containing the execution priority identifier is input into the sub-process queue generation module, and the queue is divided according to the two dimensions of transmission process and execution process to generate sub-process priority queue data.

5. The method as described in claim 1, characterized in that, The priority queue data for each process is deployed and coordinated throughout the entire process to generate stable operational data that supports the complete remote driving workflow, including: The network connection data blocks in the priority queue of the sub-process are directly deployed to generate stable connection data for the vehicle-warehouse direct connection channel. The process deployment is as follows: the cockpit initiates a control request, sends the vehicle number, the cloud server verifies the legality of the vehicle number, the cloud server obtains the cockpit IP and pushes the cockpit IP address to the vehicle, the vehicle configures a direct connection route based on the SOCKET protocol, and the vehicle and warehouse establish a public network direct connection. The deployment unit is the vehicle number and IP binding pair, and the number of connection timeout retries is calculated based on the SOCKET direct connection parameters. The camera image data blocks in the priority queue of the sub-process are processed for view scheduling to generate image output data adapted to the driving scenario; The control instruction data blocks in the sub-process priority queue are executed and scheduled to generate stable control data for closed-loop instruction execution. The vehicle status data blocks in the sub-process priority queue are monitored and scheduled to generate real-time and controllable status monitoring data. The various types of data blocks deployed and scheduled above are collaboratively integrated to generate stable operating data that supports the entire remote driving process.

6. The method as described in claim 5, characterized in that, The system performs full-process monitoring and dynamic output coordination of stable operating data to generate a continuous and controllable data stream covering the entire remote driving process, including: Anomaly identification and hierarchical monitoring of vehicle status data in stable operation data are performed to generate real-time anomaly warning information; The camera image data in the stable operation data is processed for viewpoint priority and display adaptation, and an image output stream adapted to driving needs is generated; Perform execution feedback and closed-loop monitoring on the control command data in the stable operation data to generate a command execution status stream; Perform direct connection status and fault location processing on network connection data in stable operation data to generate a network status monitoring stream; Customized task data in stable operation data is processed for task collaboration and progress monitoring to generate a task execution status stream. The customized task data includes task type, task parameters, and task-related instructions. The task and target vehicle are bound by task ID and vehicle unique number. When the task parameters are abnormal, a linkage warning is triggered and route correction suggestions are pushed to the driver's cabin. A task execution report is automatically generated after the task is completed. The various types of data after the above monitoring and processing are output collaboratively and integrated in a time sequence to generate a continuous and controllable data stream covering the entire remote driving process.

7. A control device for remote driving of a bus, characterized in that, The device includes: The acquisition module is used to acquire the raw data of the entire remote driving process and parse and process the raw data of the entire remote driving process to generate parsed remote driving information; The processing module performs end-to-end adaptation and optimization on the parsed remote driving information, obtains the target data stream supporting the complete remote driving workflow, adds process coordination adjustment factors, and generates optimized data. It then encodes the optimized data into process modules and embeds end-to-end identifiers, generating execution data with process-specific identifiers. The execution data with process-specific identifiers undergoes end-to-end transmission and execution priority determination, generating sub-process priority queue data. This sub-process priority queue data undergoes end-to-end deployment and collaborative scheduling, generating stable operating data supporting the complete remote driving workflow. The stable operating data undergoes end-to-end monitoring and dynamic output coordination, generating a continuous and controllable data stream covering the entire remote driving process. Finally, the continuous and controllable data stream undergoes resource allocation and performance optimization, generating parallel processing results.

8. An electronic device, characterized in that, include: First processor; And a memory for storing executable instructions of the first processor; wherein the first processor is configured to execute the control method for remote bus driving according to any one of claims 1 to 6 by executing the executable instructions.

Images