A remote assistance work order processing method, apparatus and device

Abstract

The embodiment of the specification provides a remote assistance work order processing method, device and equipment, and relates to the technical field of automatic driving. The method comprises the following steps: collecting remote assistance process data to buffer storage based on a collection link outside a control link of a vehicle, and collecting driving scene data to persistent storage during a remote assistance work order; the remote assistance process data comprises running control data of the vehicle and log data of a remote operation seat; the driving scene data comprises driving scene analysis data and driving real scene video data of the vehicle; in response to the remote assistance work order being closed, data with a work order identifier, a vehicle identifier and a remote operation seat identifier of the remote assistance work order in the remote assistance process data is extracted from the buffer storage, and is aggregated with driving scene data associated with the work order identifier in the persistent storage to generate a playback data package of the remote assistance work order in the persistent storage.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of autonomous driving technology, specifically to a remote assistance work order processing method, apparatus, and equipment. Background Technology

[0002] Remote assistance is a driver assistance mode that combines autonomous driving capabilities with remote human intervention. When an autonomous vehicle encounters a situation it cannot handle autonomously, cloud-based dispatch generates a remote assistance work order, assigning the task to a remote operator. The operator then assists the vehicle in resuming operation through video observation or necessary remote control. A single work order may be reassigned during its lifecycle, meaning it may be served by multiple operators in succession. After a work order is closed, operations, R&D, and safety audit departments need to retrieve the complete vehicle status, dispatch events, operator actions, bypass video, and operator logs for review, accountability, and training feedback.

[0003] However, in existing remote-assisted data acquisition, acquisition anomalies may interfere with vehicle control, data with different characteristics are stored in a unified manner, and large-capacity videos can easily crowd out the buffer, affecting the retention of high-frequency messages, making it difficult to review and trace the event afterward. Summary of the Invention

[0004] The purpose of the embodiments in this specification is to provide a remote assistance work order processing method, apparatus, and equipment to overcome the difficulties in post-event review and traceability in existing methods.

[0005] To solve the above-mentioned technical problems, the specific technical solutions of the embodiments in this specification are as follows: On the one hand, the embodiments of this specification provide a remote assistance work order processing method, including: Outside the vehicle's control link, based on the acquisition link, remote assistance process data is collected and stored in a buffer, and driving scene data is collected and stored in persistent storage during the remote assistance work order; the remote assistance process data includes vehicle operation and control data and remote operator log data; the driving scene data includes vehicle driving scenario analysis data and driving real-time video data; In response to the closure of a remote assistance work order, data with the route identifier of the remote assistance work order, the work order identifier, the vehicle identifier, and the remote operator identifier are extracted from the buffer storage and aggregated with the driving scene data associated with the work order identifier in persistent storage to generate a playback data packet for the remote assistance work order in persistent storage.

[0006] Furthermore, embodiments of this specification provide a remote assistance work order processing device, comprising: The data acquisition module is used to collect remote assistance process data to buffer storage based on the data acquisition link outside the vehicle's control link, and to collect driving scene data to persistent storage during the remote assistance work order; the remote assistance process data includes vehicle operation and control data and remote operator log data; the driving scene data includes vehicle driving scenario analysis data and driving real-time video data. The generation module is used to extract the work order identifier, vehicle identifier, and remote operator identifier data with the routing identifier of remote assistance work order from the buffer storage in response to the closure of the remote assistance work order, and aggregate them with the driving scene data associated with the work order identifier in the persistent storage to generate the playback data packet of the remote assistance work order in the persistent storage.

[0007] Furthermore, embodiments of this specification provide a computer device, including: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes these computer instructions to implement the aforementioned remote assistance work order processing method.

[0008] Furthermore, embodiments of this specification provide a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the aforementioned remote assistance work order processing method.

[0009] In another aspect, embodiments of this specification provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the aforementioned remote assistance work order processing method.

[0010] As can be seen from the technical solutions provided in the embodiments of this specification above, the embodiments of this specification collect data outside the vehicle control link, decoupling the data collection process from vehicle control and not affecting normal vehicle control; writing process data to buffer storage and driving scene data to persistent storage, realizing hierarchical storage of data with different characteristics, avoiding large-capacity driving scene data crowding out the buffer; after the work order is closed, process data is extracted according to the work order identifier, vehicle identifier, and operator's seat identifier, and aggregated with the driving scene data to generate a playback data package, realizing complete archiving by work order unit, which is convenient for post-event review and traceability. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below.

[0012] Figure 1 This is a schematic diagram illustrating an application scenario of a remote assistance work order processing method provided in the embodiments of this specification; Figure 2This is a flowchart of a remote assistance work order processing method provided in the embodiments of this specification; Figure 3 This is a schematic diagram of the overall system architecture in a specific scenario example provided in the embodiments of this specification; Figure 4 This is a schematic diagram of the overall system flow in a specific scenario example provided in the embodiments of this specification; Figure 5 This is a schematic diagram of the seat allocation filtering process in a specific scenario example provided in the embodiments of this specification; Figure 6 This is a schematic diagram of phased archiving in a specific scenario example provided in the embodiments of this specification; Figure 7 This is a schematic diagram of the structural composition of a remote assistance work order processing device provided in the embodiments of this specification; Figure 8 This is a schematic diagram of the structural composition of a computer device provided in the embodiments of this specification. Detailed Implementation

[0013] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments in this specification. Other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort should fall within the scope of protection of this specification.

[0014] It should be noted that the terms "first," "second," etc., used in this specification, claims, and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0015] The remote-assisted driving system, either cloud-based or independently deployed on the data acquisition side, is used to bypass the vehicle control link to collect multi-source data during remote assistance, filter it by work order, aggregate and archive it, generating work order-level archived playback data packages for review, accountability, and training replay. The system includes a data acquisition link, buffer storage, persistent storage, a lifecycle event subscription module, a data filtering module, and an archiving module.

[0016] The data acquisition link is built outside the vehicle control link and is isolated from it. The acquisition link only bypasses and copies the vehicle uplink data, cloud downlink data, and remote operator control data transmitted in the control link, appending the original subject, reception time, and routing identifier before writing them to the buffer storage; simultaneously, it retrieves log data from the remote operator's resident log process and writes it to the buffer storage. Vehicle uplink data, cloud downlink data, remote operator control data, and remote operator log data together constitute the remote assistance process data. The buffer storage is a short-cycle, read-back storage system used to temporarily store remote assistance process data continuously generated across work orders before the work order is closed.

[0017] During remote assistance work orders, the data acquisition chain also collects driving scene data and writes it directly to persistent storage. Driving scene data includes driving scenario analysis data and driving real-time video data. Driving scenario analysis data collection begins after the remote assistance work order generation event, and driving real-time video data recording begins after the playback event. Both are written directly to persistent storage without buffering, and are associated with the remote assistance work order identifier. This way, the large volume of driving scene data does not occupy the limited buffer storage space, while ensuring its long-term availability.

[0018] When a remote assistance work order is closed, the lifecycle event subscription module has been continuously receiving work order lifecycle events. The system determines the start and end times of the work order based on the generation and closure events, and extends forward by a first preset duration and backward by a second preset duration based on these start and end times to form an extraction time window. After the end time of the extraction time window arrives, the data filtering module extracts data from the remote assistance process data whose reception time falls within the extraction time window, carrying at least one of the following identifiers: work order identifier, vehicle identifier, or remote operator identifier.

[0019] The data filtering module processes data according to identifier type: remote assistance process data carrying work order identifiers or corresponding vehicle identifiers is directly extracted; remote assistance process data carrying remote operator identifiers is extracted based on the service period of the remote operator for the remote assistance work order, and only data whose reception time falls within that service period is extracted; for remote assistance process data with empty identifier fields, the work order identifier, vehicle identifier, or remote operator identifier in its payload is parsed, and if it corresponds to the remote assistance work order, it is extracted. The remote operator service period is maintained by events assigned to and leaving the remote operator. Assignment events close unclosed old service periods and open new service periods at the corresponding time, while departure or closure events close service periods at the corresponding time.

[0020] The archiving module aggregates the remote assistance process data extracted by the data filtering module with the driving scene data associated with the remote assistance work order identifier in persistent storage, generating an archived playback data package for the remote assistance work order and writing it to persistent storage. The archived playback data package retains the business time and reception time of each data point; the business time is used for playback alignment, and the reception time is used to determine the remote operator's service period. If the driving scene data has not yet been collected, the archiving module generates an archived playback data package based on the currently collected driving scene data and extracted remote assistance process data, recording items to be supplemented. The archived playback data package is regenerated after the data is supplemented. Furthermore, the archiving module can transcode the driving scene video data. After transcoding, a visual playback package is generated based on the archived playback data package, allowing reviewers to view the entire event process in chronological order.

[0021] Through the aforementioned remote assisted driving system, the vehicle operation control, operator logs, driving scenario analysis, and real-scene video data involved in remote assistance work orders can be completely and accurately collected outside the control link. The data is then filtered and aggregated according to work order boundaries and seat service periods to form long-term usable work order-level archive playback data packages. This ensures both the high availability of the control link and the realization of work order-level closed-loop evidence collection from multiple data sources.

[0022] Reference Figure 1 As shown in the example scenario, an autonomous driving test vehicle, V-317, equipped with a Level 4 autonomous driving system, is driving at the entrance of an urban expressway ramp. Due to temporary construction work, cones and guide signs have been placed in front of the vehicle. The autonomous driving system determines that it cannot safely pass through autonomously and sends a remote assistance request to the cloud-based dispatch system at 14:23:05. The cloud-based dispatch system then generates a remote assistance work order W-20250907-001, with the vehicle identifier V-317 and the work order status "pending assignment." At this time, the remote assisted driving system deployed in the cloud is already running, and the data acquisition link bypasses the vehicle control link to collect process data from V-317 and surrounding remote control stations.

[0023] The acquisition link bypasses the message broker to copy the V-317's uplink vehicle data, cloud downlink data, and control data from each remote operator's station. Each copied data entry is appended with the original subject, reception time, and routing identifier before being written to a buffer. This buffer is a short-cycle, read-back storage system with a retention period of 20 minutes. Simultaneously, the resident log processes of remote operators A, B, and C continuously upload their respective operation logs, and the acquisition link also writes this log data to the buffer. These data collectively constitute the remote assistance process data.

[0024] After the work order generation event arrives at 14:23:05, the data acquisition link initiates the acquisition of driving scenario analysis data. It directly writes the analysis data generated by the autonomous driving system, including the distribution of obstacles around the vehicle, planned paths, and decision states, into persistent storage, and associates it with the work order identifier W-20250907-001. The driving scenario analysis data is not buffered, therefore it does not occupy the short-term retention space of the buffer storage.

[0025] At 14:23:12, the cloud-based dispatch system assigns work order W-20250907-001 to remote operator A. The lifecycle event subscription module receives the assignment event, and the data filtering module maintains the service period for operator A based on this: starting at 14:23:12, and not yet closed. At this time, remote operator A has not yet started viewing the vehicle's live video feed; it only understands the vehicle's status through messages and status data. Therefore, live driving video data recording has not yet begun.

[0026] At 14:24:30, remote operator A issued a start playback command. The acquisition link responded to the playback event, initiating the recording of real-time driving video data. The real-time footage from the V-317's forward-facing camera was recorded in bypass mode and directly written to persistent storage, also linked with a work order identifier. The video recording used the original encoding format without real-time transcoding to ensure low latency in the recording link.

[0027] At 14:25:40, remote operator A attempted to guide the vehicle to a detour, but failed due to network latency and inaccurate vehicle position judgment. At 14:26:00, the cloud dispatch system decided to reassign the work order W-20250907-001 to remote operator B. The lifecycle event subscription module received the assignment event. The data filtering module first closed the service period of operator A with 14:26:00 as the end time, obtaining the service period of operator A as [14:23:12, 14:26:00]; then, with 14:26:00 as the start time, it opened a new unclosed service period for operator B.

[0028] At 14:26:10, remote operator B issued a start playback command. The acquisition link responded to the new playback event, started recording the second segment of real-time driving video data, and continued writing it to persistent storage. At this time, there were already two video data segments in persistent storage, corresponding to the service time periods of operator A and operator B, respectively, both associated with work order W-20250907-001.

[0029] At 14:28:45, remote operator B successfully guided vehicle V-317 around the construction area, and the vehicle resumed autonomous driving. The cloud dispatch system issued a work order closure event at 14:28:45. After receiving the closure event, the lifecycle event subscription module used 14:28:45 as the end time to close the service period of operator B, resulting in the service period of operator B as [14:26:00, 14:28:45]. At this point, the service periods for all operators corresponding to this work order are determined.

[0030] After the work order is closed, the remote assisted driving system enters the archiving stage. Based on the generation and closure events, the system determines the work order start and end times as 14:23:05 to 14:28:45. Following preset rules, it extends forward by a first preset duration of 2 minutes and backward by a second preset duration of 30 seconds, forming an extraction time window of [14:21:05, 14:29:15]. After the extraction time window ends at 14:29:15, the system begins retrieving remote assistance process data from the buffer storage.

[0031] The data filtering module processes remote assistance process data received within the extraction time window according to the identifier type. Data carrying work order identifier W-20250907-001 is directly extracted, including lifecycle event records and cloud-issued instructions; data carrying vehicle identifier V-317 is directly extracted, including vehicle speed, steering angle, obstacle distance, and other status data reported by the vehicle between 14:21:05 and 14:23:05, 1 minute and 58 seconds before the work order was generated. This data is crucial for analyzing the causes of vehicle entrapment; data carrying seat A identifier is extracted only when the receiving time falls within seat A's service period [14:23:12, 14:26:00]. Control data generated by seat A after 14:26:00 belongs to other work orders and is excluded; data carrying seat B identifier is extracted only when the receiving time falls within seat B's service period [14:26:00, 14:28:45]. For the small amount of data without a routing identifier, the data filtering module parses the work order identifier, vehicle identifier, or remote operator identifier in the load, and extracts those that match work order W-20250907-001.

[0032] The archiving module aggregates the extracted remote assistance process data with the driving scenario analysis data associated with work order identifier W-20250907-001 and two segments of real-time driving video data from persistent storage to generate an archived playback data package for the remote assistance work order, which is then written to persistent storage. In the archived playback data package, each data entry retains both the business time and the reception time. The business time is used to align the playback data according to the actual order of events; the reception time is used to determine the service period of the work order. For example, if the business time of control data from work order A is 14:25:39 and the reception time is 14:25:40, its reception time falls within work order A's service period and is therefore included; while the reception time of control data from work order A is 14:26:03, which is outside the service period and is excluded.

[0033] Because the driving scenario analysis data and driving real-time video data were directly written to persistent storage during the work order period, the archiving process did not require waiting for video transcoding or analysis data upload to complete. The archived playback data package was generated at 14:30:00, with the entire process taking 1 minute and 15 seconds, far shorter than the 20-minute retention period of the buffer storage. Subsequently, even if the buffer storage evicts data from 14:21:05 to 14:29:15 according to its policy, the work order's messages and logs will remain available in the archived playback data package for an extended period.

[0034] Furthermore, the system transcodes the driving video data in the archived playback data package to generate a unified preview track suitable for web playback. After transcoding, a visual playback package is synthesized based on the archived playback data package. When the reviewer opens the visual playback package, they can see: vehicle status and obstacle data from 14:21:05 to 14:23:05 are presented in charts and curves, explaining why the vehicle was stuck; from 14:23:12 to 14:26:00, remote operator A's logs, control data, and corresponding video are displayed synchronously, reviewing their operational intent and reasons for failure; from 14:26:00 to 14:28:45, remote operator B's video and control commands are aligned according to business time, clearly presenting the successful detour process. The switching point between the two video segments precisely corresponds to the service time of each operator, with no cross-work order control data mixed in.

[0035] In the above scenario, the data acquisition link operates entirely outside the vehicle control link, having no impact on normal vehicle control. Although the buffer storage contains data from V-317, other vehicles, and multiple operator seats, a two-layer filtering mechanism—first coarsely filtering by time window, then finely filtering by identifier and service period—ensures that the archived playback data package only contains the actual process data of work order W-20250907-001. Vehicle headroom allows for the retention of data on the causes of entrapment before the work order is generated, while strict limitations on operator service periods prevent data crosstalk when the same operator serves adjacent work orders. Driving scenario analysis data and real-time video data are directly persisted, without occupying buffer storage, while ensuring timely archiving. The entire process, with work order lifecycle events as boundaries, operator service periods as filtering criteria, and persistent storage as the final sedimentation, achieves a precise work order-level closed loop between remote assistance process data and driving scene data.

[0036] Figure 2 A flowchart of a remote assistance work order processing method according to an embodiment of this specification is shown. This method can be applied to a remote assisted driving system. While this specification provides method operation steps or apparatus structures as shown in the following embodiments or figures, more or fewer operation steps or module units may be included in the method or apparatus based on conventional or non-inventive effort. In steps or structures where there is no logically necessary causal relationship, the execution order of these steps or the module structure of the apparatus is not limited to the execution order or module structure described in the embodiments and figures of this specification. When the method or module structure is applied in a practical device or terminal product, it can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed processing environment) according to the method or module structure shown in the embodiments or figures.

[0037] Specifically, such as Figure 2 As shown in the embodiments of this specification, the remote assistance work order processing method applicable to remote assisted driving systems may include the following steps.

[0038] S201: Outside the vehicle's control link, based on the acquisition link, remote assistance process data is collected to buffer storage, and driving scene data is collected to persistent storage during the remote assistance work order; the remote assistance process data includes vehicle operation and control data and remote operator log data; the driving scene data includes vehicle driving scenario analysis data and driving real-time video data.

[0039] In autonomous driving remote assistance systems, the vehicle control link plays a crucial role in transmitting remote operation commands to the vehicle's execution unit in real time. The response latency and transmission reliability of this link directly impact driving safety and rescue success rates. In existing technologies, data acquisition modules are often embedded within the control link in a plug-in manner. By setting interception points or hook functions along the control command transmission and confirmation paths to copy control messages, when the acquisition module experiences processing blockage due to storage exhaustion, write delays, or decoding anomalies, this blockage signal propagates backward along the shared buffer or synchronous call chain to the control command transmission channel. This prevents remote operation commands from reaching the vehicle on time, leading to rescue interruptions or vehicle loss of control. Furthermore, the data generated during remote assistance includes various types of data. Different types of data have different acquisition timing, storage media, and retention period requirements. Writing all data indiscriminately to the same storage medium can cause the buffer storage to be quickly filled by large binary files, thus blocking the writing of high-frequency status messages. Simultaneously, driving scenario analysis data still has long-term retention value after the work order ends, and the limited retention period of the buffer storage cannot meet this requirement. To address the security risks arising from the coupling of the control link and data acquisition actions, as well as the resource contention issues caused by differing data storage requirements for different types, a hierarchical storage method is provided, with an independent acquisition link outside the control link. This method deploys a separate acquisition link outside the vehicle's control link. This acquisition link is used to handle the acquisition and transmission of all remote assistance process data. There are no shared execution threads, shared memory queues, or synchronous call interfaces between this acquisition link and the control link. Any blocking, timeout, or exception occurring on the acquisition link does not send a signal back to the control link. Within this acquisition link, the system categorizes the data to be acquired into remote assistance process data and driving scene data based on data type. The remote assistance process data represents the operational and control-related data generated by the vehicle during remote assistance, as well as the log-related data generated by the remote operator during the service process. The driving scene data represents the scene analysis data and driving real-time video data generated by the vehicle at the driving scene. The acquisition link writes the remote assistance process data into a buffer storage. This buffer storage represents a short-cycle storage component that is sorted by the time of reception and provides time-range readback capability. This buffer storage is shared by multiple vehicles, multiple operators, and multiple work orders, and its retention period is a configurable finite duration. The data acquisition link writes driving scene data to persistent storage, which is a long-term storage medium independent of buffer storage. This persistent storage has a longer retention period than the buffer storage and is unaffected by buffer storage eviction policies. During the duration of a remote assistance work order, the data acquisition link continuously collects driving scene data and writes it to persistent storage. This driving scene data remains in persistent storage even after the work order is closed, for later review and training.In one implementation, message data and log data in the remote assistance process data are written to different topic partitions of the same buffer storage, and these different topic partitions share the same retention period configuration. In another implementation, message data in the remote assistance process data is written to buffer storage, while log data is directly written to a dedicated log storage area in persistent storage. The retention period of this dedicated log storage area is longer than that of the buffer storage but shorter than that of the driving scene data. This method is used for tiered storage optimization when the long-term retention requirement for log data is higher than that for message data but lower than that for driving scene video data. By deploying the acquisition link independently outside the control link and writing the remote assistance process data and driving scene data to buffer storage and persistent storage respectively, complete isolation between acquisition actions and control command paths is achieved. Failures in any acquisition link will not be transmitted to the vehicle control channel. At the same time, media tiering avoids competition for the same storage resources by different types of data, ensuring that the writing of high-frequency status messages is not blocked by large blocks of video data.

[0040] In some embodiments, the acquisition link and the control link are independent of each other.

[0041] In existing systems, some solutions claim to employ bypass acquisition, but in practice, they still execute acquisition tasks by sharing the operating system kernel's network protocol stack, disk I / O channels, or thread pool resources. When the acquisition task experiences a sudden high load, these shared resources create implicit blocking propagation paths, leading to increased network packet processing delays or I / O response delays in the control link, thus failing to achieve true fault isolation. To address the implicit coupling problem between the acquisition and control links caused by resource sharing, a method is proposed that allows the acquisition and control links to be independent. This independence indicates that the acquisition and control links do not share competing resources at the hardware or operating system resource levels. At the hardware resource level, the acquisition link is deployed on a separate physical network interface card, which is different from the physical network interface card used by the control link. Each link has independent network bandwidth resources and independent transmission queues, ensuring that network traffic surges on the acquisition link do not crowd out the network bandwidth of the control link. At the operating system resource level, the acquisition link runs in an independent message broker process. There is no inter-process shared memory communication between this message broker process and the main control process of the control link. The message broker process of the acquisition link obtains message copies from the exchange nodes of the control link through a mirrored queue mechanism. The consumption progress of this mirrored queue does not affect the message delivery progress on the main exchange node. Through the above-mentioned hardware-level or operating system-level resource isolation methods, the independence between the acquisition link and the control link is concretized into an implementable system deployment constraint, ensuring that even under extremely high load or abnormal congestion conditions, the command issuance capability of the control link remains unaffected.

[0042] In some embodiments, operational control data includes vehicle uplink data, cloud downlink data, and remote operator control data.

[0043] Vehicle uplink data represents messages reported from the autonomous vehicle to the cloud dispatch system via the network. This uplink data includes at least one or more of the following: vehicle location information, vehicle speed information, vehicle fault code information, obstacle perception information, decision-making and planning status information, and remote assistance request signals. These messages are sent to the cloud by the vehicle's communication gateway according to a preset reporting cycle or event triggering conditions. Cloud downlink data represents instruction messages issued from the cloud dispatch system to the autonomous vehicle. This downlink data includes at least one or more of the following: remote control instructions, path planning update instructions, speed limit instructions, and mode switching instructions. These instructions are generated by the cloud dispatch system based on inputs from the remote operator's console or outputs from the automatic decision-making module and sent to the vehicle's execution end via the downlink channel. Remote operator control data represents the sequence of control signals generated by the remote operator through operating input devices during service work orders. This remote operator control data includes at least one or more of the following: steering wheel angle signal, throttle opening signal, brake pressure signal, gear shift signal, and auxiliary function start / stop signal. These signals are collected by the remote operator's console and forwarded to the vehicle by the cloud dispatch system. By fully covering the three directions of vehicle uplink, cloud downlink, and remote operator control with operational control data, a closed-loop data system is achieved, from vehicle status perception to cloud decision-making and scheduling to remote manual control. During review, the system can fully reconstruct the vehicle response corresponding to each control input and the operator decision-making process corresponding to each vehicle anomaly.

[0044] In some embodiments, vehicle uplink data, cloud downlink data, and remote operator control data transmitted in the control link can be copied.

[0045] In existing technologies, a common method for obtaining message copies is to set up interception proxies at the sending and receiving endpoints of the control link. These interception proxies copy messages by modifying the message processing flow of the control link. This method is essentially still instrumentation recording, and a failure of the interception proxies can interrupt message delivery in the main processing flow. Another method is to configure port mirroring at the network layer, copying all network traffic from the control link to the acquisition link. While this method does not intrude on the message processing logic of the control link, port mirroring copies the original network packets. The acquisition link needs to parse the packet protocol and restore the message boundaries itself, resulting in high complexity and the risk of data loss due to parsing errors. To address the problems of instrumentation recording blocking the main link and the high complexity of port mirroring parsing, a method for obtaining message copies through message broker-side replication subscription is provided. This method configures a mirror queue or replication subscription relationship on the message broker side of the control link. This message broker receives messages reported by the vehicle and routes messages sent from the cloud to the corresponding downlink channels. In this message broker, a separate acquisition subscriber is created for the acquisition link. The subscription relationship between this acquisition subscriber and the message broker is independent of the original publish-subscribe relationship of the control link. Message copies received by the acquisition subscriber from the message broker are generated internally by a replication mechanism that does not change the storage state or forwarding progress of the original message within the message broker. In some embodiments, each replicated data message can be appended with its original topic, reception time, and routing identifier, and written to a buffer. The routing identifier can be a work order identifier, vehicle identifier, remote operator identifier, or empty.

[0046] During message transmission, messages often only carry business-level payload content. Their original topics may change with system version evolution. The message broker automatically generates the received data but doesn't store it in association with the message payload. Routing identifiers may exist in different field locations within the payload, and their locations differ for different message types. When subsequent filtering and retrieval from buffer storage by time range, entity identifier, or topic type is required, the system needs to parse and extract fields from the payload of each message to obtain filtering conditions due to the missing or inconsistent metadata fields. Message payload parsing depends on the specific protocol definition of the message type; protocol changes necessitate synchronous modifications to the filtering logic, increasing system maintenance complexity and runtime parsing overhead. To address the issue of message replicas lacking unified metadata fields, requiring payload parsing for subsequent filtering and retrieval, a method is provided to append envelope information to each replicated data and write it to buffer storage. This method, after acquiring a message copy, does not parse the business fields in the payload of each copied data. Instead, it obtains the original topic field, proxy reception time field, and routing identifier field of the message from the message broker's exchange node. It combines the original topic, proxy reception time, and routing identifier into an envelope structure, appending this envelope structure before the original message payload or storing it as independent metadata associated with the original message payload before writing it to the buffer storage. The original topic represents the topic name used when the message is published in the message broker. This original topic is specified by the message publisher at the time of publication. The acquisition link directly reads the original topic from the message attributes of the message broker without parsing the payload content. The proxy reception time represents the system timestamp when the message is received by the network access layer of the message broker. This proxy reception time is generated by the message broker at the moment of receiving the message and recorded in the message's metadata attributes. The acquisition link reads the proxy reception time from this metadata attribute. The routing identifier is used to characterize the main entity category and main entity identifier value associated with the message. The priority of the routing identifier value is as follows: take the value of the routing key field explicitly specified by the publisher of the message payload as the routing identifier; when there is no explicitly specified routing key field in the message payload, complete the routing identifier according to the matching result of the original topic of the message and the preset topic-entity mapping rule. The preset topic-entity mapping rule is used to map different original topics to the corresponding entity categories and entity identifier extraction methods; when the routing identifier cannot be obtained by the above two methods, the routing identifier of the message is set to empty.

[0047] In some embodiments, log data from the remote operator's console can be written to a buffer storage.

[0048] On the remote operator's side, the operator's log data has different generation methods and transmission characteristics than the aforementioned operational control data. Operational control data is forwarded via a message broker using messages as the basic unit; each message has a clear message boundary and routing identifier. Log data, on the other hand, is a continuous text stream output by the operator's operating system and applications. This text stream is written to a local log file line by line or in fixed block sizes. The log collection process needs to read from and upload this local log file. In existing technologies, some solutions treat log collection as an auxiliary action independent of message collection, only starting the log collection process after a work order is opened. This results in the inability to capture the operator's system status and application initialization logs before the work order is generated. Furthermore, if log data and message data are written to different storage systems, subsequent work order data packets require reading logs and messages from two different storage systems, increasing the complexity of data association. To address the problem of lost context before the work order due to delayed log collection startup and the increased association complexity caused by separate storage of logs and messages, a method is provided to write the remote operator's log data to a buffer storage. This method deploys a resident log collection process after the remote operator's operating system starts. This resident log collection process runs independently of any work order opening or closing event, continuously monitoring change events in the operator's local log file. When a new log line is added to the log file, the resident log collection process reads the new log line line by line and encapsulates it into a log message. An envelope with the same format as the envelope structure described in the previous step is appended to this log message. The original subject field in this envelope is set to a predefined log topic, the proxy reception time field is set to the time when the log line was read by the collection process, and the routing identifier field is set to the operator's seat identifier. The log message is then written to the same buffer storage as the operation control data. In one implementation, the log message is written to a log topic partition in the buffer storage that is independent of the operation control data. This log topic partition shares the same retention period configuration as the message topic partition where the operation control data resides. During retrieval, log fragments belonging to the same work order candidate time window are filtered out by combining the topic partition and the routing identifier. In another implementation, when the log generation rate exceeds a preset threshold, the resident log collection process merges multiple log lines into a batch log message before writing it to the buffer storage. The load of the batch log message contains multiple log lines and their respective timestamps. The proxy receiving time in the envelope information of the batch log message is taken as the reading time of the first log line in the batch message, so as to reduce the writing pressure on the buffer storage caused by high-frequency log writing.In another implementation, the resident log collection process maintains a log sending cache on the local disk. When the network connection is interrupted, newly generated log lines are temporarily stored in this local sending cache. After the network is restored, they are sent to the buffer storage in order. The maximum size of this local sending cache is configurable. When the cache overflows, the oldest log lines are discarded according to their generation time to ensure the memory stability of the collection process. By writing the remote operator's log data into the same buffer storage as the operation control data in the form of log messages, the log data and message data are uniformly arranged in the time dimension. When a work order is closed, message data and log data are retrieved from the buffer storage simultaneously according to the candidate time window. This avoids the complex data association problems caused by reading from different storage systems separately. At the same time, resident log collection ensures that the operator's system status log already exists in the buffer storage before the work order is generated, providing contextual information from the operator's side for troubleshooting analysis.

[0049] In some embodiments, in response to the generation event of a remote assistance work order, driving scenario analysis data can be collected and written to persistent storage. In some embodiments, in response to a playback event, real-time driving video data can be recorded and written to persistent storage.

[0050] The volume of driving scene analysis data and driving real-time video data in driving scene data is significantly larger than that of operation and control data. Driving real-time video data, in its original encoding format, can reach bitrates of several megabits per second. If this data is written to the same buffer storage as the operation and control data, continuous recording for tens of minutes would exhaust the limited storage capacity of the buffer storage, causing high-frequency status messages in the buffer storage to be prematurely discarded by the video data, resulting in compromised message data integrity and availability. On the other hand, driving scene analysis data and driving real-time video data only have archiving value during the existence of remote assistance work orders. Driving scene analysis data and video data from before the work order is generated, when the vehicle is not in a state awaiting rescue, do not need to be included in the work order package. Therefore, these two types of data do not need to be collected continuously like operation and control data. To solve the problem of premature message discarding caused by writing video data and message data to the same buffer storage, and the problem that driving scene data does not need to be collected continuously, a method is provided to write driving scene data to persistent storage separately after work order generation and after playback event triggering. In this method, the acquisition of driving scenario analysis data is triggered by the generation event of a remote assistance work order. This driving scenario analysis data is used to characterize the structured analysis data generated by the vehicle's autonomous driving software stack during operation. This structured analysis data includes at least one or more of the following: obstacle list and lane line information output by the perception module, driving trajectory planning results output by the decision planning module, and vehicle pose information output by the localization module. These data are serialized in a structured format frame by frame or periodically and output as independent stripes. When the acquisition link receives the generation event of a remote assistance work order, the generation event carries a work order identifier and a vehicle identifier. The acquisition link starts a bypass acquisition session for driving scenario analysis data using the work order identifier as the associated primary key. This bypass acquisition session writes the driving scenario analysis data from the vehicle to the analysis data storage area in persistent storage in stripes through a write channel independent of the buffer storage. This stripe is used to characterize data blocks aggregated according to a fixed time window or a fixed number of frames. Each stripe contains the structured data of all frames within that time window and the corresponding timestamp index. The acquisition of real-time driving video data is triggered by a playback event. This playback event is issued by the dispatch system when the remote operator starts watching the vehicle video stream. The playback event carries the pull stream address parameter and the channel identifier parameter. The pull stream address parameter is used to represent the source address of the video stream, and the channel identifier parameter is used to represent the video channel number corresponding to the current pull stream.After receiving a playback event, the acquisition link establishes a bypass streaming session with the video encoding device at the vehicle end based on the streaming address parameters. This bypass streaming session is independent of the real-time streaming session used by the remote operator. The bypass streaming session obtains the original encoded video stream data from the video encoding device and writes the video stream data to the video storage area in persistent storage using a packaged copy method. This packaged copy method indicates that the acquisition link does not perform transcoding on the video stream's encoding format; it only packages the received video stream data into a media container format according to the order of reception before storing it. By writing driving scene analysis data to persistent storage triggered by a work order generation event and driving real-scene video data to persistent storage triggered by a playback event, driving scene data and operation control data are diverted to different storage media and different write channels. Large blocks of video data and strip data no longer crowd out the limited reserved capacity of the buffer storage. At the same time, driving scene data is not collected before a work order is generated and is stopped after the work order is closed, avoiding invalid data from irrelevant periods occupying persistent storage space.

[0051] S202: In response to the closure of the remote assistance work order, retrieve the work order identifier, vehicle identifier, and remote operator identifier data with the routing identifier of the remote assistance work order from the buffer storage, and aggregate them with the driving scene data associated with the work order identifier in the persistent storage to generate a playback data packet of the remote assistance work order in the persistent storage.

[0052] After a remote assistance work order is closed, the buffer storage retains all remote assistance process data continuously collected throughout the work order's lifecycle, while the persistent storage retains all driving scene data collected during the work order period. In existing technologies, after a work order is closed, these two types of data are typically stored independently: message data and log data are stored in archived copies in the buffer storage, while driving scenario analysis data and driving real-time video data are stored in different directories in the persistent storage. There is a lack of a unified primary key to establish time alignment and content mapping between the two. When the work order needs to be replayed later, the playback tool needs to load data from multiple storage locations and perform timeline alignment manually. However, message data is recorded with millisecond-level timestamps, driving scenario analysis data with frame numbers, and driving real-time video data with timecodes. The inconsistency in these time bases leads to accumulated alignment errors, resulting in a misalignment of several to dozens of frames between the vehicle status displayed on the playback screen and the actual control commands. To address the issue of scattered storage of remote assistance process data and driving scene data after a work order is closed, leading to inaccurate alignment of the playback timeline due to the lack of a unified association key, a method is provided to aggregate data and generate a playback data package using the work order identifier as the association key after the work order is closed. This method, upon the arrival of the remote assistance work order closure event, uses the work order identifier as the aggregation key. It extracts all remote assistance process data from buffer storage that matches the work order identifier, the vehicle identifier, and the remote operator's identifier. Simultaneously, it reads driving scene analysis data and driving real-time video data stored in persistent storage, with the work order identifier as the directory name or filename prefix. The data extracted from buffer storage and read from persistent storage are aggregated according to the work order identifier, thus associating all remote assistance process data and all driving scene data belonging to the same work order identifier into the same data set. Finally, this aggregated data set is packaged into a playback data package file and written to the archive area in persistent storage. The playback data package file is stored in persistent storage with the work order identifier as its filename or directory name. When the playback tool loads this playback data package subsequently, it can read all messages, logs, driving scenario analysis data, and index information of the driving scene video data from the same file. In one implementation, the aggregation operation is performed after data retrieval from buffer storage and before writing to persistent storage; that is, the remote assistance process data and driving scene data are first merged in memory by timestamp sorting before being serialized into the playback data package file. In another implementation, the aggregation operation uses a reference aggregation method, where the playback data package file only stores the storage path index of the driving scene data and the offset index of the remote assistance process data; the actual data remains stored in its original storage location, and during playback, data is read from the original location using the index.By using the work order identifier as the aggregation primary key to associate the remote assistance process data in the buffer storage with the driving scene data in the persistent storage and package them into a unified playback data package, the problem of inconsistent alignment benchmarks caused by the scattered storage of multi-source data is eliminated from the data organization level. After loading the playback data package, the playback tool can directly play back from the same data set according to a unified timeline without having to switch between multiple storage systems to read and manually align the time benchmark.

[0053] In some embodiments, the start and end times of each remote assistance work order can be determined based on the generation and closing events in the lifecycle events of each remote assistance work order.

[0054] Before retrieving remote assistance process data from the buffer storage using the work order identifier as the aggregation primary key, it is necessary to determine the start and end times of reading data from the buffer storage to limit the data range of the retrieval operation. In existing technologies, a common practice is to use the receipt time of the closure event as the retrieval end time and the receipt time of the generation event as the retrieval start time. The problem with this approach is that there are network transmission delays and scheduling decision delays between the receipt time of the generation event and the actual time when the vehicle begins to show signs of being stranded. Using the generation event as the start time will result in the loss of stranding cause data already reported to the buffer storage before generation, while using the receipt time of the closure event as the end time will truncate messages that have entered the message broker forwarding queue before the closure event is issued but have not yet fallen into the buffer storage. To solve the problems of lost stranding context before the work order and truncation of messages in transit when closing due to improper selection of the retrieval time window boundaries, a method is provided to determine the start and end times of the work order based on the generation and closure events in the work order lifecycle events. In this method, the acquisition link continuously receives work order lifecycle events broadcast by the scheduling system during operation. These lifecycle events include at least a generation event and a closing event. The generation event represents the moment the scheduling system creates a remote assistance work order, and the closing event represents the moment the scheduling system terminates a remote assistance work order. The acquisition link maintains a case start time field and a case end time field for each work order. When a generation event arrives, the event occurrence time carried by the generation event is written into the case start time field. This event occurrence time is recorded in the event payload by the scheduling system when the work order is generated, representing the actual creation time of the work order in the scheduling system. When a closing event arrives, the event occurrence time carried by the closing event is written into the case end time field. This event occurrence time is recorded in the event payload by the scheduling system when the work order is closed, representing the actual closing time of the work order in the scheduling system. The case start time represents the logical start boundary of the remote assistance service corresponding to the work order in the scheduling system, and the case end time represents the logical end boundary of the remote assistance service corresponding to the work order in the scheduling system. In one implementation, when a generation event occurs before the acquisition link starts (i.e., the generation event of a historical work order is received after the acquisition link starts), the acquisition link uses the event occurrence time in the generation event payload as the case start time, and does not use the acquisition link start time as the start boundary. In another implementation, when a closure event arrives late and the work order has already generated a presumed closure durability packet through the presumed closure mechanism, after the actual closure event arrives, the case end time field is updated with the event occurrence time carried by the actual closure event, and subsequent extraction operations are re-executed.By extracting the event occurrence time recorded by the scheduling system from the load of the generated event and the closed event respectively as the start and end time of the work order, the start and end time is strictly aligned with the business boundary of the scheduling system and is not affected by the reception time delay of the acquisition link itself. At the same time, the case start time is earlier than the time when the acquisition link receives the generated event, so that the trapped cause data that already exists in the buffer storage before the generated event is received can be included in the extraction range.

[0055] In some embodiments, an extraction time window can be obtained by forward expansion of a first preset duration and backward expansion of a second preset duration based on the start and end times.

[0056] Before the start of the work order, the data on the cause of the vehicle's entrapment reported may fall into the buffer storage milliseconds to seconds earlier than the time the event was sent. This data is crucial for analyzing why the vehicle triggered remote assistance. After the end of the work order, the tail messages that entered the message broker forwarding queue before the close event was sent may fall into the buffer storage milliseconds to hundreds of milliseconds after the close event is received. This data contains the confirmation response of the last frame of control instructions in the work order or the feedback of the vehicle's execution result, which is essential for confirming whether the remote assistance was successful. If the extraction time window is strictly equal to the time from the start to the end of the case, neither of the above two parts of data can be included in the extraction scope. To solve the problem that key context data on both sides of the start and end time boundaries cannot be included in the extraction due to network transmission delays, a method is provided to obtain the extraction time window by forward extension of a first preset duration and backward extension of a second preset duration based on the start and end times. This method uses the case start time as a reference point, subtracting a first preset duration forward from this reference point to obtain the left boundary of the extraction window. This first preset duration characterizes the additional data time range that needs to be included before the case start time. The value of this first preset duration is at least greater than the maximum network transmission delay from the vehicle reporting the message to the message broker receiving the message, ensuring that the distress cause data reported before the work order is triggered is completely included in the extraction window. Using the case end time as a reference point, adding a second preset duration backward from this reference point to obtain the right boundary of the extraction window. This second preset duration characterizes the additional data time range that needs to be included after the case end time. The value of this second preset duration is at least greater than the maximum network transmission delay from the dispatch system sending a closure event to the acquisition link receiving the closure event, ensuring that messages already in transit in the forwarding queue before the closure event is sent are completely included in the extraction window. The left and right boundaries of the extraction window together define an extraction time window, which characterizes the time range for reading remote assistance process data from the buffer storage. In one implementation, the first and second preset durations are configured with the same fixed value. In another implementation, the first preset duration and the second preset duration are configured independently. The first preset duration is set based on the statistical delay distribution of the uplink at the vehicle end, while the second preset duration is set based on the statistical delay distribution of the downlink in the dispatching system. Their values ​​do not affect each other. In yet another implementation, when the work order duration is extremely short (i.e., the difference between the case end time and the case start time is less than the sum of the first and second preset durations), the total span of the extraction time window remains the first preset duration plus the second preset duration, without compressing the total width of the extraction window due to the short work order duration.By extending the first preset duration forward and the second preset duration backward, additional data redundancy intervals are set on both sides of the extraction time window, so that key data that arrives in the buffer storage earlier than the generation event or later than the closing event due to network transmission delay can still be included in the extraction range. At the same time, the independent configuration of the first and second preset durations allows the delay characteristics of the uplink and downlink to be adapted separately, avoiding insufficient or excessive redundancy caused by unified configuration.

[0057] In some embodiments, in response to the arrival of the end time of the extraction time window, data including the work order identifier, vehicle identifier, and remote operator identifier of the remote assistance work order can be extracted from the remote assistance process data received within the extraction time window.

[0058] Once the left and right boundaries of the extraction time window are determined, the timing of reading data from the buffer directly impacts data integrity. If the read operation is performed before the right boundary of the extraction time window is reached, messages in transit falling within the second preset time range of backward expansion will not be fully written to the buffer, resulting in missing data in the read result. If a fixed delay is waited before reading, this delay may be shorter than the second preset time, leading to incomplete data, or it may be much longer, causing data near the left boundary of the extraction window in the buffer to be discarded and unreadable. To address the problem of premature extraction resulting in messages in transit within the backward expansion range not being fully written to the buffer, and the contradiction between data integrity and data viability that a fixed delay time cannot balance, a method is provided that reads are performed only when the end of the extraction time window is reached. After the right boundary of the extraction time window is determined, the acquisition link does not immediately perform a read operation on the buffer storage. Instead, it continuously monitors the current system time. When the current system time reaches or exceeds the time corresponding to the right boundary of the extraction time window, this time is used as the trigger time. Starting from this trigger time, all remote assistance process data whose received time falls within the extraction time window are read from the buffer storage. The end time of the extraction time window is used to characterize the latest time point at which the last piece of data within the extraction time window may fall into the buffer storage. Before the end time is reached, data within the backward extension of the second preset time range is still being written to the buffer storage. In one implementation, while waiting for the end of the extraction time window, the acquisition link checks at fixed heartbeat intervals whether data near the left boundary of the extraction window in the buffer is still readable. When the difference between the left boundary of the extraction window and the earliest currently readable time in the buffer is detected to be less than a safety threshold, the acquisition link reads the data between the left boundary of the extraction window and the current time in segments and temporarily stores them in a local temporary file. After the end time arrives, the data in the temporary file is merged with the data read after the end time according to the receiving time, to prevent the left boundary of the extraction window from being removed by the buffer's eviction policy during the waiting period for the end time. In another implementation, when the difference between the system time and the end time of the extraction time window is less than a preset timeout threshold, the acquisition link considers that the end time can no longer be waited for. At this time, it reads all the data that has fallen into the buffer storage with the current time as the truncation boundary, and marks the data in the backward extension range that could not be read as truncated and lost, recording it in the archived metadata of the work order. By responding to the end of the extraction time window before reading, it ensures that all messages in transit within the second preset time range of backward expansion have sufficient time to be written from the message broker forwarding queue to the buffer storage before being read. At the same time, the protection mechanism of segmented reading before the end time is reached solves the problem of data expiration and elimination at the left boundary of the extraction window during the waiting period.

[0059] In some embodiments, if the routing identifier in the remote assistance process data is the work order identifier of the remote assistance work order or the vehicle identifier of the vehicle corresponding to the remote assistance work order, the remote assistance process data can be extracted.

[0060] When reading remote assistance process data from the buffer storage, the reading result includes all messages received within the extraction time window, but only a portion of these messages belong to the current work order. Specifically, multiple work order lifecycle events may exist simultaneously within the extraction time window, status messages reported by different vehicles are interspersed, and control commands from multiple remote operators overlap on the timeline. If all messages within the extraction time window are included in the playback data package without distinction, the playback will present a jumble of information from all vehicles, all work orders, and all operators within that time period, making it impossible to focus on the complete context of the current work order. To address the problem of being unable to focus on the current work order due to the jumble of multi-source data within the extraction time window, a method is provided to extract data matching the work order identifier, vehicle identifier, and remote operator identifier of the current work order from the remote assistance process data within the extraction time window. This method uses the work order identifier of the current remote assistance work order, the vehicle identifier of the vehicle corresponding to the current work order, and the remote operator identifier in the seat service interval sequence of the current work order as filtering conditions to determine the attribution of each record read from the buffer storage. During the attribution determination process, the receiving time falling within the extraction time window is a prerequisite for inclusion, but not a condition for inclusion. The inclusion decision is determined by the matching relationship between the routing identifier in the envelope information of the record and the aforementioned filtering conditions. For records where the routing identifier equals the work order identifier, the record is included in the current work order's data set. For records where the routing identifier equals the vehicle identifier, the record is included in the current work order's data set. For records where the routing identifier equals the remote operator identifier, the acquisition link further determines whether the receiving time of the record falls within the service period of the remote operator in the current work order. If it does, it is included; otherwise, it is not. For records with an empty routing identifier, the acquisition link parses the payload content of the record to extract the work order identifier, vehicle identifier, and remote operator identifier carried in the payload. Then, it determines whether to include the record based on the matching relationship between the identifiers in the payload and the work order identifier, vehicle identifier, and remote operator identifier in the current work order's service interval sequence. In one implementation, for records where the routing identifier equals the work order identifier and records where the routing identifier equals the vehicle identifier, both types of records are included within the extraction time window without additional time period clipping conditions. In another implementation, for a record whose routing identifier is equal to the remote operator identifier, if the routing identifier of the record matches multiple seat identifiers in the current work order's seat service interval sequence, then the record can only be included if it is determined to fall within each matching seat service period. That is, the timestamp of the record needs to satisfy multiple time period constraints at the same time.By using the above extraction method with three levels of filtering conditions—work order identifier, vehicle identifier, and seat service area—only records with an entity association with the current work order within the extraction time window are included in the data set. Work order lifecycle events of other work orders, status messages of other vehicles, and control instructions of other seats are excluded, ensuring the purity of the content of the current work order playback data package.

[0061] In some embodiments, if the routing identifier in the remote assistance process data is a remote operator identifier, the service period of each remote operator for the remote assistance work order can be determined based on the work order assignment event and work order departure event of each remote operator.

[0062] In remote assistance process data, records where the route identifier equals the vehicle identifier carry vehicle operation status information such as vehicle location, speed, fault codes, obstacle perception, and decision-making and planning status. This information has already been reported to the buffer storage before the work order is generated, and it is direct evidence for analyzing why the vehicle triggered remote assistance. Simultaneously, records where the route identifier equals the work order identifier carry the work mode switching instructions, resource allocation instructions, and priority adjustment instructions issued by the scheduling system for that work order. These instructions are carried out throughout the entire lifecycle of the work order. In existing technologies, records where the route identifier equals the vehicle identifier and the work order identifier are processed using the same service period pruning method as the seat identifier records. That is, constraints on the time margin before and after are also applied to the vehicle identifier, causing the status data reported by the vehicle before the work order is generated to be discarded because it falls outside the margin range, resulting in a break in the evidence chain for analyzing the cause of the entrapment. To solve the problem of discarding pre-work order status data due to applying the same pruning constraints as the service period to vehicle identifier and work order identifier records, a method for full extraction window extraction of records where the route identifier is the work order identifier and the vehicle identifier is provided. In the attribution determination process, this method includes all records within the extraction time window of a record whose routing identifier equals the work order identifier of the current work order, without imposing any time-based pruning conditions. Similarly, for records whose routing identifier equals the vehicle identifier corresponding to the current work order, all records within the extraction time window are included in the data set, regardless of whether the record's reception time falls within a specific seat service period. Full extraction window extraction characterizes records where, throughout the entire extraction time window, the routing identifier matches either the current work order identifier or the current vehicle identifier, regardless of the record's specific position within the extraction time window—whether it's in the pre-margin interval, the work order start / end interval, or the post-margin interval—and is included in the replay data packet. In one implementation, for records whose routing identifier equals the vehicle identifier, if the reception time of some vehicle status records within the extraction time window falls outside the buffer retention period and cannot be read, they are supplemented from the fallback data of the continuous packets according to the same full extraction window rule. In another implementation, for records where the routing identifier is equal to both the vehicle identifier and the work order identifier (i.e., the routing identifier field in the envelope information of the record is set to the vehicle identifier but the work order identifier field in the load is consistent with the current work order identifier), the record enters the vehicle identifier branch according to the routing identifier branch and performs full extraction window extraction, without transferring to the work order identifier branch for secondary judgment. By using the full extraction window extraction method for records where the routing identifier is equal to both the work order identifier and the vehicle identifier, key context data located on both sides of the start and end times, such as the vehicle fault code and pending assistance signal reported before the work order is generated, and the mode switching confirmation instruction issued after the work order is closed, are completely preserved in the playback data packet, without being restricted by the seat service period pruning rules, ensuring the integrity of the evidence for the analysis of the cause of the entrapment.

[0063] In some embodiments, data received during the service hours of each remote operator can be extracted from the remote assistance process data.

[0064] In remote assistance process data, records where the route identifier equals the remote operator's identifier carry control commands such as steering wheel angle, throttle opening, and brake pressure. The data volume of this type of record is far greater than that of vehicle status records, and the same operator may serve multiple work orders sequentially within a short period. If records where the route identifier equals the remote operator's identifier are also extracted using a full extraction window, when the same operator serves other work orders within the pre- or post-extraction margin of the extraction time window, the control commands of those other work orders will be incorrectly included in the current work order's data set, causing cross-work order control data retrieval. If no operator's identifier records are extracted within the entire extraction time window, all control commands issued by the operator corresponding to the current work order during the service period will be lost, making it impossible to recover the operator's decision-making behavior. To solve the cross-work order data retrieval problem caused by full-window extraction of operator's identifier records and the control command loss problem caused by not extracting them, a method is provided to determine the service period based on the work order assignment event and work order departure event of each remote operator and then extract the data within that period. In this method, the acquisition link maintains a seat service interval sequence for each work order. This seat service interval sequence is updated driven by the seat allocation event and seat departure event received during the work order's existence. For a work order, each remote operator corresponding to the work order has zero or one or more service intervals in this sequence. Each service interval includes the remote operator's seat identifier, interval start time, and interval end time. The interval start time is the time when the remote operator is assigned to the work order, and the interval end time is the time when the remote operator leaves the work order, or when the work order is closed, carries the event. When determining the attribution of records where the routing identifier equals the remote operator's identifier, the acquisition link extracts the routing identifier field and the proxy reception time field from the envelope information of the record. The value of the routing identifier field is used to search for a matching seat identifier in the seat service interval sequence. When a matching seat identifier is found, it is further determined whether the proxy reception time of the record falls between the start and end times of the service interval corresponding to that seat identifier. If it does, the record is included in the current work order's data set; otherwise, it is not included. For unclosed intervals that have not yet received a departure event, their end time is considered positive infinity, and records with proxy reception times greater than the start time of the interval are included. In one implementation, to accommodate minor timing deviations between allocation events and control commands, when determining whether a proxy reception time falls within a service interval, a configurable delay margin is added forward to the start time of the interval, and this delay margin is added backward to the end time of the interval. That is, records with proxy reception times between the start time of the interval minus the delay margin and the end time of the interval plus the delay margin are included.In another implementation, when the same seat identifier exists in multiple discontinuous service intervals within the seat service interval sequence, records whose agent reception time falls within any one of these service intervals and its delay margin range are included, without needing to fall within all intervals simultaneously. Through the extraction method described above, which determines the service period based on allocation and departure events and uses the agent reception time falling within that period as the inclusion condition, seat identifier records are precisely limited to the time range during which each seat actually serves the current work order. Control instructions from that seat serving other work orders within the work order's pre- and post-margin ranges are not included, while control instructions from that seat serving the current work order are fully captured.

[0065] In some embodiments, if the routing identifier in the remote assistance process data is empty, the work order identifier, vehicle identifier, and remote operator identifier in the remote assistance process data payload can be parsed. In some embodiments, data matching the remote assistance work order can be extracted from the remote assistance process data based on the parsing results.

[0066] In remote assistance process data, some records contain empty routing identifier fields in their envelope information. These empty routing identifier records originate from sources including, but not limited to: internal system heartbeat messages where the message publisher did not set a routing key field when publishing the message; exception messages where the message broker defaults to filling in empty values ​​when routing key retrieval fails; and compatibility messages sent by some legacy system components according to older protocols that do not carry routing identifiers. In existing technologies, common approaches to handling empty routing identifier records are either to discard them directly or to retain them all within a time window. Directly discarding them results in the permanent loss of crucial information such as work order identifiers, vehicle identifiers, or seat identifiers that may be carried in these records. If this record happens to be the last fault code before the vehicle triggered remote assistance, discarding it will lead to a lack of crucial evidence in the analysis of the cause of the entrapment. Retaining them all within a time window mixes all empty routing identifier records for all vehicles, all work orders, and all seats within the retrieval time window into the current work order's data set, most of which are irrelevant to the current work order, causing data packet bloat and noise interference. To address the issues of losing crucial evidence due to directly discarding empty route identifier records and the noise inflation caused by retaining all records, this method provides a method for parsing entity identifiers from the load of empty route identifier records for matching and extraction. When the method detects that the route identifier field in the envelope information of a record is empty during the attribution determination process, it does not perform a fast matching branch based on the route identifier for that record, but instead enters a load parsing branch. In the load parsing branch, the acquisition link parses the load content of the record according to preset protocol parsing rules. These preset protocol parsing rules are used to extract the work order identifier field, vehicle identifier field, and remote operator identifier field from the load of different message types. After parsing, the acquisition link compares the work order identifier extracted from the load with the work order identifier of the current work order. If they are equal, the record is included in the dataset. If the work order identifier in the load is not equal to the current work order identifier, or if there is no work order identifier field in the load, the acquisition link compares the vehicle identifier extracted from the load with the vehicle identifier of the current work order. If they are equal, the record is included. If the vehicle identifier in the load does not match or there is no vehicle identifier field in the load, the acquisition link will compare the remote operator identifier extracted from the load with the seat identifiers in the seat service interval sequence of the current work order one by one. If a matching seat identifier exists and the proxy receiving time of the record falls within the service interval corresponding to the seat identifier, it will be included. In one implementation, when load parsing fails, i.e. the load does not conform to any known message protocol format, the acquisition link will include the record by default in the dataset of the current work order and merge it to generate an alarm log. This alarm log is used to notify system maintenance personnel that there is an unresolved empty route identifier message, for subsequent manual inspection to see if the message is critical data necessary for evidence collection.In another implementation, when load parsing is successful but the work order identifier, vehicle identifier, and remote operator identifier in the load cannot match the corresponding identifier of the current work order, the acquisition link discards the record and does not include it in the dataset, and records an unmatched log for offline analysis. Through the above extraction method of cascading load parsing of work order identifiers, vehicle identifiers, and remote operator identifiers and matching them step by step, the portion of the empty route identifier record that has an entity association with the current work order is included in the dataset, while the unrelated portion is excluded, achieving a balance between effective forensic utilization of empty route identifier records and noise filtering.

[0067] In some embodiments, archived data packets of remote assistance work orders can be generated and stored in persistent storage.

[0068] After the extraction process based on route identifier branch determination and load parsing matching, the data extracted from the buffer storage has been filtered and aggregated, forming a complete remote assistance process dataset for the current work order. At this point, the dataset is in a scattered message object state in memory and has not yet been persisted as a unified file entity that can be directly loaded by the playback tool. If this dataset is kept in a scattered state and written to persistent storage, the playback tool needs to traverse multiple files and re-establish the temporal order association between messages when loading, increasing the latency of playback startup. At the same time, the scattered files are prone to becoming unusable due to the loss or corruption of some files during long-term storage. After completing the attribution filtering extraction, all remote assistance process data included in the current work order can be arranged in ascending order according to the agent's reception time to form an ordered message sequence. This ordered message sequence can be converted into an archive data packet file. The archive data packet file contains a file header and a message body. The file header is used to store metadata information such as the work order identifier, vehicle identifier, start and end time, total number of messages, and index offset of the current work order. The message body is used to store the original payload content of each message and its corresponding envelope information. After serialization, the acquisition link writes the archived data packet file to an archive directory in persistent storage. This archive directory is named after the work order identifier or contains the work order identifier as part of the filename. In persistent storage, the archived data packet file, driving scene analysis data, and driving real-time video data of the same work order are stored in the same parent directory, and all three are associated with the same work order identifier. In one implementation, the serialization process uses an incremental write method, that is, each record determined to be included during the attribution filtering extraction process is appended to the end of the archived data packet file, without waiting for all records to be determined before writing all at once, in order to reduce memory usage. In another implementation, the serialization process divides the message body into multiple fragment files and stores them separately. The file header records the time range index of each fragment file. During playback, only the corresponding fragment file is loaded according to the playback start and end times, in order to reduce the latency of loading large files. In yet another implementation, after the archived data packet file is written to persistent storage, a file checksum is calculated. This file checksum is associated with the work order identifier and stored in an archived list file for subsequent reading to verify the integrity of the archived data packet.

[0069] In some embodiments, the driving real-time video data in the archived data package can be transcoded to obtain a playback data package.

[0070] After the archived data package file is written to persistent storage, the driving scene video data contained in the file still retains the original encoding format recorded at the time of acquisition. This original encoding format is determined by the video encoding equipment at the vehicle end, and different vehicle models and batches may use different encoding standards. If the video data in the original encoding format is directly used for playback tools, the playback tool needs to integrate multiple decoders to adapt to different encoding formats, which is complex and difficult to maintain. At the same time, the video data in the original encoding format is usually high bitrate and high-definition, and its data volume is much larger than the image quality level required for playback preview. In network transmission playback scenarios, directly transmitting the video stream in the original encoding format will consume a lot of bandwidth and increase the waiting time for playback to start. To solve the problem of poor decoding compatibility of playback tools due to the inconsistent encoding standards of the original encoding format video data and the problem of high network transmission bandwidth consumption due to the high bitrate of the original encoding format, this section provides a method for transcoding the driving scene video data in the archived data package to obtain the playback data package. This method involves extracting the storage path index of the driving scene video data from the archived data packet file after it has been written to persistent storage. Based on this index, the acquisition link or a separate transcoding service reads the original encoded video file from the video storage area and initiates a transcoding task. This task decodes the original encoded video file into a sequence of original image frames, and then re-encodes this sequence into a preview video file in a target encoding format according to a preset unified encoding parameter. This parameter includes at least the target encoding standard, target resolution, target frame rate, and target bitrate. The target encoding standard is independent of the original encoding format, and all preview video files for work orders use the same target encoding standard, target resolution, and target bitrate. Because the message and log data in the archived data packet are no longer subject to the buffer retention period after being written to persistent storage, the queuing time and execution time of the transcoding task, even if exceeding the limited retention period of the buffer, will not result in the loss or unavailability of archived work order messages. After transcoding is complete, the acquisition link or transcoding service re-aggregates the transcoded target encoding format preview video file with the message data and log data in the archived data packet file and the driving scenario analysis data in persistent storage to generate a playback data packet file. The video portion of this playback data packet file references the transcoded preview video file, not the original encoding format video file. In one implementation, the transcoding task is triggered immediately after the archived data packet file is generated, without blocking the persistent write operation of the archived data packet file.In another implementation, when the transcoding task fails, the acquisition link or transcoding service retains the original archived data packet file and marks the transcoding failure status in the archived metadata, allowing operators to manually re-trigger the transcoding task later. Upon successful transcoding, a playback data packet file is generated to overwrite the previous state. The compositing process after successful transcoding only reads the archived data packet file and the transcoded preview video file from persistent storage, without reading back from the buffer storage. Therefore, even if the data in the buffer storage has been evicted, it does not affect the final generation of the work order playback data packet. In yet another implementation, the transcoding task only performs transcoding on video files generated by bypass recording sessions triggered by playback events in the driving scene video data. Structured data in the driving scene analysis data is not transcoded, and this structured data retains its original format in the playback data packet. By performing transcoding processing on the driving scene video data independently of the archive package to generate playback data packets, the playback tool only needs to support a unified target encoding standard to play the preview videos of all work orders. It is no longer troubled by the differences in the original encoding standards on the vehicle side. At the same time, the bitrate of the transcoded preview video file is reduced to a fraction of the original bitrate, reducing network bandwidth consumption and startup waiting time during playback. Furthermore, since the transcoding operation is decoupled from message durability, even if the preview transcoding time is longer than the buffer layer retention period, it will not cause the work order message to be lost. The synthesis after successful transcoding is always based on the archive data packets and transcoding results in persistent storage, without the need to retrieve expired buffer data.

[0071] In some embodiments, determining the service period of each remote operator for a remote assistance work order based on the work order assignment event and work order departure event of each remote operator includes: in response to receiving a work order assignment event from a first remote operator, closing the service period of one or more unclosed second remote operators with the time corresponding to the work order assignment event as the end time, and using the time corresponding to the work order assignment event as the start time of the first remote operator's service period; and in response to receiving a work order departure event from a first remote operator, using the time corresponding to the work order departure event as the end time of the first remote operator's service period.

[0072] During the lifecycle of a remote assistance work order, the same work order may be sequentially assigned to multiple remote operators due to operator handover, skill matching, or load balancing, forming a multi-operator relay service scenario. When a work order is reassigned from the second remote operator to the first remote operator, if the scheduling system fails to issue a departure event for the second remote operator due to network jitter or logical anomalies, the service period of the second remote operator will remain unclosed. If a new interval is opened by directly using the time corresponding to the work order assignment event as the start time of the first remote operator's service period without closing the unclosed interval of the second remote operator, two service intervals will appear simultaneously on the same work order. This will cause the seat identifier record falling within the overlapping time period to match two service intervals during subsequent attribution determination, resulting in duplicate inclusion of control instructions or attribution ambiguity. To solve the problem of multiple service intervals being simultaneously open due to missed departure events in reassignment scenarios, a service period update method is provided that first closes the unclosed interval and then opens a new interval. In this method, the acquisition link maintains a service period sequence for each remote assistance work order. Each service period in the sequence represents the continuous time range during which a remote operator provides services for that work order. Each service period includes the remote operator's seat identifier, the start time, and the end time. An unclosed end time indicates that the remote operator is still providing services for the work order. When the acquisition link receives a work order allocation event from the first remote operator, this event carries the first remote operator's seat identifier and the event's occurrence time. Using this occurrence time as the end time, the acquisition link traverses the work order's service period sequence, searching for all service periods with unclosed end times. The end times of all found unclosed service periods are then uniformly set to the occurrence time. After closing all unclosed service periods, a new service period is created. The new service period has the first remote operator's seat identifier as its own, the occurrence time as its start time, and the unclosed end time as its end time. This new service period is then appended to the end of the service period sequence. When the acquisition link receives a work order departure event from the first remote operator, the work order departure event carries the seat identifier of the first remote operator and the time of occurrence of the departure event. The acquisition link takes the time of occurrence of the departure event as the end time, traverses the service time sequence of the work order, searches for service time periods in the sequence where the seat identifier is equal to the seat identifier of the first remote operator and the end time is in an unclosed state, and sets the end time of the found service time period as the time of occurrence of the departure event.In one implementation, when the acquisition link receives a work order allocation event from the first remote operator, if there are no service periods with an unclosed end time in the service period sequence of the work order, the service period for the first remote operator is created directly with the occurrence time of the allocation event as the start time, without performing a closure operation. In another implementation, when the acquisition link receives a work order allocation event from the first remote operator, if the seat identifier of the found unclosed service period is the same as the seat identifier of the first remote operator, that is, the scheduling system repeatedly issues the allocation event for the same seat without reassignment, the acquisition link still closes the unclosed service period first, and then reopens a new service period for the same seat with the occurrence time of the same allocation event as the start time, forming two consecutive intervals for the seat on the time axis. By adopting the above-mentioned update rules of closing all unclosed service periods in response to allocation events and opening new intervals, and closing matching unclosed service periods in response to departure events, it is ensured that at any time, there is at most one unclosed service period in the service period sequence of the work order. Even if the departure event is missed, the subsequent allocation event will automatically close the previous unclosed period, thus eliminating the data ownership overlap and ambiguity caused by multiple intervals being opened at the same time.

[0073] In some embodiments, the above-mentioned collection of driving scene data to persistent storage during a remote assistance work order includes: associating the driving scene data with the work order identifier of the remote assistance work order and writing it to persistent storage.

[0074] During the duration of a work order, driving scene data consists of driving scenario analysis data and driving real-time video data, which are collected under different triggering conditions. Driving scenario analysis data is triggered by a work order generation event, while driving real-time video data is triggered by a playback event. The start points of these two types of data may differ by several seconds to tens of seconds, but both types of data need to be associated with and archived along with the remote assistance process data of the same work order after the work order is closed. In existing technologies, driving scene data is typically written to persistent storage using vehicle identifiers or collection timestamps as filenames. After a work order is closed, a mapping relationship needs to be established by performing an association query between the two storage locations using the vehicle identifier and time range. However, the same vehicle may generate multiple work orders within a short period, and relying solely on the vehicle identifier and time range can lead to association errors or delays. To address the problem of the lack of a direct association between driving scene data and work order identifiers, which prevents rapid data location after work order closure, a method is provided to associate driving scene data with work order identifiers and write them to persistent storage. When the acquisition of driving scenario analysis data is triggered by a work order generation event, the method uses the work order identifier in the generation event. The acquisition link uses this work order identifier as the session identifier for the driving scenario analysis data acquisition session. When the acquisition session is written to persistent storage, a first-level storage directory is created using the work order identifier as the directory name, and all driving scenario analysis data stripes belonging to this work order are written to the analysis data subdirectory under this first-level storage directory. When the acquisition of driving real-scene video data is triggered by a playback event, the playback event uses the work order identifier and the channel identifier corresponding to the playback event. The acquisition link uses this work order identifier as the session identifier for the bypass recording session. When the bypass recording session is written to persistent storage, a first-level storage directory is found or created using the work order identifier as the directory name, and the driving real-scene video data files belonging to this work order are written to the video data subdirectory under this first-level storage directory. Video files from multiple seats or multiple channels are distinguished by the seat identifier or channel identifier plus the start time as the filename. In one implementation, a separate storage space named after the work order identifier is maintained for each work order in persistent storage. This separate storage space includes an analysis data subdirectory, a video data subdirectory, and a process data subdirectory. Driving scenario analysis data is written to the analysis data subdirectory, driving real-scene video data is written to the video data subdirectory, and extracted remote assistance process data is written to the process data subdirectory. The three subdirectories share the same parent directory. In another implementation, when driving scene data is written to persistent storage, each data block or video file has its work order identifier field written to its metadata header. Even if the data is moved to another storage location or copied to offline storage media, its association with the work order can be restored by reading the metadata header.By using the work order identifier as the directory name or metadata tag to associate driving scenario analysis data and driving real-world video data with the work order identifier and writing them to persistent storage, the aggregation operation after the work order is closed does not need to perform multi-dimensional association queries across storage locations. All driving scene data can be located directly using the work order identifier as the path prefix, which greatly reduces the time overhead of data aggregation and the risk of association errors.

[0075] In some embodiments, the method further includes: in response to the incomplete collection of driving scene analysis data or driving real-scene video data in the driving scene data, generating a playback data package based on the collected driving scene data and extracted remote assistance process data, and recording the data to be supplemented; in response to the completion of the collection of the data to be supplemented, regenerating the playback data package.

[0076] When archiving and playback data packets are generated after a work order is closed, the acquisition of some media may not be complete for various reasons. The acquisition of driving scenario analysis data relies on the software stack output at the vehicle end; at the moment the work order is closed, the last strip may not yet have been transmitted from the vehicle end to the acquisition link. The acquisition of driving scene video data relies on the encapsulation and copying operation of the bypass recording session; when the closure event arrives, the video encoding device may still be outputting video frames for milliseconds before the closure event, and the last video data segment may not yet have been completely written to persistent storage. If playback data packets are generated only after all media have been fully acquired, the waiting time depends on network transmission latency and video encapsulation speed. This waiting time may exceed the remaining retention period of remote assistance process data in the buffer storage, causing extracted message data to be permanently lost and evicted from the buffer storage during the waiting period. If playback data packets are generated before all media acquisition is complete, the missing driving scene data cannot be displayed during playback, reducing the integrity and availability of the playback data packets. To address the issues of message data expiration and loss due to waiting for all media acquisition to complete, and incomplete playback data packets due to delays in acquisition, a method is provided that generates playback data packets based on already acquired data, records items to be supplemented, and regenerates the data after the supplemented items are completed. After the work order is closed and the remote assistance process data is filtered and extracted, the acquisition link checks the acquisition status of the driving scenario analysis data and the driving real-world video data, identifying the completed and incomplete portions. The acquisition link aggregates the completed driving scenario analysis data strips and the completed driving real-world video data files with the extracted remote assistance process data to generate a first version of the playback data packet, which is then written to persistent storage. Simultaneously, a list of items to be supplemented is generated in persistent storage for the work order. This list records the identifiers of driving scenario analysis data strips that have not yet been acquired and the identifiers of driving real-world video data files that have not yet been fully written to persistent storage. When the acquisition link subsequently detects that any strip or video file recorded in the list of items to be supplemented has been acquired and written to persistent storage, the acquisition link reads the acquired strip or video file from persistent storage, merges it with the original data in the existing first version of the playback data packet, regenerates the second version of the playback data packet, and overwrites the first version of the playback data packet in persistent storage with the second version of the playback data packet. At the same time, the corresponding entry in the list of items to be supplemented is marked as completed. In one embodiment, when regenerating the playback data packet, the acquisition link does not reread the remote assistance process data from the buffer storage, but only appends the newly acquired driving scene data to the end of the existing playback data packet or inserts it into the corresponding time position to reduce the computational overhead of re-aggregation.In another implementation, when all entries in the list of items to be supplemented are marked as completed, the acquisition link marks the work order's archiving status as fully archived. When at least one item remains incomplete, the work order's archiving status is marked as partially archived. The playback data package in the partially archived state displays a data missing prompt on the playback interface when the playback tool loads. By generating playback data packages based on the collected data, recording items to be supplemented, and then regenerating and overwriting them after the supplementary items are completed, the remote assistance process data is immediately solidified into playback data packages after the work order is closed, unaffected by delays in driving site data acquisition. This ensures that message data is persisted in a timely manner before being evicted from buffer storage. Simultaneously, the item supplementation mechanism ensures that driving site data can be added to the playback data packages after subsequent acquisition, ultimately achieving the goal of complete archiving.

[0077] In some embodiments, the playback data packet includes the service time and reception time of each playback data, the reception time is used to determine the remote operator service period, and the service time is used for playback alignment.

[0078] In the playback data packets, each remote assistance process data entry contains two independent time fields: the business time and the reception time. The business time, populated by the original message publisher at the time of publication, characterizes the actual moment the business event described by the message occurred at the source, such as the vehicle location time in a vehicle status message or the joystick sampling time in a control instruction message. The reception time, generated by the message broker upon receiving the message and recorded in the message envelope information, characterizes the system moment the message was captured by the message broker's network access layer. These two time fields differ: the business time reflects the physical occurrence of the event at the source, while the reception time reflects the system's perception of the event. The difference between them includes source processing latency, network transmission latency, and message broker queuing latency, and is used to measure the system's end-to-end latency performance. When retrieving remote assistance process data from buffer storage and generating playback data packets, two time fields—the business time field and the reception time field—can be retained for each included message. The value of the business time field is extracted from the original business timestamp field in the message payload, and the value of the reception time field is extracted from the broker reception time field in the message envelope information. The metadata of the replay data packet records the indices of all messages in the work order, sorted by business time and by reception time. When loading the replay data packet, the replay tool selects different timeline benchmarks based on the type of replay scenario: in normal replay mode, business time is used as the timeline benchmark, and messages are presented sequentially according to their occurrence order at the source end to reconstruct the actual behavior sequence of the vehicle and operator; in latency audit mode, reception time is used as the timeline benchmark, and messages are presented in the order they are actually received by the system to determine the latency distribution and queuing status of each message in the transmission link. In service period determination, the acquisition link compares the reception time of each message with the start and end times of the seat service interval from the attribution filtering stage, rather than using business time for comparison. This is because the reception order of seat allocation events and departure events determines the service boundary perceived by the system, and using reception time as the benchmark for service period determination can eliminate the impact of source end clock deviation on the attribution determination result. In one implementation, the replay data packet stores a time deviation value for each message. This time deviation value is equal to the reception time minus the service time. The replay tool displays the time deviation distribution of all messages within the work order in the form of a bar chart or line graph on the replay interface, allowing maintenance personnel to quickly identify time periods with abnormal delays. In another implementation, when the service time field is missing or invalid in the message payload, the replay data packet sets the service time of the message to be the same as the reception time and records a time invalidation flag. The replay tool replaces the service time with the reception time during replay.

[0079] In some embodiments, the method further includes: temporarily storing the allocation event or the departure event of the remote assistance work order in response to the arrival of the allocation event or the departure event before the generation event; and determining the seat service period of the remote assistance work order based on the event time of the generation event when the generation event arrives.

[0080] In the actual operation of autonomous driving remote assistance systems, network transmission between the scheduling system and the data acquisition link is not always ideal. When the scheduling system generates a work order, the generation event of the work order needs to be forwarded by a message broker and transmitted through the network link before reaching the data acquisition link. Before the generation event arrives, the corresponding allocation event and departure event of the work order may have already arrived. In existing technologies, the data acquisition link usually assumes that lifecycle events arrive in the order of generation, allocation, departure, and closure. When the allocation event arrives before the generation event, since the data acquisition link has not yet created a file record for the work order, the seat identifier and allocation time in the allocation event have nowhere to be stored, causing the allocation event to be discarded. The service period of the seat at the actual start time of the work order cannot be recorded, and the control instructions of the seat cannot be included in the subsequent attribution determination due to the lack of service period. To solve the problem of event discarding and service period loss caused by the lack of work order file creation when the allocation event or departure event arrives before the generation event, a method is provided to temporarily store the first-arriving event and fill in the service period after the generation event arrives. In this method, the acquisition link maintains an unfiled event buffer. This buffer temporarily stores assignment and departure events whose work order identifiers do not exist in the current set of filed work orders. When the acquisition link receives an assignment or departure event, it extracts the work order identifier carried by the event and searches the set of filed work orders using the work order identifier as the key. If no corresponding work order file is found, the assignment or departure event is stored in the unfiled event buffer using the work order identifier as the grouping key. When the acquisition link subsequently receives a generation event for the same work order identifier, it uses the event occurrence time carried by the generation event as the case start time of the remote assistance work order, creates a file record for the work order, and then searches the unfiled event buffer for the temporarily stored assignment and departure events using the work order identifier as the key. Based on the event occurrence time carried by each temporarily stored event, these temporarily stored events are added to the work order's seat service time sequence in chronological order. In one implementation, when a generated event arrives, the undocumented event buffer temporarily stores multiple allocation events and departure events with the same work order identifier. The acquisition link arranges these temporarily stored events in ascending order of their occurrence time and then sequentially adds them to the service time sequence. If, during the arrangement process, there are two allocation events without departure events between them, the addition operation is performed according to the rule of first closing the unclosed interval and then opening a new interval. In another implementation, when a generated event fails to arrive within the timeout threshold, the acquisition link discards all the corresponding temporarily stored events in the undocumented event buffer using the work order identifier as the key and records a discard record in the system log. This discard record contains the work order identifier and the number of temporarily stored events, allowing maintenance personnel to check whether the scheduling system has generated an invalid work order identifier.In another implementation, when an allocation event arrives before a generation event, but the seat identifier field carried in the allocation event is empty or missing, the allocation event is still temporarily stored in the unfiled event buffer. When it is added after the generation event arrives, the empty seat allocation event is only used to close existing unclosed intervals and does not open new service periods. By temporarily storing the first-arriving events in the unfiled event buffer and adding them according to their own time after the generation event arrives, even if the allocation event and departure event arrive before the generation event due to network transmission disorder, the seat identifier and event time information contained in these events can still be completely preserved and added to the service period sequence, avoiding the problems of missing service periods and unassigned seat control data caused by event discarding.

[0081] In some embodiments, the method further includes: if the closing event of the remote assistance work order does not arrive within a preset time period, taking the event time of the last lifecycle event as the closing time of the remote assistance work order, performing the steps in response to closing the remote assistance work order, and marking the playback data packet as a presumed closure; if the closing event arrives after the event, taking the time of the closing event as the closing time of the remote assistance work order, and re-executing the steps in response to closing the remote assistance work order.

[0082] During the actual operation of a work order, the scheduling system may experience delays in delivering the closure event to the acquisition link due to service restarts, network partitions, or message queue backlogs. If the acquisition link waits indefinitely for the closure event, the bypass recording session and buffer storage read handle corresponding to the work order will continuously occupy system resources. Simultaneously, the archived data packet for the work order cannot be generated due to the lack of a closure event trigger, resulting in the data being permanently lost after the buffer storage retention period expires. If the work order is forcibly closed and archived data packets are generated directly at the current time when the closure event is missing, the end time of the archived data packet is not the actual time the scheduling system closed the work order. When the actual closure event arrives later, the generated data packet needs to be corrected. However, during the correction process, the data packet already written to persistent storage may have been read by the playback tool or distributed to downstream systems, thus compromising data consistency. To address the resource consumption and data loss issues when the closure event has not arrived for an extended period, as well as the data consistency issues when the actual event arrives after a forced closure, a method is provided that presumes closure, marks the presumed state, and re-executes archiving after the actual closure event arrives. In this method, the acquisition link maintains a last activity timestamp for each work order that has been archived but has not yet received a closure event. The last activity timestamp is updated to the time of occurrence of the event carried by the event each time the work order receives a lifecycle event. The acquisition link starts a timed scanning thread, which traverses all unclosed work orders at a fixed scanning interval. For each work order, it calculates the difference between the current system time and the last activity timestamp of the work order. When the difference exceeds a configurable timeout threshold, the acquisition link determines that the actual service of the work order has ended but the closing event is missing. Using the last activity timestamp as the estimated closing time, the following operations are performed: closing all unclosed service periods of the work order up to the estimated closing time, stopping all bypass recording sessions of the work order, forming an estimated extraction time window with the case start time of the work order as the left endpoint and the estimated closing time as the right endpoint, and applying preset durations before and after, and waiting for the estimated extraction time window to end, performing attribution filtering and archived data packet generation operations, writing an estimated closing status flag into the metadata of the generated archived data packet, and marking the archived data packet as estimated closing and storing it in persistent storage. When the acquisition link subsequently receives the actual closure event of the work order, the actual closure time is taken as the event occurrence time carried by the actual closure event. The lock on the presumed closed state archive data packet is released, and a pre- and post-preset duration is applied with the actual closure time as the right endpoint to form an actual extraction time window. After the actual extraction time window ends, the attribution filtering and archive data packet generation operations are re-executed. The newly generated archive data packet overwrites the presumed closed state archive data packet in persistent storage, and the presumed closure status flag in the metadata is updated to the actual closure status flag.In one implementation, when regenerating archived data packets after a real closure event, if the left boundary of the real retrieval time window has exceeded the retention period of the buffer storage, the excess data is read from the continuous packets, merged with the still readable data in the buffer storage, and then uniformly subjected to attribution filtering. In another implementation, the presumed closed state archived data packets are marked as read-only. During the regeneration of archived data packets after a real closure event, this read-only presumed closed state archived data packet can be loaded in read-only mode by the playback tool. When loading, the playback tool displays a presumed closed state prompt on the playback interface. Through the above-mentioned presumed closure mechanism and the reswitching mechanism after a real closure event, even if the closure event is not delivered for a long time due to network failure, the system can still automatically close the work order and generate a presumed closed state archived data packet after the timeout, avoiding long-term resource occupation and data loss due to buffer storage eviction. At the same time, the reswitching operation after a real closure event accurately corrects the presumed closed state data to the real closed state data, and the overwrite process during the correction process ensures that there is always only one final version of the archived data packet in the persistent storage.

[0083] In some embodiments, the method further includes: in response to the sum of the duration of the remote assistance work order and the first preset duration and the second preset duration exceeding the retention period of the buffer storage, performing segmented extraction of the remote assistance work order; and writing the remote assistance process data obtained from each segmented extraction into the same playback data packet.

[0084] When remote assistance work orders are long-lasting, such as when a vehicle is waiting for rescue for an extended period or when there are multiple interactions between the operator and the vehicle, the work order duration can reach tens of minutes or even hours. After extending the start time of the work order forward by a first preset duration and the end time backward by a second preset duration, the total span of the extraction time window equals the first preset duration plus the second preset duration. When this total span exceeds the buffer storage retention period, if all data within the entire extraction time window is read from the buffer storage at once after the work order is closed, data near the left boundary of the extraction time window will have been removed by the buffer storage's eviction policy while waiting for the end time of the extraction time window, resulting in incomplete data. To address the problem of the left boundary data becoming expired and unreadable due to the total span of the extraction time window exceeding the buffer storage retention period for extremely long work orders, a method is provided to extract extremely long work orders in segments and write each segment into the same playback data packet. This method continuously monitors the duration of a work order during its existence. This duration equals the current system time minus the case start time. When the duration plus a first preset duration exceeds a preset safety threshold for buffer storage retention, the acquisition link triggers a segmented extraction operation. In the segmented extraction operation, the acquisition link uses the current system time as the right endpoint and the earliest currently readable time in the buffer storage as the left endpoint. It reads all records from the left endpoint to the right endpoint from the buffer storage, temporarily stores the read records in the local temporary file corresponding to the work order, and records the start and end timestamps of the segmented extraction. After completing one segmented extraction, the acquisition link continues to monitor the work order's duration. When the work order's duration continues to increase and the cumulative amount of segmented extracted data reaches a preset segmentation interval, the segmented extraction operation is executed again, appending the newly extracted records to the end of the same local temporary file. When the work order closure event arrives and the extraction time window ends, the acquisition link merges the existing records in the local temporary file with the records read from the buffer storage after closure, according to the reception time, performs unified attribution filtering, and writes them into the same playback data packet file. In one implementation, the trigger condition for segmented extraction is that the sum of the work order duration and the first preset duration exceeds half of the buffer storage retention period. Under this condition, performing segmented extraction in advance ensures that the left boundary data has been read out and temporarily stored in the local temporary file before the retention period expires. In another implementation, the segmented extraction operation does not perform attribution filtering; that is, during segmented extraction, data is only read and temporarily stored according to the reception time, without judging whether the routing identifier of the record belongs to the current work order. Attribution filtering is uniformly performed in the merging stage after the work order is closed to ensure the consistency of the attribution filtering logic. In yet another implementation, when the work order closure event arrives, if the amount of data already temporarily stored in the local temporary file is large, the acquisition link reads data into memory in batches of fixed-size blocks during the merging stage to perform attribution filtering, in order to avoid memory overflow caused by loading all data at once.By extracting the data in segments from the aforementioned long work orders and merging them into the same playback data packet after the work order is closed, the data at the left boundary of the extraction time window is read out in segments and temporarily stored in a local temporary file before the retention period expires. This avoids the data at the left boundary expiring and being discarded due to waiting for the end of the extraction time window. At the same time, the segmented extraction does not perform the attribution filtering but only performs it uniformly during the merging stage, ensuring the consistency of the filtering rules.

[0085] In some embodiments, the method further includes: rolling the data in the buffer storage into continuous packets; in response to the absence of a remote assistance work order closing event, retrieving remote assistance process data from the continuous packets or the buffer storage according to the extraction time window, and generating a playback data packet.

[0086] During the normal operation of the buffer storage, its retention period is configured to be a limited duration to balance storage resource consumption and the time range for data readback. However, in actual operation, the closing events of some work orders may never arrive due to scheduling system anomalies, or the duration of the work orders may far exceed expectations, making it impossible for segmented retrieval to fully cover the entire data range. In order to still be able to retrieve obsolete data from the storage system under the above extreme conditions, a longer-term fallback storage layer needs to be established outside the buffer storage. To solve the problem of unrecoverable data after obsolescence in the buffer storage, a method is provided to roll over the data in the buffer storage into continuous packets and extract data from the continuous packets or the buffer storage according to the retrieval time window when the closing event is missing. In this method, the acquisition link starts a rolling dump thread while the buffer storage is running. This rolling dump thread reads the data accumulated before the current time from the buffer storage at a fixed dump interval, packages this data into continuous packet files in the order of receipt time, and the retention period of the continuous packet files is longer than the retention period of the buffer storage. The continuous packet files are written to a continuous packet storage area in persistent storage. When the size of a continuous packet file reaches a preset upper limit or the time span of the data contained in the continuous packet file reaches a preset upper limit, the rolling dump thread closes the current continuous packet file and begins generating the next continuous packet file. When the acquisition link detects that the closing event of a work order is missing and the estimated closing time of the work order has been determined, the acquisition link first applies a preset duration before and after the estimated closing time as the right endpoint to form an estimated extraction time window. Then, it checks whether the left boundary of the estimated extraction time window is still within the retention period of the buffer storage. If it is within the retention period, all data is read from the buffer storage. If the left boundary has exceeded the retention period of the buffer storage, the continuous packet file covering the period from the left boundary of the estimated extraction time window to the start readable time of the buffer storage is searched in the continuous packet storage area. This part of the data is read from the continuous packet file. The data between the start readable time of the buffer storage and the right boundary of the estimated extraction time window is read from the buffer storage. The data read from the continuous packet and the data read from the buffer storage are merged according to the receiving time and then uniformly subjected to attribution filtering to generate a playback data packet. In one implementation, when generating consecutive packet files, the rolling dump thread records the reception times of the first and last data entries in each consecutive packet file as a time range index. When the acquisition link searches for data in the consecutive packet storage area, it uses this time range index to quickly locate the consecutive packet file to be read. In another implementation, when the space occupied by the consecutive packet storage area reaches a preset limit, the rolling dump thread deletes the earliest consecutive packet file in chronological order of their generation to free up storage space. This deletion operation is not notified to the acquisition link. If the acquisition link finds that the target consecutive packet file has been deleted when reading consecutive packets, it records a consecutive packet missing marker in the archived metadata.By employing the aforementioned rolling dump mechanism and the method of extracting data from continuous packets or buffer storage according to the extraction time window when a shutdown event is missing, data that has been phased out in the buffer storage is retained in the continuous packets for a certain period of time. When a shutdown event is missing, the acquisition link can combine data from both the continuous packets and the buffer storage to cover the entire range of the extraction time window, significantly reducing the risk of permanent data loss due to buffer storage phase-out.

[0087] In some embodiments, the above-mentioned extraction of data received within the service period of each remote operator's station further includes: for a closed service period, relaxing the start time of the service period forward by a preset delay amount and relaxing the end time of the service period backward by a preset delay amount; for an unclosed service period, relaxing the start time of the service period forward by a preset delay amount; and extracting data received within the relaxed service period.

[0088] When retrieving records from the buffer storage where the routing identifier equals the remote operator's identifier, the retrieval condition is that the record's reception time falls within the service period corresponding to that remote operator. However, in actual systems, there may be a slight deviation between the occurrence time of the allocation event and the actual time the operator begins sending control commands. This deviation arises from the fact that the operator needs several milliseconds to tens of milliseconds to adjust their operating posture and activate the input device after the allocation event arrives. If the boundary of the service period is strictly limited to the occurrence time of the allocation event, control commands issued by the operator within a very short time after the allocation event will be excluded because their reception time is slightly later than the start time of the service period. Similarly, a similar deviation may exist between the occurrence time of the shutdown event and the time when the operator sends their last control command. Strict boundaries can cause the last few control commands at the end of the work order to be excluded because their reception time is slightly later than the end time of the service period. To address the problem of control commands near the service period boundary being excluded due to slight timing deviations caused by strict service period boundaries, a relaxed extraction method with an added delay margin to the service period boundary is provided. In this method, for a closed service period, which includes a start time and an end time, when the acquisition link determines whether the reception time of a record falls within the service period, it relaxes the start time of the service period by a preset delay amount, that is, it moves the start comparison boundary of the service period forward from the start time to the position of the start time minus the preset delay amount, and relaxes the end time of the service period by the preset delay amount, that is, it moves the end comparison boundary of the service period backward from the end time to the position of the end time plus the preset delay amount. Records whose reception time is between the relaxed start boundary and the relaxed end boundary are all determined to fall within the service period. For service periods that are not yet closed, these periods only include the start time and not the end time. When determining whether the reception time of a record falls within the unclosed service period, the acquisition link relaxes the preset delay by moving the start time of the service period forward. This means the starting comparison boundary of the service period is moved forward from the start time to the position where the start time is minus the preset delay. The end boundary of the service period is the current system time plus the preset delay. Records whose reception time falls between the relaxed start boundary and the current system time plus the delay are all determined to fall within the unclosed service period. In one implementation, the preset delay is less than half the minimum time interval between the allocation event and the departure event to ensure that the relaxed service period does not overlap with the service periods of the same seat in adjacent work orders on the timeline. In another implementation, for a closed service period, the preset delay amount for forward relaxation at the start time and the preset delay amount for backward relaxation at the end time are configured to have different values, wherein the amount of backward relaxation at the end time is greater than the amount of forward relaxation at the start time, in order to adapt to the actual working conditions where the closing operation time of the operator at the end of the work order is relatively long.

[0089] In some embodiments, the above-mentioned extraction of data from remote assistance process data within the service time period of each remote operator's seat further includes: using the service time period of the remote operator's seat as the extraction basis, and using the first preset duration and the second preset duration to form an extraction time window, which is not used for the service time period of the remote operator's seat.

[0090] The forward extension of the extraction time window by a first preset duration and the backward extension by a second preset duration are used to include message data that arrives earlier than the generation event or later than the closing event due to network transmission delays on both sides of the case start and end times. These first and second preset durations primarily apply to records where the routing identifier equals the work order identifier and the vehicle identifier, to preserve the context of the stuck cause and the confirmation feedback after closing. If these first and second preset durations are also applied to the seat service period, when the same seat serves other work orders within the work order's pre- or post-work order margin, the incorrectly extended service period will include the control instructions of adjacent work orders into the current work order, causing cross-work order data retrieval. The extraction time window is jointly defined by the forward extension of the case start time by a first preset duration and the backward extension of the case end time by a second preset duration. This extraction time window is used to read a candidate set of all records from the buffer storage, which contains all records whose reception time falls within the extraction time window. When extracting records whose route identifier equals the remote operator's identifier from the candidate set, the attribution criterion is whether the record's reception time falls within the remote operator's service period. This service period is determined by the occurrence times of the assignment and departure events and is unaffected by the first and second preset durations. Specifically, even if the reception time of a record whose route identifier equals the remote operator's identifier is within the pre- or post-retrieval margin interval of the extraction time window, the record is included as long as the reception time falls within the remote operator's service period. Conversely, even if the reception time is within the work order start and end interval of the extraction time window, but falls outside the remote operator's service period, the record is not included. In one implementation, when the same remote operator is serving a previous work order within the pre-retrieval margin of the extraction time window and the previous work order's closing time is before the pre-retrieval margin, the remote operator has not yet started the current work order's service period. Therefore, the control instructions from that operator within the pre-retrieval margin are not included in the current work order because their reception time is not within the current work order's service period. In another implementation, when the same remote operator serves the next work order within the margin of the extraction time window, and the allocation time of the next work order is within the margin, the service period of the remote operator for the current work order has ended. Therefore, the control commands of the operator within the margin are not included in the current work order because the receiving time is not within the service period of the current work order. By using the seat service period as the extraction basis and the constraints that the first and second preset durations are not used to extend the service period, the vehicle identification and work order identification records obtain the complete context of the trapped cause through the full-window expansion of the extraction time window, while the seat identification records are strictly limited to their respective service periods. This ensures the optimal extraction range for different types of data and fundamentally eliminates the problem of cross-work order data retrieval caused by mistakenly using the preset durations before and after the seat service period.

[0091] In some embodiments, the above-mentioned extraction of data from the buffer storage that has a routing identifier of a remote assistance work order, a vehicle identifier, and a remote operator identifier from the remote assistance process data further includes: if the routing identifier in the remote assistance process data is not empty and is not a remote assistance work order identifier, a vehicle identifier, or a remote operator identifier, parsing the work order identifier in the remote assistance process data payload; if the work order identifier in the payload is the work order identifier of the remote assistance work order, extracting the remote assistance process data.

[0092] In the above branch extraction rules based on routing identifiers, records where the routing identifier equals the work order identifier, vehicle identifier, and seat identifier are respectively overridden by their respective extraction rules. However, in actual systems, there are also some records whose routing identifier field in the envelope information is not empty, but this value is neither equal to the work order identifier of the current work order, nor the vehicle identifier of the current work order, nor any seat identifier in the seat service period of the current work order. Typical sources of such records include, but are not limited to: the message publisher filling in the correct work order identifier in the payload but mistakenly filling in the identifier of other entities in the routing identifier field; incompatibility between the old and new routing identifier naming rules during system component upgrades, resulting in inconsistencies between the routing identifier and the entity identifier in the payload; and the routing identifier being overridden to the default value by middleware during the forwarding of some messages. If these records are discarded directly, the work order identifier information correctly carried in the payload will be wasted, which may result in some critical messages not being included in the current work order. If all these records are included, a large amount of noisy data unrelated to the current work order will be mixed in. Therefore, during the attribution determination process, when the acquisition link detects a record whose routing identifier is not empty, and that the routing identifier is not equal to the current work order's work order identifier, not equal to the current work order's vehicle identifier, and not equal to any seat identifier in the current work order's seat service time sequence, the acquisition link does not directly discard the record. Instead, it enters the load parsing recovery branch. In this load parsing recovery branch, the acquisition link parses the load content according to the protocol format corresponding to the record's message type, extracts the work order identifier field carried by the message from the load content, and compares the extracted load work order identifier with the current work order's work order identifier. If they are equal, the record is included in the current work order's data set; if they are not equal or the load does not contain the work order identifier field, the record is discarded and not included.

[0093] In some embodiments, the above-mentioned extraction of the work order identifier, vehicle identifier, and remote operator identifier of the remote assistance process data with routing identifiers of remote assistance work orders from the buffer storage further includes: if the routing identifier in the remote assistance process data is empty and the load cannot be resolved, or the load does not contain the work order identifier, vehicle identifier, and remote operator identifier, extracting the remote assistance process data and generating an alarm record.

[0094] In remote assistance process data, some records contain empty routing identifier fields in their envelope information. Furthermore, the payload content of these records may be unparseable due to system anomalies, protocol changes, or data corruption, or the payload content may conform to the protocol format but lack any of the work order identifier, vehicle identifier, or remote operator identifier fields. These records may contain critical internal diagnostic information, anomaly stacks, or security audit logs, and their forensic value cannot be determined solely based on the routing identifier or the entity identifier explicitly marked in the payload. Discarding these records during the attribution determination phase could result in the loss of crucial evidence essential for diagnosing system anomalies and conducting security audits. Including all these records in the dataset of each work order would introduce a large amount of irrelevant internal system noise into the playback data packets of each work order. During the attribution determination process, when the acquisition link detects an empty routing identifier field in the envelope information of a record, the acquisition link enters the empty routing identifier processing branch. In this empty route identifier processing branch, the acquisition link first attempts to parse the load content of the record according to the preset protocol parsing rules. It then extracts the work order identifier field, vehicle identifier field, and remote operator identifier field sequentially from the load content, and determines the attribution according to the hierarchical matching rules of the aforementioned empty key branch. When load parsing fails (i.e., the load content does not conform to any known message protocol format), or when load parsing succeeds but the load content does not contain any of the work order identifier field, vehicle identifier field, or remote operator identifier field, the acquisition link defaults to including the record in the current work order's data set and generates an alarm record. This alarm record contains the record's original subject field, proxy reception time field, load length field, and the reason code for the unparsed record. This alarm record is written to an alarm log file in persistent storage, which is associated with the work order identifier.

[0095] In some embodiments, the above-mentioned extraction of the work order identifier, vehicle identifier, and remote operator identifier of the remote assistance process data with routing identifiers as remote assistance work orders from the buffer storage further includes: extracting the log data of the remote operator from the buffer storage in whole segments according to the extraction time window; and marking the service period of each remote operator for the remote assistance work order on the log data.

[0096] In terms of log data processing at the remote control console, log data differs fundamentally from the aforementioned message-type data in its structural characteristics. Log data is a continuous stream of text lines output by the console's operating system and applications. Each log line contains a timestamp, thread identifier, log level, and free text description. Log lines are linked by process and thread identifiers to form a context. A complete call stack or exception context is often distributed across dozens or even hundreds of consecutive log lines. If the log stream is precisely truncated according to the routing identifier or service period boundary when extracting log data—that is, only log lines between the start and end timestamps within a service period are extracted—the call stack and exception context crossing the service period boundary will be truncated into two segments. The first segment lacks entry parameter information, and the second segment lacks return value information, making it impossible to reconstruct the complete call chain of the exception during review. During the collection phase, the remote control console's log data is continuously written to a buffer in the form of log messages. Each log message contains a log line and its corresponding generation timestamp, and the log lines are arranged according to their generation timestamps. After the work order is closed and the extraction time window is determined, the acquisition link reads all log messages from the buffer storage across the entire span of the extraction time window. Specifically, it reads all log lines whose timestamps fall between the left and right boundaries of the extraction time window. No filtering is performed by routing identifier, and no trimming is done by service period boundaries. All read log lines are then sorted in ascending order by their timestamps to form a log fragment file for the work order. This log fragment file is then written to the log channel of the playback data packets. Simultaneously with generating this log fragment file, the acquisition link writes the work order's seat service period sequence as a metadata block into the header of the log fragment file. This metadata block contains the seat identifier, start time, and end time for all service periods of the work order.

[0097] In some embodiments, the method further includes: in response to a failure to transcode the driving real-scene video data in the archived data packet, keeping the archived data packet in a playable state; and in response to a successful transcoding, synthesizing a visual playback packet based on the archived data packet and the transcoded driving real-scene video data.

[0098] In the transcoding of live driving video data, the transcoding operation is the process of decoding the original encoded video file and then re-encoding it into the target format. This process involves a large amount of computing resources and disk I / O operations, and the transcoding time can range from several seconds to tens of minutes. During the transcoding process, various abnormal situations may occur, such as insufficient transcoding server resources leading to task queuing timeouts, corruption of the original video file leading to decoding failures, and insufficient target storage space leading to write failures. In existing technologies, the generation of archived data packets usually takes successful transcoding as a prerequisite for the final archiving completion. That is, message data, log data, and transcoded video data are only merged into the final archive package after successful transcoding. When transcoding fails, the entire archiving operation is rolled back or marked as a failure. At this time, even if the message data and log data have been successfully extracted and persisted, these data are still inaccessible to playback tools because the overall state of the archive package is a failure. After the work order is closed and the attribution filtering and extraction are completed, the acquisition link immediately aggregates and writes the extracted remote assistance process data, driving scenario analysis data, and original encoded real-time driving video data to persistent storage, forming an archived data package. This archived data package is marked as playable upon completion of the write process. The playable status indicates that the message data and driving scenario analysis data in the archived data package are complete and can be loaded by the playback tool. Even if the real-time driving video data has not yet been transcoded or has failed to transcode, the playback tool can still load the archived data package and display the message data and driving scenario analysis data; only the video portion will be displayed as a placeholder indicating transcoding in progress or transcoding failure. After the archived data package is written to persistent storage, the acquisition link or a separate transcoding service starts an asynchronous transcoding task. This asynchronous transcoding task performs transcoding operations on the original encoded video files referenced in the archived data package. When the transcoding task completes successfully, the transcoding service writes the transcoded preview video file to persistent storage. It then reads the archived data package and the transcoded preview video file from persistent storage, merges them into a single visualization playback package, and writes it to persistent storage. This visualization playback package contains message data, log data, driving scenario analysis data, and the transcoded preview video data. When the transcoding task fails, the transcoding service does not modify the archived data package in persistent storage; the archived data package remains in a playable state. The acquisition chain records the transcoding failure status and failure reason code in the archived metadata, allowing operators to manually re-trigger the transcoding task. The re-triggered transcoding task still reads the original encoded video file from persistent storage, without relying on data in buffer storage.

[0099] In some embodiments, the above method further includes: recording the status information generated by the acquisition link inside the acquisition link, the status information including at least one of acquisition success, storage capacity status, and transmission status; and controlling the link to transmit data along the original path.

[0100] In the entire data acquisition and archiving process described above, the acquisition link, operating as a bypass system independent of the control link, has its own operational status, including information such as whether acquisition was successful, whether storage capacity is sufficient, and whether network transmission is normal. This status information reflects the health of the acquisition link and the risk to data integrity. In existing technologies, the status information of the acquisition link is often recorded in a log system shared with the control link, or transmitted back to the monitoring platform through the same network channel. When the acquisition link experiences a storage capacity shortage or network transmission interruption, the transmission of status information may consume the bandwidth resources of the control link, or abnormal logs in the status information may be misjudged by the log acquisition process of the control link as abnormalities in the control link itself, triggering unnecessary alarms and interventions. The acquisition link continuously generates status information during operation. This status information includes at least one of the following: acquisition success status, storage capacity status, and transmission status. Acquisition success status indicates whether the acquisition link's operation of replicating messages from the message broker was executed correctly and whether the most recent replication operation returned a success code. Storage capacity status indicates the current used and remaining capacity of buffer storage and the current available space of persistent storage. Transmission status indicates whether the network connection between the acquisition link and the message broker is active and the network latency and packet loss rate of the most recent data transmission. The acquisition link writes this status information to a local status log file within the acquisition link itself. This local status log file is stored on the local disk of the server where the acquisition link resides, and this local disk is not mounted under any storage path of the control link. The acquisition link does not send this status information to the message broker or monitoring port of the control link through any network channel. The control link transmits vehicle uplink, cloud downlink, and remote operator control data according to the original path. The status information of the acquisition link does not participate in any data transmission actions of the control link.

[0101] The following is a specific scenario example provided in this manual, for reference. Figures 3-6 As shown: A remote assistance work order processing method is applied to autonomous driving remote assistance scenarios and is implemented when the following business premises are met: the same work order corresponds to only one vehicle at a time; the same work order can be sequentially assigned to multiple remote operators within its lifecycle. It is important to emphasize that the isolation between the vehicle control main link and the data acquisition side is not an external assumption of this method, but rather a step characteristic required by the method itself: all data acquisition actions at the continuous layer and work order layer are completed on a bypass outside the vehicle control main link, without interfering with the control command path, and any bypass anomaly must not send a blocking signal back to the vehicle control channel.

[0102] The remote assistance work order processing method is organized into three operational phases: Phase A, data collection process runtime (across work orders): Steps S1 and S2 are executed immediately after the data collection process starts, and do not depend on the existence of a particular work order.

[0103] Phase B, single work order lifecycle: Steps S3 and S4 are only within the interval from the generation of the work order to its closure, driven by lifecycle events and playback control events.

[0104] In phase C, after the work order is closed: steps S5 to S8 are executed in a fixed order. Whether the preview transcoding has started or completed must not be used as a prerequisite for message cutting—the transcoding job itself is allowed to run in parallel with step S7, but the synthesis of step S8 must be scheduled after step S7; nor should the retrieval be interrupted before the right end of the candidate time window has been reached.

[0105] Step S1: Use the work order lifecycle as the unique case boundary.

[0106] After the acquisition side starts the process, it subscribes to the work order lifecycle events broadcast by the scheduling system, identifying at least: work order generation, allocation to a work seat, leaving the work seat or returning to the queue, and work order closure. The work order identifier serves as the primary key for data organization; the vehicle identifier, work seat identifier, and time are used as attributes on the work order, rather than as the primary key alongside the work order itself.

[0107] The generation event is used to create a work order file, recording the work order identifier, vehicle identifier, and case start time. The closing event is used to record the case end time and trigger step S5 and subsequent case closure processing. The data collection side does not automatically cut off the process based on whether the vehicle is stuck, whether a timeout has occurred, or whether a stagnation signal has been received. If the generation event has not yet arrived, a work order file must not be created using other business signals to avoid "ghost cases" without a work order identifier.

[0108] Lifecycle events may be late or out of order, and must be handled according to the following rules, without introducing a second set of segmented triggers: If an allocation or departure event carrying the same work order identifier arrives before a generation event, the event is temporarily stored and added to the seat service interval sequence according to its own time after the generation event arrives; the generation event is always the sole basis for filing. If the generated event does not ultimately arrive, the temporary event will be discarded after a configurable time limit, and no work order file will be generated. If a work order file has been created but the closure event has not arrived for an extended period, after the configurable time limit has expired since the last lifecycle event, a presumed closure is allowed, using the last event's time as the case's end time. Based on this, the closed interval is activated, the bypass session for this work order is stopped, and a durable work order package is generated according to steps S5 to S7. Simultaneously, it is marked as presumed closed in the work order list. When the actual closure event arrives, the end time is corrected according to its time, and steps S5 to S7 are re-executed. Presumed closure is only used to release long-pending sessions and avoid archiving gaps. The case boundary is still taken from the lifecycle event itself, and no segmentation logic parallel to scheduling is introduced. Subscriptions themselves are cross-work order.

[0109] Each work order's file is created only when its generation event arrives, and it enters stage C only when the closing event arrives (or is presumed to be closed according to the above rules). Unclosed work orders remain in stage B.

[0110] Step S2: The continuous layer is always open and isolated from the vehicle control system. No work order trimming is performed at the entrance.

[0111] The persistence layer is independent of work order opening and closing; it runs from the start of the data collection process until the process stops. Both actions of the persistence layer are completed on a bypass outside the main vehicle control link, performing only message copying and log uploading. It does not insert nodes into the control command path, occupy control channels, or alter message delivery behavior on the main link. On the message broker side, messages such as vehicle uplink, cloud downlink, seat control, and work order events are forwarded to a time-series buffer layer that can be read back by time. Deploy a persistent log collector at the remote workstation to continuously upload runtime logs, instead of only starting to log after a work order is opened.

[0112] When forwarding messages, each record carries envelope information, including at least the original subject, the time the agent received the message, and a routing key for differentiation. The routing key can be one of the work order identifier, vehicle identifier, or seat identifier, or it can be empty. The routing key is preferentially taken from the explicit key in the envelope; if it is empty, it is allowed to be completed according to the subject naming convention; if it is still empty after completion, it is treated as an empty key. The original payload is passed through during the forwarding process, and business fields are not parsed at the entry point, nor is it filtered according to "whether there are currently open work orders". This entry point is a non-recoverable link: once it is pruned by work order here, the discarded unreliable delivered messages cannot be recovered from the source. All attribution determinations are postponed to step S6.

[0113] When bypass acquisition fails, storage is full, strip upload is interrupted, or transcoding is abnormal, a blocking signal must not be sent back to the vehicle control channel. The buffer layer is a short-cycle storage shared by multiple vehicles, multiple operators, and multiple work orders. The retention period is configurable and limited, and it is used to provide the vehicle status context before work order generation.

[0114] There is a necessary constraint between the buffer layer retention period and the timing of retrieval after order closure: Since step S5 requires retrieval only after the right end of the candidate time window has been reached, the left end of the window has already passed by the time retrieval occurs, which includes "pre-emptive margin + order duration + post-emptive margin + processing delay". Therefore, the system must ensure that the total duration does not exceed the buffer layer retention period. Specific measures include at least: limiting the sum of the pre-emptive and post-emptive margins to within the retention period and reserving processing margin; for orders with abnormally long durations, performing segmented retrieval and writing segments into the same durable order package when the retention period is about to fall below the left end of the window. If the above constraints are still breached, the next segment's continuous package path should be used as a fallback, and it should not be assumed that data within the window can still be read back.

[0115] Optionally, the buffer layer can be rolled over and dumped into short-cycle continuous packages as a fallback when there are no closed orders, missed triggers for closure, or when the left end of the candidate window has exceeded the retention period; the retention period of the continuous package is shorter than that of the work order package and does not replace the durable work order package in step S7.

[0116] Step S3: After the work order is generated, create a file and start the perception and visualization bypass.

[0117] Upon receiving the generation event, the data acquisition side creates the work order file and initiates the perception visualization bypass in the work order layer: the autonomous driving stack visualization data is written to long-term storage in stripes via an independent bypass, without entering the message buffer layer, to avoid large blocks of binary data crowding out the buffer reservation quota. This bypass is initiated by work order generation, because at this time the work order identifier and vehicle identifier have already been aggregated by the scheduler.

[0118] The difference between the work order layer and the continuous layer is that the continuous layer does not start when a work order is opened, nor does it stop when a work order is closed; the work order layer only runs within the lifespan of the work order. Video bypass also belongs to the work order layer, but its start signal is not a generation event, but rather the playback control in step S4.

[0119] Step S4: Maintain the service area of ​​the work order within its lifespan and start / stop video bypass according to the playback control.

[0120] In addition to recording the final seat for a work order, the system maintains a sequence of seat service intervals based on lifecycle events. Each interval includes at least a seat identifier, an opening time, and a closing time. The closing time of an unclosed interval is considered positive infinity, indicating that the seat is still serving the current order.

[0121] Update rules must be executed mutually exclusively according to event type: When an event is received that is assigned to a seat: all unclosed intervals are closed at the time of the event, and then a new interval is opened starting from the time of the event and using the seat identifier. Closing before opening is to accommodate reassignment when a departure event is missed, and to avoid two seat segments being open at the same time. If the assigned event lacks a seat identifier, only the old interval is closed and the new interval is not opened; in this case, there is no comparable service interval for high-frequency data with the seat as the routing key within this time period, and step S6 step 3 will fail. It can only be remedied by step 5 using the work order identifier in the load, and the rest will not be included. This trade-off is intentional: it is better to include less seat data whose ownership cannot be confirmed than to widen the interval boundaries to cause cross-order mutual retrieval. When an event such as leaving a seat, returning to the queue, or closing is received: all unclosed intervals will be closed at the time of the event, and no new intervals will be opened.

[0122] The start and end times of the interval are taken from the time of the lifecycle event itself, without finding an additional clock source. Observing more granular process states such as successful connection and successful remote control connection only determines which themes will have high-frequency data within the interval, and does not serve as filtering boundaries. The filtering boundaries only recognize "work order to work station / work order to work station".

[0123] Video bypass and perception bypass both belong to the work order layer, but their triggers differ. When the scheduling side issues a start playback control, the collector uses the provided pull stream address to start the corresponding "work order—seat—channel" bypass recording session; when a stop playback control is issued, the session ends. Real-time recording primarily uses encapsulation copying, preserving the original encoding, and does not consider real-time transcoding as a necessary step for bypass recording. When the same work order is assigned to different seats multiple times, each session is an independent segment and does not overwrite others. The close event serves as a fallback for stopping the stream, see step S5. Transcoding of the preview encoding is performed asynchronously after the work order is closed, see step S8. The operator views the real-time stream, which does not pass through this archiving link; therefore, a delay of minutes is allowed between archiving completion and playback availability.

[0124] Step S5: Disable the event processing order.

[0125] Once a shutdown event arrives, it must be processed in the following order: 1. Follow step S4 to close all unclosed seat service intervals using the closing time, and write the case end time; 2. Stop all streaming media bypass recording sessions and perception visualization bypass sessions for this work order; the generated original video segments and visualization strips should now be in or subsequently uploaded to long-term storage, without relying on the message buffer layer; 3. A pre-emptive margin is reserved before the case's start time, and a post-emptive margin is reserved after the case's end time, forming a candidate time window for retrieval. The pre-emptive margin is used to include trapped cause-related data still in the buffer layer before the work order is generated; the post-emptive margin is used to include messages still in transit at the moment of closure. Both margins are configurable. 4. After the right end of the candidate time window arrives, proceed to step S6 to read from the buffer layer. If reading is performed immediately after closing, the in-transit messages corresponding to the back margin will not have fallen into the buffer layer and will be truncated. During the waiting period, the left end of the window must be checked against the constraints of step S2 to see if it is still within the buffer layer retention period; if it has exceeded the period, the excess part will be read from the continuous packet, and the remaining part will still be read from the buffer layer. The two parts will be merged and written into the same durable work order packet according to the same attribution rule.

[0126] The candidate time window only determines "which time records are eligible for judgment," and does not replace the attribution determination. Data from the same vehicle, other vehicles, the same seat, and other seats may exist within the window simultaneously. Step S6 must be processed again.

[0127] If the closure event is missed, the work order layer package will not be automatically generated. In this case, there are two fallback paths that are not mutually exclusive: First, assume closure after timeout according to step S1, form a candidate time window, and execute this step, as well as steps S6 and S7. The result is marked as a presumed closure state, and the package is re-switched according to the actual closure event. Second, supplement data with short-cycle continuous packages from the persistence layer—while the buffer layer is still readable, once a closure event is detected, the durability package is still switched according to this step; if the buffer layer has expired, it is allowed to supplement from the continuous package according to the same time window and the same attribution rule. The supplementation results of both paths are written to the durability work order package; "there is still data in the buffer layer" should not be used as a criterion for whether archiving is complete.

[0128] Step S6: Filter by button type after closing the order.

[0129] The system retrieves records whose reception time falls within the candidate time window, based on the source determined in step S5, S4—normally a timing buffer layer; records exceeding the retention period at the left end of the window are retrieved from consecutive packets—and then determines whether to include each record in the current order. The determination is not based on running the following rules for each record individually, but rather on branching according to the record's routing key type: the same record falls into only one branch, and these branches are mutually exclusive; only the layer of "if a non-null key is not found in this branch, then remedy it according to the work order identifier in the payload" constitutes a priority relationship. Except for the default inclusion when a null key cannot be determined, a record is not included if it is not found; "already falling within the candidate window" cannot be used as a reason for inclusion.

[0130] Before proceeding to the judgment stage, internal records related to the preview transcoding progress are removed to prevent the transcoding job's own process data from contaminating the work order package. This removal is an engineering cleanup and is not part of the attribution judgment itself.

[0131] 1. Lifecycle Records: Match the current work order to the work order identifier in the load; if the load cannot parse or the work order identifier is missing, include it by default and issue an alarm to avoid missing status transition points for this work order. Lifecycle events of multiple work orders within the same time period may share the same channel, therefore they cannot be collected in batches by time window.

[0132] 2. If the routing key is equal to the work order identifier or the vehicle identifier for this order: include. The reason for using the vehicle key across the entire window is: for a work order involving a vehicle, low-frequency heartbeats, malfunctions, and stuck reports for that vehicle within the entire window are considered to be the causes of stuck situations for this order; high-frequency operations on the seat side usually have not yet occurred before the work order is generated.

[0133] 3. The routing key is equal to the identifier of a seat in the service interval sequence of this order: it is included only if the reception time of the record falls within the service interval corresponding to that seat. Here, the comparison is made on "the key is equal to the identifier of the seat already registered in this order", not "the key can be formally resolved into a seat number". Keys that can be formally resolved but do not belong to any seat in this order are included in step 5. A limited delay margin is allowed during the comparison: the start point of the closed interval is widened forward and the end point is widened backward; the unclosed interval has no upper bound, and is only widened forward for the start point. This margin is only used to compare whether the seat key falls within the interval and does not change the start and end records of the interval itself. The seat key cannot be superimposed with the pre-margin of step S5, nor can the post-margin be used as a seat service extension: the same seat may serve two work orders in a short period of time, and forward and backward extensions will bring in the control instructions of the previous or next order. Therefore, the pre-margin and post-margin mainly take effect on the work order key and the vehicle key; the seat key must still fall within the service interval (including the delay margin) in the same candidate window. There is no need to further subdivide the interval into sub-stages such as observation and remote control; the sub-stages only affect which themes appear within the interval.

[0134] 4. If the routing key is empty: Cascade by load. If the work order identifier equals this order, it will be included; otherwise, if the vehicle identifier equals this order, it will be included; otherwise, if the seat identifier falls within the seat service range (including latency margin) of this order, it will be included; if none of the three can be determined or load parsing fails, it will be included by default and an alarm will be triggered. Default inclusion prioritizes ensuring complete evidence collection, and subsequent manual or rule-based contraction will be performed, rather than silently discarding it during archiving.

[0135] 5. Load Response Layer, only for non-empty keys: When the routing key is not empty but steps 2 and 3 fail, the load is parsed again. If the work order identifier in the load is equal to the current work order, it is included; otherwise, it is not included. This remedy is used for a small number of records where the routing key and load are inconsistent. It is not an empty key rule, nor is it a "let all records through" rule.

[0136] The seat logs do not follow the routing key rules mentioned above. Instead, they extract segments of the current order from the uploaded logs in the persistent layer according to the same retrieval candidate time window and write them to the log channel in the work order directory or durability package. The reason for allowing full-window extraction of logs while adhering to service intervals for high-frequency seat messages is that the granularity and risks are different: logs continuously scroll by seat process and are not fragmented by work order or routing key. Forcibly truncating them by interval would fragment the call stack and exception context, destroying the evidence value; and their data volume and frequency are much lower than control commands. Even if a small number of segments from adjacent work orders are included, the review personnel can interpret them by time, without causing seat mismatch in statistics and training backflow like high-frequency controls. If the same seat serves multiple work orders successively within the candidate window, the corresponding seat service interval can be marked on the log segment for subsequent shrinking as needed. The perception visualization strip is categorized into the current order directory according to the work order identifier and is not retrieved through the message buffer layer.

[0137] Step S7: Immediately generate a durability work order package.

[0138] After step S6 is completed, the system immediately writes the messages included in this order (and the seat log fragments extracted according to the time window) into the durability work order package and marks it as durable. The written object is long-term storage, rather than relying on the buffer layer again. The only prerequisites for this step are "the close event has been processed or has been presumed to be closed according to step S1, the right end of the candidate window has been reached and the attribution filtering has been completed", not the completion of preview transcoding, whether the sensing strips are complete, or whether all original videos have been uploaded. When media is missing, the durability package is still written to disk first, and the missing items are marked in the list to be supplemented.

[0139] The meaning of "durable state" is: this message has exceeded the retention period constraint of the buffer layer. Afterwards, the buffer layer will discard data for that period according to its strategy, without affecting the long-term availability of this message. The work order detailed package and the continuous package in the persistent layer are allowed to have two copies within the work order's time frame, respectively handling long-term detailed investigation and short-term backup. Deletion of the continuous package upon expiration does not negate the detailed package.

[0140] Step S8: After the preview transcoding is completed, derive the visual playback package.

[0141] Preview transcoding is tracked as a separate job, and can be performed in parallel or delayed with step S7. The goal of transcoding is to generate a unified preview track for playback tools and web playback; the original video is retained independently as training frame extraction and forensic assets, and is not forced to be embedded in the playback package.

[0142] Once all preview transcoding operations related to the work order are successful, the system reads the durable work order package, preview video, and perceptual visualization strip already stored in long-term storage, synthesizes a visualization playback package, and marks it as archived. The timing buffer layer is no longer read back during the synthesis stage. The original video and playback package are bound together through the work order list, eliminating the need for the playback tool to align with external original video files.

[0143] Similar to step S7, this step also does not adopt an "all or nothing" approach for missing media: if the perceived visualization strip is missing or the original video has not been fully uploaded, but the preview transcoding has been successful, the visualization playback package for the currently available media is still synthesized first, and the missing items are marked in the list. The package is then re-synthesized and overwritten once the missing media is complete. This step is only skipped if the preview transcoding itself fails, maintaining the durability state. The criterion is that "the visualization playback package is always a derivative of the durability package and can be redone," therefore, synthesis should not be delayed to ensure all media are available.

[0144] If transcoding times out or fails for an extended period, the system remains in durable state, allowing for preview-less review using durable packages, the original video, and sensor stripes. This step can also be performed again after successful transcoding. The composite after successful transcoding still only reads from long-term storage, without needing to re-pull the buffer layer. This design allows preview transcoding to take longer than the buffer layer retention period without resulting in lost work order messages.

[0145] During archiving, both the business time within the message and the reception time in the envelope are retained. The business time is used for subsequent playback alignment, while the reception time is used for latency auditing and seat interval comparison in step S6.

[0146] See Figure 7 As shown in the embodiments of this specification, a remote assistance work order processing device 700 is also provided, which may specifically include the following structural modules: The acquisition module 701 is used to acquire remote assistance process data to buffer storage based on the acquisition link outside the vehicle's control link, and to acquire driving scene data to persistent storage during the remote assistance work order; the remote assistance process data includes vehicle operation and control data and remote operator log data; the driving scene data includes vehicle driving scenario analysis data and driving real-time video data. The generation module 702 is used to extract the work order identifier, vehicle identifier and remote operator identifier data with the routing identifier of remote assistance work order from the buffer storage in response to the closure of the remote assistance work order, and aggregate it with the driving scene data associated with the work order identifier in the persistent storage to generate a playback data packet of the remote assistance work order in the persistent storage.

[0147] As can be seen from the technical solutions provided in the embodiments of this specification above, the embodiments of this specification collect data outside the vehicle control link, decoupling the data collection process from vehicle control and not affecting normal vehicle control; writing process data to buffer storage and driving scene data to persistent storage, realizing hierarchical storage of data with different characteristics, avoiding large-capacity driving scene data crowding out the buffer; after the work order is closed, process data is extracted according to the work order identifier, vehicle identifier, and operator's seat identifier, and aggregated with the driving scene data to generate a playback data package, realizing complete archiving by work order unit, which is convenient for post-event review and traceability.

[0148] It should be noted that the units, devices, or modules described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. For ease of description, the above devices are described by dividing them into various modules according to their functions. Of course, in implementing this specification, the functions of each module can be implemented in one or more software and / or hardware, or a module that implements the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are illustrative. For example, the division of units is a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0149] Based on the above-described remote assistance work order processing method, this specification also provides a computer device in its embodiments. (See attached document) Figure 8 As shown, the computer device 800 includes a network communication port 801, a processor 802, and a memory 803. These structures are connected by internal cables so that each structure can perform specific data interaction.

[0150] Specifically, the processor 802 can be used to collect remote assistance process data to buffer storage based on the acquisition link outside the vehicle's control link, and to collect driving scene data to persistent storage during the remote assistance work order; the remote assistance process data includes vehicle operation and control data and remote operator log data; the driving scene data includes vehicle driving scene analysis data and driving real-time video data; in response to the closure of the remote assistance work order, it extracts the work order identifier, vehicle identifier, and remote operator identifier data with routing identifiers of remote assistance work orders from the buffer storage, and aggregates them with the driving scene data associated with the work order identifier in the persistent storage to generate a playback data packet of the remote assistance work order in the persistent storage.

[0151] The memory 803 can be used to store the corresponding instruction program, as well as related data such as target data.

[0152] Based on the above method, the relevant structural performance of computer equipment can be effectively utilized to improve the data processing speed of electronic equipment and efficiently realize data processing for remote assistance work order processing.

[0153] In this embodiment, the network communication port 801 can be a virtual port bound to different communication protocols, thereby enabling the sending or receiving of different data. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. Furthermore, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM or CDMA; it can also be a Wi-Fi chip; or it can be a Bluetooth chip.

[0154] In this embodiment, the processor 802 can be implemented in any suitable manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. This specification is not limiting.

[0155] In this embodiment, the memory 803 may include multiple layers. In a digital system, anything that can store binary data can be a memory. In an integrated circuit, a circuit with storage function but no physical form is also called a memory, such as RAM, FIFO, etc. In a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0156] This specification also provides a computer-readable storage medium based on any of the above-described remote assistance work order processing methods. The computer-readable storage medium stores computer program instructions that, when executed, enable the following: outside the vehicle's control link, based on a data acquisition link, remote assistance process data is collected to a buffer storage, and driving scene data is collected to persistent storage during the remote assistance work order period. The remote assistance process data includes vehicle operation and control data and remote operator log data; the driving scene data includes vehicle driving scenario analysis data and driving real-time video data; in response to the closure of the remote assistance work order, data with routing identifiers indicating a remote assistance work order, such as the work order identifier, vehicle identifier, and remote operator identifier, are extracted from the buffer storage and aggregated with the driving scene data associated with the work order identifier in persistent storage to generate a playback data packet for the remote assistance work order in persistent storage.

[0157] In this embodiment, the storage medium includes, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Cache, Hard Disk Drive (HDD), or Memory Card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured according to standards specified in the communication protocol for network connection communication.

[0158] In this embodiment, the specific functions and effects implemented by the program instructions stored in the computer-readable storage medium can be explained in comparison with other embodiments, and will not be repeated here.

[0159] This specification also provides a computer program product, comprising at least a computer program that, when executed by a processor, can implement the following method steps: outside the vehicle's control link, based on a data acquisition link, acquiring remote assistance process data to buffer storage, and acquiring driving scene data to persistent storage during a remote assistance work order; the remote assistance process data includes vehicle operation and control data and remote operator log data; the driving scene data includes vehicle driving scenario analysis data and driving real-time video data; in response to the closure of a remote assistance work order, extracting data from the buffer storage that has a routing identifier of a remote assistance work order, a vehicle identifier, and a remote operator identifier, and aggregating them with the driving scene data associated with the work order identifier in persistent storage, to generate a playback data packet for the remote assistance work order in persistent storage.

[0160] While this specification provides the steps of operation for the methods described in the embodiments or flowcharts, more or fewer steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is one possible order of execution among many steps and does not represent the only possible order. In actual device or client product execution, the methods may be executed sequentially or in parallel as shown in the embodiments or drawings (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that includes a list of elements does not include those elements, but also includes other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. The terms "first," "second," etc., are used to denote names and do not indicate any particular order.

[0161] Those skilled in the art will also know that, besides implementing the controller using purely computer-readable program code, the same functions can be achieved by logically programming the method steps, making the controller function as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers (PLCs), and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the devices within it used to implement various functions can also be considered structures within that hardware component. Alternatively, the devices used to implement various functions can be considered as both software modules implementing the method and structures within a hardware component.

[0162] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer-readable storage media, including storage devices.

[0163] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this specification can be implemented by means of software plus necessary general-purpose hardware servers. Based on this understanding, the technical solutions of this specification can essentially be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of this specification.

[0164] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. This specification can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0165] Although this specification has been described by way of examples, those skilled in the art will recognize that many variations and modifications are possible without departing from the spirit of this specification, and it is intended that the appended claims cover such variations and modifications without departing from the spirit of this specification.

Claims

1. A method for processing remote assistance work orders, characterized in that, include: Outside the vehicle's control link, based on the acquisition link, remote assistance process data is collected to buffer storage, and driving scene data is collected to persistent storage during remote assistance work orders. The remote assistance process data includes vehicle operation and control data and remote operator log data; the driving scene data includes vehicle driving scenario analysis data and driving real-time video data. In response to the closure of a remote assistance work order, data with the route identifier of the remote assistance work order, the work order identifier, the vehicle identifier, and the remote operator identifier are extracted from the buffer storage and aggregated with the driving scene data associated with the work order identifier in persistent storage to generate a playback data packet for the remote assistance work order in persistent storage.

2. The remote assistance work order processing method according to claim 1, characterized in that, The acquisition link and the control link are independent of each other.

3. The remote assistance work order processing method according to claim 1, characterized in that, The operational control data includes vehicle uplink data, cloud downlink data, and remote operator control data. The process of collecting remote assistance data and storing it in a buffer includes: Copy the vehicle uplink data, cloud downlink data, and remote operator control data transmitted in the control link; For each copied data entry, append the original subject, reception time, and routing identifier, and write it to the buffer storage; the routing identifier can be a work order identifier, vehicle identifier, remote operator identifier, or blank. Write the log data from the remote operator's console to the buffer storage.

4. The remote assistance work order processing method according to claim 1, characterized in that, The process of collecting driving scene data and storing it in persistent storage during remote assistance work orders includes: In response to the generation of remote assistance work orders, collect driving scenario analysis data and write it to persistent storage; In response to a playback event, record live driving video data and write it to persistent storage.

5. The remote assistance work order processing method according to claim 1, characterized in that, The data extracted from the buffer storage that has the routing identifier of a remote assistance work order, the vehicle identifier, and the remote operator's seat identifier as the routing identifier of the remote assistance process data includes: The start and end times of each remote assistance work order are determined based on the generation and closing events in the lifecycle events of each remote assistance work order. Based on the start and end times, the extraction time window is obtained by forward expansion of a first preset duration and backward expansion of a second preset duration. In response to the end of the extraction time window, data including the work order identifier, vehicle identifier, and remote operator identifier of the remote assistance work order are extracted from the remote assistance process data received within the extraction time window.

6. The remote assistance work order processing method according to claim 1, characterized in that, The process of retrieving data from the buffer storage that has a routing identifier of a remote assistance work order, including the work order identifier, vehicle identifier, and remote operator identifier, also includes: If the routing identifier in the remote assistance process data is the work order identifier of the remote assistance work order or the vehicle identifier of the vehicle corresponding to the remote assistance work order, extract the remote assistance process data.

7. The remote assistance work order processing method according to claim 1, characterized in that, The process of retrieving data from the buffer storage that has a routing identifier of a remote assistance work order, including the work order identifier, vehicle identifier, and remote operator identifier, also includes: If the routing identifier in the remote assistance process data is the remote operator identifier, the service period of each remote operator for the remote assistance work order is determined based on the work order assignment event and work order departure event of each remote operator. Extract data from the remote assistance process data, focusing on the time of receipt within the service hours of each remote operator's workstation.

8. The remote assistance work order processing method according to claim 7, characterized in that, The process of determining the service period for remote assistance work orders for each remote operator based on work order assignment events and work order departure events includes: In response to receiving a work order assignment event from the first remote operator, the service time of one or more unclosed second remote operators is closed, with the time corresponding to the work order assignment event as the end time, and the service time of the first remote operator is started with the time corresponding to the work order assignment event. In response to receiving a work order departure event from the first remote operator, the time corresponding to the work order departure event is taken as the end time of the service period of the first remote operator.

9. The remote assistance work order processing method according to claim 1, characterized in that, The process of retrieving data from the buffer storage that has a routing identifier of a remote assistance work order, including the work order identifier, vehicle identifier, and remote operator identifier, also includes: If the routing identifier in the remote assistance process data is empty, parse the work order identifier, vehicle identifier, and remote operator identifier in the remote assistance process data payload. Based on the analysis results, extract data that matches the remote assistance work order from the remote assistance process data.

10. The remote assistance work order processing method according to claim 1, characterized in that, The process of collecting driving scene data and storing it in persistent storage during remote assistance work orders includes: The driving site data is associated with the work order identifier of the remote assistance work order and written to persistent storage.

11. The remote assistance work order processing method according to claim 1, characterized in that, The process of generating the playback data packet for the remote assistance work order in persistent storage includes: Generate archived data packets for remote assistance work orders and store them in persistent storage; The driving scene video data in the archived data package is transcoded to obtain the playback data package.

12. The remote assistance work order processing method according to claim 1, characterized in that, The method further includes: If the driving scene analysis data or driving real-time video data in the driving scene data is not collected, a playback data package is generated based on the collected driving scene data and the extracted remote assistance process data, and the data to be supplemented is recorded. In response to the completion of data collection for the items to be supplemented, a new playback data packet is generated.

13. A remote assistance work order processing device, characterized in that, include: The data acquisition module is used to collect remote assistance process data to buffer storage based on the data acquisition link outside the vehicle's control link, and to collect driving scene data to persistent storage during the remote assistance work order. The remote assistance process data includes vehicle operation and control data and remote operator log data; the driving scene data includes vehicle driving scenario analysis data and driving real-time video data. The generation module is used to extract the work order identifier, vehicle identifier, and remote operator identifier data with the routing identifier of remote assistance work order from the buffer storage in response to the closure of the remote assistance work order, and aggregate them with the driving scene data associated with the work order identifier in the persistent storage to generate the playback data packet of the remote assistance work order in the persistent storage.

14. A computer device, characterized in that, include: The system includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to implement the remote assistance work order processing method according to any one of claims 1 to 12.

15. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 12.

16. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 12.

Images