A task timing synchronization method and device for multi-device joint collection

Abstract

The application discloses a kind of task timing synchronization method and device for multi-device joint collection, its method includes: according to acquisition instruction, request subtask to cloud task system, obtain task session identification;Task confirmation instruction containing the identification is sent to main control collection device;Judge whether main control collection device returns successful confirmation information, if yes, then it sends start recording instruction, makes it start subordinate collection device, executes track and / or video recording after preset delay, and records the clock synchronization round trip delay between main control collection device;The trajectory and / or video file and metadata file generated are associated with task session identification, form task collection data group and upload to cloud end.This application realizes the task consistency and timing verifiable of multi-device joint collection by two-stage task confirmation, delay start recording, clock synchronization evidence recording and cloud end step-by-step reporting, reduces invalid sample and data loss.

Description

Technical Field

[0001] This invention relates to the field of robot teaching data acquisition technology, and in particular to a task timing synchronization method and apparatus for multi-device joint acquisition. Background Technology

[0002] In fields such as robot teaching data acquisition and human-robot collaborative control, multiple devices are often required to work together to collect data. This often involves a head-mounted extended reality acquisition terminal, a main control acquisition device, a wrist camera, grippers or dexterous hand peripherals, and a cloud-based task system working in concert to obtain multimodal training data. Key challenges in multi-device collaborative acquisition include: inconsistent task identifiers between devices leading to data mismatch; asynchronous device startup sequences causing misalignment between video and trajectory data; uncontrollable acquisition status under weak network conditions or peripheral malfunctions; and unobservable progress of cloud-based tasks.

[0003] In the existing technology, there are already some multi-device collaborative data acquisition solutions. For example, some patent documents disclose a system that achieves inter-device collaboration through trigger signal generation, merging, and communication delay measurement. However, these solutions only focus on communication delay compensation and do not involve data acquisition task consistency confirmation and state machine management, thus failing to solve the problem of multi-device task mismatch caused by network latency or repeated triggering.

[0004] Other solutions involve multi-camera synchronous marking, command sequence control, or command retrieval, but none of them offer a complete solution for robot teaching scenarios where head-mounted extended reality acquisition terminals and external acquisition devices (such as main control acquisition devices and wrist cameras) jointly acquire data. This solution covers everything from task consistency verification and recording timing synchronization to a closed-loop cloud status mechanism. Existing technologies typically implement task identification, clock synchronization, and status reporting as independent functions, lacking a holistic mechanism that combines these steps into a specific process to form a pre-acquisition gating, in-acquisition verification, and post-acquisition closed-loop mechanism. This leads to problems such as task mismatch, invalid sample generation, missing clock synchronization round-trip latency, and opaque cloud status in unattended acquisition scenarios with weak networks or unstable peripherals.

[0005] Therefore, there is an urgent need for a method that can achieve consistency confirmation and recording sequence synchronization of multi-device joint data acquisition tasks, so as to improve the task attribution and auditability of robot training data and reduce the generation of invalid samples. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a technology that can realize the consistency confirmation and recording timing synchronization of multi-device joint acquisition tasks.

[0007] The objective of this invention is achieved through the following technical solution: Therefore, according to one aspect of the present invention, a task timing synchronization method for multi-device joint data acquisition is provided, comprising the following steps: requesting a sub-task from a cloud-based task system according to an acquisition instruction and obtaining a corresponding task session identifier; sending a task confirmation instruction to the master acquisition device; wherein the task confirmation instruction includes the task session identifier; determining whether the master acquisition device returns a success confirmation message, and if so, sending a start recording instruction to the master acquisition device to enable the master acquisition device to start the subordinate acquisition device, and after a preset start delay, performing trajectory data acquisition and / or video recording, and recording the clock synchronization round-trip delay between the master acquisition device and the master acquisition device; associating the trajectory file and / or video file generated by the trajectory data acquisition and / or video recording, as well as the metadata file, with the task session identifier to form a task acquisition data group; and uploading the task acquisition data group to the cloud-based task system.

[0008] Specifically, determining whether the main control acquisition device returns a success confirmation message includes the following steps: if no success confirmation message is received within the preset timeout threshold, or if a failure confirmation message is received, the current request subtask is terminated; if a success confirmation message is received within the preset timeout threshold, the corresponding acquisition directory and acquisition file are created.

[0009] The above-mentioned clock synchronization round-trip delay between the main control acquisition device and the main control acquisition device includes the following steps: clearing the clock synchronization round-trip delay list; periodically sending clock synchronization requests to the main control acquisition device; calculating the round-trip delay based on the sending time of the clock synchronization request and the receiving time of the corresponding acknowledgment, and updating the clock synchronization round-trip delay list; and writing the clock synchronization round-trip delay list into the metadata file after completing trajectory data acquisition and / or video recording.

[0010] Another specific step involves an initialization process before requesting subtasks from the cloud task system according to the acquisition instructions. The initialization process includes the following steps: creating a WebSocket send / receive queue; configuring the heartbeat processing parameters of the master acquisition device; and initializing the clock synchronization request set.

[0011] More specifically, during trajectory data acquisition and / or video recording, heartbeat data packets sent by the master acquisition device are periodically received according to heartbeat processing parameters, and the abnormal identifiers in the heartbeat data packets are parsed; the level of the abnormal identifier is determined according to the preset information level mapping table; the abnormal identifier includes at least one of the following: no data state of the master acquisition device and / or slave acquisition device, process crash, network disconnection, or insufficient storage space.

[0012] Another specific step involves uploading the task data set to the cloud task system, which includes the following steps: uploading the task data set to the cloud task system and sending a completion request to the cloud task system after the upload is complete; receiving the completion confirmation information returned by the cloud task system based on the completion request and deleting the local file of the task data set; if no completion confirmation information is received, the local file is retained.

[0013] Another specific measure includes: before requesting a subtask from the cloud task system according to the acquisition command, checking whether the current state is idle; if so, switching the state to waiting for local task retrieval and responding to the acquisition command to request a subtask from the cloud task system; if not, suppressing the triggering of the acquisition command and generating a waiting prompt; and / or after sending a task confirmation command to the main acquisition device, switching the state to waiting for confirmation from the main acquisition device.

[0014] According to another aspect of the present invention, an apparatus for a task timing synchronization method for multi-device joint acquisition is provided, comprising: The task request and confirmation module is used to request sub-tasks from the cloud task system according to the acquisition instructions, obtain the corresponding task session identifier, and send task confirmation instructions to the main control acquisition device. The recording control module is used to send a start recording command to the main control acquisition device when a successful confirmation message is received, so that the main control acquisition device starts the slave acquisition device and performs trajectory data acquisition and / or video recording after a preset start delay; The synchronization recording module is used to record the clock synchronization round-trip delay between the main control acquisition device and the acquisition device. The data association module is used to associate the generated trajectory files and / or video files, as well as metadata files, with the task session identifier to form a task acquisition data group; The upload module is used to upload the task-collected data sets to the cloud-based task system.

[0015] According to another aspect of the present invention, a computing device is provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor. When the processor executes the instructions, it implements the steps of the task timing synchronization method for multi-device joint acquisition described above.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided that stores computer instructions, which, when executed by a processor, implement the steps of the task timing synchronization method for multi-device joint acquisition described above.

[0017] The beneficial effects of this invention are: 1. By using a two-level task confirmation barrier, consistency verification is moved to the pre-recording gating stage, thus avoiding invalid samples caused by repeated triggering or task mismatch from the source.

[0018] 2. By clearing the historical clock synchronization delay list, periodically sending clock synchronization requests and recording round-trip delays, and coordinating with the timing control of starting the main control acquisition device first and delaying the acquisition terminal, a verifiable clock synchronization round-trip delay chain is formed, which facilitates multimodal data alignment.

[0019] 3. The cloud task status is split into two steps: uploading and completed. Local files are only deleted after a completion confirmation is received. This adapts to weak network recovery and resume interrupted downloads, reducing the risk of data loss.

[0020] 4. Through heartbeat analysis and anomaly classification, recording is promptly blocked in case of severe anomalies to reduce invalid data entering subsequent processes.

[0021] 5. It is compatible with multiple data collection task types and has strong scalability; it suppresses repeated triggering in the waiting state, reducing the probability of erroneous operation.

[0022] 6. Move the time for detecting erroneous samples from post-upload cleaning to pre-recording gating to reduce invalid field actions and the generation of large files.

[0023] 7. Differentiated recovery paths are adopted for different fault types to avoid general retries and improve system robustness. Attached Figure Description

[0024] The invention can be better understood by describing exemplary embodiments disclosed herein in conjunction with the accompanying drawings, in which: Figure 1 The diagram shown is a schematic flowchart of a task timing synchronization method for joint data acquisition by multiple devices according to Embodiment 1 of the present invention. Figure 2 The diagram shown is a schematic of the program modules of a task timing synchronization device for multi-device joint acquisition according to Embodiment 1 of the present invention; Figure 3 The diagram shown is a hardware structure schematic of a computing device according to Embodiment 1 of the present invention. Detailed Implementation

[0025] The following describes specific embodiments of the present invention. It should be noted that, in order to provide a concise description, this specification cannot exhaustively describe all features of the actual embodiments. It should be understood that, in the actual implementation of any embodiment, just as in any engineering or design project, various specific decisions are often made to achieve the developer's specific goals and to meet system-related or business-related constraints, and this can change from one embodiment to another. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this invention, some design, manufacturing, or production modifications based on the technical content disclosed in this invention are merely conventional technical means and should not be construed as insufficient content of this invention.

[0026] Unless otherwise defined, the technical or scientific terms used in the claims and description shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in the patent application description and claims of this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. The terms "an" or "a" and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms "comprising" or "including" and similar terms mean that the element or object preceding "comprising" or "including" encompasses the element or object listed following "comprising" or "including" and its equivalents, and do not exclude other elements or objects. The terms "connected" or "linked" and similar terms are not limited to physical or mechanical connections, nor are they limited to direct or indirect connections.

[0027] Example 1 Please see Figure 1 This embodiment proposes a task timing synchronization method for multi-device joint data acquisition, including the following steps S1 to S5: S1: System Initialization On the acquisition terminal, in response to the operation of entering the joint acquisition mode, the communication manager and task consistency state machine are initialized.

[0028] In this embodiment, initializing the communication manager specifically includes the following steps: (1) Create a WebSocket send / receive queue. The WebSocket send / receive queue is created to cache messages to be sent to the main control acquisition device and to receive messages from the main control acquisition device, and to process them in order to ensure the order of message sending and receiving and avoid concurrent conflicts.

[0029] (2) Configure the heartbeat processing parameters of the master acquisition device. When the master acquisition device performs the acquisition task, it will periodically receive heartbeat data packets sent by the master acquisition device according to the heartbeat processing parameters.

[0030] In this embodiment, the heartbeat data packet includes peripheral status fields such as capture (acquisition process status), disk (remaining disk space), wifi (WiFi signal strength), and system (system load).

[0031] (3) Establish a request retransmission and timeout management table. The request retransmission and timeout management table is used to record the request type, request identifier, sending timestamp, and number of retries for requests that have been sent but for which no receipt has been received.

[0032] In this embodiment, the request types include pull_job (pull task command), start_record (start recording command), and stop_record (stop recording command), etc.

[0033] (4) Initialize the clock_sync request set. The clock_sync request set is used to store the identifiers and timestamps of the clock_sync requests that have been sent.

[0034] (5) Configure the information level mapping table. The information level mapping table is used to mark the severity level of error events. Among them, the severity level of error events includes high-priority errors and low-priority errors.

[0035] The initialization of the consistency state machine is specifically as follows: it is initialized to an idle state (i.e., null value / NONE), indicating that the master control acquisition device is not currently recording and has no pending tasks.

[0036] In step S1, by initializing the communication manager and the task consistency state machine, a WebSocket communication channel, a heartbeat monitoring mechanism, a request retransmission and timeout management mechanism, and a clock synchronization request set are established between the acquisition terminal and the main control acquisition device. This provides a traceable and controllable basic operating environment for subsequent task confirmation, timing synchronization, and exception handling, and avoids acquisition errors caused by communication chaos or unclear status.

[0037] S2: Response to data collection trigger, execute two-level task consistency confirmation, specifically including the following S21 to S26: In response to a start data collection request triggered by the data collector via gesture recognition or volume + button press, the data collection terminal performs the following steps: S21: Perform a precondition check to determine whether the preset preconditions are met.

[0038] In this embodiment, the specific steps include: (1) Check whether the user task selection status stored in the acquisition terminal has selected a major category of task (such as joint acquisition, UMI acquisition, etc.). (2) Check whether the data acquisition terminal has obtained and saved the environment identifier (environment_id) and scene identifier (scene_id) by scanning the code; (3) Check whether the external tracking device (tracker) connected to the data acquisition terminal is enabled and working properly.

[0039] If any of the above preconditions are not met during the checks, a corresponding prompt will be generated and the process will be terminated. If the above prerequisites are met, proceed to step S22.

[0040] S22: Consistency state machine wait state check. Checks whether the task consistency state machine inside the acquisition terminal is currently in an idle state.

[0041] In this embodiment, the specific steps include: Check if the current state of the consistent state machine is idle; If so, the current state of the consistency state machine is switched to the WaitingLocalPullJob state, and the subtask is requested from the cloud task system in response to the collection command to obtain the corresponding subtask object. If not, the triggering of the acquisition command is suppressed, and a waiting prompt is generated to prevent repeated triggering from causing state conflicts.

[0042] If the response to the corresponding subtask object is empty, times out, or returns an error code, the current state of the consistency state machine is restored to the idle state.

[0043] In this embodiment, the subtask object includes fields such as task session identifier, case name, task type, and task description; the task session identifier includes a task identifier (TaskID) and a data identifier (data_id). The data identifier is a unique identifier assigned by the cloud-based task system to each data collection session and is used throughout the entire data collection process.

[0044] Furthermore, after acquiring the subtask data, the subtask data is parsed and displayed in the task history list and system notification bar.

[0045] Furthermore, before requesting subtasks from the cloud-based task system, idempotency control can be used to generate idempotency keys and avoid duplicate sending by querying the local idempotency state table.

[0046] S23: Send a task confirmation command to the main control acquisition device; the task confirmation command includes a task session identifier.

[0047] In this embodiment, the specific steps include: Clear the clock synchronization round-trip time list (clock_sync_rtt_list); Construct a clock synchronization request carrying a send timestamp (send_time) and send it to the main control acquisition device; at the same time, construct a pull task instruction (pull_job or command_pull_job) carrying a data identifier (data_id), a project universal unique identifier (project_uuid), and a timestamp (timestamp) and send it to the main control acquisition device via WebSocket.

[0048] S24: Determine whether the main control acquisition device returns a success confirmation message, which includes the following steps: Switch the current state of the consistency state machine to the state of waiting for confirmation from the master acquisition device, listen for the return information from the master acquisition device in the WebSocket send / receive queue, and parse the payload, result, and error message in the return information.

[0049] If no successful confirmation message (i.e., a return message indicating "ok" in the WebSocket send / receive queue) is received within the preset timeout threshold, or if a failure confirmation message is received (e.g., an empty payload, a result not equal to "ok", or a non-empty error_message), the subtask of the current request is terminated, the current state of the consistency state machine is restored to the idle state, and a "Master control acquisition device failed to pull task / timeout" message is generated. At this time, subsequent recording operations are prohibited (the acquisition directory is not created, and the corresponding acquisition device is not started).

[0050] If a success confirmation message is received within the preset timeout threshold, the task information is assigned to RuntimeData.CurrCase (the current case), the TaskID, Case name, and data_id are recorded, the current task is displayed in the PICO interface, and the internal flag is set to allow the creation of collection directories and files.

[0051] In step S2, a two-level task consistency confirmation barrier is implemented: the first level involves the acquisition terminal requesting a subtask from the cloud task system and obtaining a task session identifier; the second level involves the acquisition terminal sending a task confirmation command containing the task session identifier to the main acquisition device and waiting for the main acquisition device to return a success confirmation message. Subsequent recording operations are only allowed after both levels of confirmation are successful; if either level fails (timeout, error return, or empty information), the current subtask is terminated and the creation of any acquisition files is prohibited. Simultaneously, by checking whether the state machine is in an idle state, the repeated triggering of acquisition commands during the waiting process is effectively suppressed, preventing task mismatch and invalid sample generation caused by network latency or slow peripheral response from the source.

[0052] S3: Create local acquisition resources and perform recording timing synchronization. After successful confirmation of consistency between the two-level tasks, the acquisition terminal creates a local acquisition directory and files, and performs recording timing synchronization, specifically including the following steps: S31: Create the corresponding collection directory and collection files locally. Generate the directory name corresponding to the current recording task. The directory name is formed by concatenating the device serial number (SN) and the current Unix timestamp. Then, create the corresponding acquisition directory on the storage device with this directory name. Finally, create the corresponding acquisition file in the acquisition directory.

[0053] The collected files include: (1) Tracking file (tracking_data), used to store trajectory data such as pose and joint angles during the acquisition process; (2) Video file (CameraVideo), used to store the captured video stream; (3) Metadata file (metadata_pico.json) is used to store information such as the currently collected task information, clock synchronization round-trip time, and device information.

[0054] Write the task ID from the task session identifier and the current Unix timestamp into the initial field of the metadata file and associate it with the metadata of the trajectory data file and the video file.

[0055] Meanwhile, the peripheral status field is extracted from the most recently received heartbeat data packet from the master control acquisition device. The acquisition process status (capture), disk remaining space (disk), network signal strength (wifi), system load (system), and gripper status (umi_gripper) are captured and written to the metadata file as a peripheral status snapshot. The snapshot is then associated with the current task session identifier and timestamp to prove that the peripheral environment was normal when the acquisition started.

[0056] S32: Recording timing synchronization starts The acquisition terminal sends a start recording command (start_record) to the master acquisition device via WebSocket, so that the master acquisition device starts the slave acquisition device and, after a preset start delay, performs trajectory data acquisition and / or video recording.

[0057] In this embodiment, the master acquisition device is the development board connected to the head-mounted display, and the slave acquisition devices include a wrist camera and an external gripper. The preset startup delay is 2 seconds, which can be configured to other fixed values ​​according to the actual device warm-up characteristics. That is, the wrist camera and / or the acquisition program on the development board side are started first by the start recording command; then, after waiting for the 2-second startup delay, the local data acquisition (tracking_data) on the acquisition terminal is started, and the metadata file is continuously written and video recording is performed.

[0058] By setting a delayed start, it can be ensured that the wrist camera has started recording and captured the start time of terminal video recording, which facilitates the alignment of multimodal data later.

[0059] S33: Records the clock synchronization round-trip delay between the main control acquisition device and the main control acquisition device, specifically including the following steps: Periodically send clock synchronization requests (clock_sync) to the master control acquisition device. Record the sending timestamp (send_time) each time it is sent, and record the receiving timestamp (recv_time) when the master control acquisition device returns an acknowledgment. Calculate the round-trip time (RTT = recv_time - send_time), and append the sending time, receiving time, and RTT / 2 as a record to the clock synchronization round-trip time list (clock_sync_rtt_list).

[0060] The clock synchronization round-trip delay list is continuously updated until recording stops.

[0061] In step S3, by first sending a start recording command to the master acquisition device and then starting local recording after a preset start delay, it is ensured that the slave acquisition devices (such as wrist cameras) controlled by the master acquisition device have entered the recording state, thus solving the data misalignment problem caused by the difference in start delay of multiple devices. At the same time, clock synchronization requests are periodically sent and round-trip delays are recorded. The clock synchronization round-trip delays are written into the metadata file, forming a verifiable clock synchronization round-trip delay chain, which facilitates the accurate alignment of subsequent multimodal data.

[0062] S4: Monitoring anomalies during recording and recording round-trip delay of clock synchronization. During trajectory data acquisition and / or video recording, the acquisition terminal continuously monitors the status of the main control acquisition device and records the clock synchronization round-trip delay, specifically including the following steps: S41: Heartbeat Reception and Anomaly Classification Processing According to the heartbeat processing parameters configured in step S1, the system periodically receives heartbeat data packets sent by the main control acquisition device.

[0063] The heartbeat data packet includes an anomaly identifier; the anomaly identifier includes at least one of the following: no data state of the master acquisition device and / or slave acquisition device, process crash, network disconnection, or insufficient storage space.

[0064] In this embodiment, the heartbeat data packet is in JSON format and is used to record the operating status of the master acquisition device and the slave acquisition devices it controls.

[0065] Parse the heartbeat data packets and identify any anomaly indicators. Anomaly indicators include, but are not limited to: (1) No data from camera (camera_no_data); (2) The capture process crashed. (3) Network disconnected (wifi_disconnected); (4) Insufficient disk space (disk_space_low).

[0066] The level of an anomaly is determined based on a pre-defined information level mapping table (info_level). For example, no camera data and data acquisition process crashes are high-priority anomalies, while network disconnection and insufficient disk space are low-priority anomalies.

[0067] If the exception is of high priority, a blocking prompt will pop up on the interface of the data acquisition terminal, displaying the error code and error message, and the recording operation will be paused (stopping the writing of new data, but retaining the data already collected). If the exception is of low priority, the event information will be written to the system notification bar and log file, without interrupting the recording.

[0068] S42: Clock synchronization round-trip delay fixed Before stopping recording, perform the following data saving operations: 1) Write the accumulated clock synchronization round-trip delay list from step S33 into the metadata file.

[0069] 2) Retrieve and write the following metadata information that has not yet been recorded: (1) The UTC start time and UTC end time of video recording; (2) Unix start timestamp and Unix end timestamp of the video recording; (3) Device firmware version; (4) Application version; (5) Environmental identifier (environment_id); (6) Scene identifier (scene_id); (7) Collector information; (8) Task description.

[0070] The above information, together with the content written in step S31 (task identifier, start timestamp, peripheral status snapshot), constitutes a complete metadata file.

[0071] In step S4, based on the heartbeat processing parameters configured during initialization, the system periodically receives heartbeat data packets from the main control acquisition device, parses the anomaly identifiers (such as no camera data, process crash, network disconnection, insufficient disk space, etc.), and processes them according to a preset information level mapping table: high-priority anomalies trigger a blocking prompt and pause recording, while low-priority anomalies only log the data. This mechanism enables the acquisition operator to promptly identify serious faults, preventing invalid data from entering subsequent processes, while also preserving a snapshot of the peripheral device's status for later analysis.

[0072] S5: Stop recording and upload the task data set to the cloud task system.

[0073] In response to a stop recording command (e.g., the operator makes another gesture or presses the stop button, not a cancel operation), the following steps are performed: S51: Stop local data acquisition and notify the main data acquisition device The video recording, trajectory data writing, and metadata file writing are stopped sequentially. Then, a stop recording command (stop_record) is sent to the master acquisition device via WebSocket to notify the master acquisition device to stop recording by itself and its subordinate acquisition devices.

[0074] Specifically, after video recording stops, the accumulated clock synchronization round-trip delay list from step S33, along with the video start and end times, device version, and other information collected in step S42, are written to the metadata file (metadata_pico.json) to ensure the metadata file is complete.

[0075] S52: Associate and store the collected files into the local database. The trajectory files and / or video files generated from trajectory data acquisition and / or video recording, along with metadata files, are associated with the task session identifier to form a task acquisition data group.

[0076] Specifically, three records are inserted into the collection history table (collecthistory) of the local SQLite database, corresponding to the trajectory file, video file, and metadata file, respectively.

[0077] In this embodiment, each record contains the following fields: Task ID; Task Name; Task Type; Local File Path; File Size; Media Type (MimeType, such as video / mp4, application / json); Expected Upload Time; IsCombine Flag; Data ID; Username; Upload Status: initially set to "Not Uploaded" or "Pending Upload"; Next Job ID: initially empty, to be updated in subsequent steps.

[0078] S53: Upload the task-collected data set to the cloud-based task system, specifically including the following steps: (1) Reporting Upload Status: Send an upload request (SetCloudUploading) to the cloud task system. This request includes the task identifier (TaskID), data identifier (data_id), and device serial number (SN). The cloud task system returns the next task identifier (nextJobId). If the request is successful and nextJobId is not empty, update the nextJobId field of all records with the corresponding TaskID in the local data collection history table, and update UploadState to "Upload in progress"; if the request fails or nextJobId is empty, retain the local file, record the error information, and jump to exception handling.

[0079] (2) Upload files to cloud storage: Upload the task data set to the cloud task system, and send a completion request to the cloud task system after the upload is complete.

[0080] In this embodiment, after the upload is completed, the object key (objectKey) of each file in the task acquisition data group is obtained in the cloud task system; then, a completion request (CompleteJob) is sent to the cloud task system. This completion request carries information such as: nextJobId, file list (each file contains an object key), video duration, frame rate (FPS), and region.

[0081] (3) Receive completion confirmation: Receive completion confirmation information returned by the cloud task system based on the completion request. Only when completion confirmation information is received, update the UploadState of the corresponding record in the local collection history table to "Completed", and delete the locally stored task collection data group (including trajectory files, video files, and metadata files). If no completion confirmation information is received (including timeout, error return, and empty return), retain all local files, record the corresponding status, keep UploadState as "Uploading" or roll back to "Pending Upload", record the error information, and jump to exception handling.

[0082] S54: Post-processing and Timing Quality Scoring After successful upload and confirmation, the time series quality score for this acquisition can be calculated based on the clock synchronization round-trip time list, the acknowledgment time of the main acquisition device, the warm-up time of the slave acquisition devices, and the local recording startup time. For example, weights can be set for each indicator; the lower the RTT, the faster the acknowledgment, and the warm-up time meeting expectations, the higher the score. If the score is lower than a preset threshold, the sample is marked as "pending review" and will not be used directly for model training, but requires manual review.

[0083] In step S5, after recording stops, the generated trajectory file, video file, and metadata file are associated with the task session identifier to form a task acquisition data group and stored in the local database. When uploading to the cloud, an upload request is first sent to obtain the next task identifier, and a completion request is sent only after the file is actually uploaded to cloud storage. Local files are only deleted when a completion confirmation is received from the cloud; otherwise, all files are retained. Local retention is also supported when canceling a task. This mechanism of step-by-step reporting and file retention is adapted to resuming interrupted uploads and manual troubleshooting in weak network environments, significantly reducing the risk of training data loss.

[0084] More specifically, in some embodiments, if the data collector actively cancels the current task during recording (i.e., before executing S5) or after recording has stopped but before cloud reporting is completed, the following cancellation processing steps are executed: (1) If the device is currently in recording mode, stop local acquisition immediately (stop video recording, trajectory data writing, and close metadata file in sequence), and send a stop recording command (stop_record) to the main acquisition device via WebSocket.

[0085] (2) Send a cancellation upload request to the cloud task system, including the task identifier, data identifier and cancellation reason.

[0086] (3) Update the upload status (UploadState) of the corresponding record in the local collection history table to "Cancel".

[0087] (4) Keep all local collected files (do not delete any files).

[0088] (5) Reset the task consistency state machine to the idle state.

[0089] (6) Display "Task canceled" on the data acquisition terminal interface and prohibit subsequent upload operations.

[0090] More specifically, in some embodiments, if an anomaly occurs during normal operation, the following steps are performed to forcibly retain the file: 1) When any of the following abnormal situations occur, capture the exception and log the error code and error message, while ensuring that the operation of deleting local collection files is not performed: (1) Subtask retrieval failed; (2) The main control acquisition device failed to confirm; (3) Clock synchronization request received no response; (4) The upload request failed or no nextJobId was returned; (5) The request to complete failed or no confirmation of completion was received; (6) File upload to cloud storage failed; (7) The field returned by the cloud is empty; (8) Any other runtime error.

[0091] 2) The original track files, video files and metadata files that are retained will not be deleted. UploadState can be set to "failed" or kept in its original state as appropriate. At the same time, the error information will be associated with the task identifier and written to the local error log table for subsequent retransmission or manual investigation.

[0092] Furthermore, in some embodiments, differentiated recovery strategies can be adopted for different fault types, as follows: (1) Fault type: Cloud task failed; Recovery strategy: Keep local files, mark UploadState as "upload failed", and allow manual retransmission or automatic retry after network recovery.

[0093] (2) Fault type: The main control acquisition device failed; Recovery strategy: Reset the task consistency state machine to the idle state and require the acquisition operator to start again.

[0094] (3) Fault type: Local recording failed; Recovery strategy: Terminate the session and prompt to check storage space.

[0095] Please continue reading. Figure 2This invention illustrates a task timing synchronization device for multi-device joint data acquisition. In this embodiment, the task timing synchronization device for multi-device joint data acquisition may include or be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the invention and implement the aforementioned task timing synchronization method for multi-device joint data acquisition. The program module referred to in this invention is a series of computer program instruction segments capable of performing a specific function, and is more suitable than the program itself for describing the execution process of a task timing synchronization device for multi-device joint data acquisition in a storage medium. The following description will specifically introduce the functions of each program module in this embodiment: The task request and confirmation module is used to request sub-tasks from the cloud task system according to the acquisition instructions, obtain the corresponding task session identifier, and send task confirmation instructions to the main control acquisition device.

[0096] The task consistency verification module is used to maintain the task consistency state machine, which includes the unrecorded state, the waiting-to-retrieve-task-local state, the waiting-to-confirm-from-master-acquisition-device state, and the recording state. Before requesting a subtask, the current state is checked. If it is in the waiting-to-retrieve-task-local state or the waiting-to-confirm-from-master-acquisition-device state, the acquisition trigger is suppressed; otherwise, it switches to the waiting-to-retrieve-task-local state. After sending a task confirmation command to the master-acquisition-device, it switches to the waiting-to-confirm-from-master-acquisition-device state.

[0097] The recording control module is used to send a start recording command to the main control acquisition device when a successful confirmation message is received, so that the main control acquisition device starts the slave acquisition device and performs trajectory data acquisition and / or video recording after a preset start delay.

[0098] The synchronization recording module is used to record the clock synchronization round-trip delay between the main control acquisition device and the acquisition device.

[0099] The data association module is used to associate the generated trajectory files and / or video files, as well as metadata files, with the task session identifier to form a task acquisition data group.

[0100] The upload module is used to upload the task-collected data sets to the cloud-based task system.

[0101] According to another aspect of the present invention, a computing device is provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor. When the processor executes the instructions, it implements the steps of the task timing synchronization method for multi-device joint acquisition described above.

[0102] This embodiment also provides a computing device, such as a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers) capable of executing programs. The computing device 20 in this embodiment includes, but is not limited to, a memory 21 and a processor 22 that can communicate with each other via a system bus. Figure 3 As shown. It should be noted that, Figure 3 Only a computing device 20 with components 21-22 is shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0103] In this embodiment, the memory 21 (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 21 may be an internal storage unit of the computing device 20, such as the hard disk or memory of the computing device 20. In other embodiments, the memory 21 may also be an external storage device of the computing device 20, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computing device 20. Of course, the memory 21 may include both the internal storage unit and the external storage device of the computing device 20. In this embodiment, the memory 21 is typically used to store the operating system and various application software installed on the computing device 20, such as the program code of a task timing synchronization device for multi-device joint acquisition in Embodiment 1. In addition, the memory 21 can also be used to temporarily store various types of data that have been output or will be output.

[0104] In some embodiments, processor 22 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor 22 is typically used to control the overall operation of computing device 20. In this embodiment, processor 22 is used to run program code stored in memory 21 or process data, for example, to run a task timing synchronization device for multi-device joint acquisition, to implement a task timing synchronization method for multi-device joint acquisition as described in Embodiment 1.

[0105] This embodiment also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., which stores a computer program. When the program is executed by a processor, it implements the corresponding function. The computer-readable storage medium of this embodiment is used to store a task timing synchronization device for multi-device joint acquisition, and when executed by a processor, it implements a task timing synchronization method for multi-device joint acquisition according to Embodiment 1.

[0106] In summary, according to the exemplary embodiments, the task timing synchronization method and apparatus for multi-device joint data acquisition of the present invention has the following beneficial effects: 1. By using a two-level task consistency confirmation barrier, consistency verification is moved to the pre-recording gating stage, thus avoiding invalid samples caused by repeated triggering or task mismatch from the source.

[0107] 2. By clearing the historical clock synchronization delay list, periodically sending clock synchronization requests and recording round-trip delays, and coordinating with the timing control of starting the main control acquisition device first and delaying the acquisition terminal, a verifiable clock synchronization round-trip delay chain is formed, which facilitates multimodal data alignment.

[0108] 3. It splits the cloud task status into two steps: uploading and completion. Local files are only deleted after a completion confirmation is received, which is suitable for weak network recovery and interrupted transmission, reducing the risk of data loss.

[0109] 4. Through heartbeat analysis and anomaly classification, recording is promptly blocked in case of severe anomalies to reduce invalid data entering subsequent processes.

[0110] 5. It is compatible with multiple data collection task types and has strong scalability; it suppresses repeated triggering in the waiting state, reducing the probability of erroneous operation.

[0111] 6. Move the time for detecting erroneous samples from post-upload cleaning to pre-recording gating to reduce invalid field actions and the generation of large files.

[0112] 7. Differentiated recovery paths are adopted for different fault types to avoid general retries and improve system robustness.

[0113] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0114] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0115] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0116] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0118] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for time-series synchronization of tasks for multi-device joint data acquisition, characterized in that, Includes the following steps: Request subtasks from the cloud task system according to the collection instructions, and obtain the corresponding task session identifier; Send a task confirmation command to the main control acquisition device; wherein, the task confirmation command includes the task session identifier; Determine whether the master control acquisition device returns a success confirmation message. If so, send a start recording command to the master control acquisition device to enable the master control acquisition device to start the slave acquisition device. After a preset start delay, the slave acquisition device performs trajectory data acquisition and / or video recording, and records the clock synchronization round-trip delay between the master control acquisition device and the slave acquisition device. The trajectory files and / or video files generated from the trajectory data acquisition and / or video recording, as well as the file recording the clock synchronization round-trip delay, are associated with the task session identifier to form a task acquisition data group; The task data collection group is uploaded to the cloud task system.

2. The task timing synchronization method for multi-device joint data acquisition according to claim 1, characterized in that, The process of determining whether the main control acquisition device returns a success confirmation message specifically includes the following steps: If no success confirmation is received within the preset timeout threshold, or if a failure confirmation is received, the subtask of the current request will be terminated. If a success confirmation message is received within the preset timeout threshold, the corresponding collection directory and collection file will be created.

3. A task timing synchronization method for multi-device joint data acquisition according to claim 1 or 2, characterized in that: The clock synchronization round-trip delay between the record and the main control acquisition device specifically includes the following steps: Clear the clock synchronization round-trip delay list; Periodically send clock synchronization requests to the main control acquisition device; The round-trip time is calculated based on the sending time of the clock synchronization request and the receiving time of the corresponding acknowledgment, and the clock synchronization round-trip time list is updated. After completing the trajectory data acquisition and / or video recording, the clock synchronization round-trip delay list is written into a file that records the clock synchronization round-trip delay.

4. The task timing synchronization method for multi-device joint acquisition according to claim 1, characterized in that, Before requesting a subtask from the cloud task system according to the acquisition instruction, an initialization step is also included; The initialization step specifically includes the following steps: Create a WebSocket send / receive queue; Configure the heartbeat processing parameters of the main control acquisition device; Initialize the clock synchronization request set.

5. The task timing synchronization method for multi-device joint acquisition according to claim 4, characterized in that: During the trajectory data acquisition and / or video recording process, the system periodically receives heartbeat data packets sent by the main control acquisition device according to the heartbeat processing parameters, and parses the abnormal identifiers in the heartbeat data packets. The level of the anomaly identifier is determined according to a preset information level mapping table; The anomaly indicators include at least one of the following: no data state of the master acquisition device and / or the slave acquisition device, process crash, network disconnection, or insufficient storage space.

6. The task timing synchronization method for multi-device joint data acquisition according to claim 1, characterized in that, Uploading the task-collected data set to the cloud-based task system specifically includes the following steps: The task data collection group is uploaded to the cloud task system, and a completion request is sent to the cloud task system after the upload is completed; Receive completion confirmation information returned by the cloud task system based on the completion request, and delete the local file of the task collection data group; If the completion confirmation message is not received, the local file is retained.

7. The task timing synchronization method for multi-device joint acquisition according to claim 1, characterized in that, Also includes: Before requesting a subtask from the cloud task system according to the collection instruction, check whether the current state is idle. If so, the state will be switched to waiting for local task retrieval, and the subtask will be requested from the cloud task system in response to the collection command. If not, then suppress the triggering of the acquisition command and generate a waiting prompt; and / or After sending the task confirmation command to the main control acquisition device, the status is switched to waiting for confirmation from the main control acquisition device.

8. A task timing synchronization device for multi-device joint data acquisition, employing the task timing synchronization method for multi-device joint data acquisition as described in any one of claims 1 to 7, characterized in that, include: The task request and confirmation module is used to request sub-tasks from the cloud task system according to the acquisition instructions, obtain the corresponding task session identifier, and send a task confirmation instruction to the main control acquisition device. The recording control module is used to send a start recording command to the master control acquisition device when a successful confirmation message is received, so that the master control acquisition device starts the slave acquisition device, and after a preset start delay, performs trajectory data acquisition and / or video recording. The synchronization recording module is used to record the clock synchronization round-trip delay between the main control acquisition device and the acquisition device. The data association module is used to associate the generated trajectory file and / or video file, as well as the file recording the clock synchronization round-trip delay, with the task session identifier to form a task acquisition data group; The upload module is used to upload the task-collected data set to the cloud-based task system.

9. A computing device, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that, When the processor executes the instructions, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing computer instructions, characterized in that, When executed by the processor, this instruction implements the steps of the method according to any one of claims 1 to 7.

Images