Training data collection and task binding method and device applied to XR head-mounted device

Abstract

The application discloses a kind of training data acquisition and task binding method and device, applied to the field of extended reality technology, its method includes: acquisition front scanning scene code obtains physical scene binding information, combines operator project binding information as access condition;Joint acquisition, head-mounted equipment sends task identification confirmation request to external device, and verifies the confirmation mark returned by external device, starts recording after consistent;With task identification as core, video, tracking data and metadata file are written into local task group record as the file group of same task and scene identification;After verifying that the task identification and scene identification in metadata are consistent with local record, upload after uploading.The application moves the consistency check of multiple device samples to the acquisition access stage, avoids sample mismatch, realizes the double binding of task and scene and full-link closed loop, guarantees file group consistency, improves robustness and data processing efficiency.

Description

Technical Field

[0001] This invention relates to the field of extended reality technology, and in particular to a method and apparatus for training data acquisition and task binding for XR head-mounted devices. Background Technology

[0002] With the rapid development of embodied intelligence and robot learning, using XR head-mounted devices to collect multimodal demonstration data of human operations has become an important way to train robot models. This type of data usually needs to simultaneously record first-person video, operator's hand and body posture, head-mounted display six-DOF pose, controller input, and sensor information from external auxiliary devices. The quality of the data acquisition and the accuracy of task binding directly determine the effectiveness of model training.

[0003] Currently, some XR head-mounted data acquisition systems can achieve basic multimodal recording and cloud-based task distribution. However, in practical engineering applications and embodied intelligence training data production, existing solutions still have significant technical shortcomings in terms of consistent correlation of multi-source data, synchronous binding of tasks and environment, and closed-loop management throughout the entire process. These shortcomings are specifically manifested as follows: In existing data acquisition processes, video, motion tracking data, and descriptive metadata are typically generated step-by-step and then correlated post-processingly based on filenames or timestamps. This loose organization lacks a unified identification mechanism throughout the entire acquisition, storage, and uploading process, failing to establish a strong, integrated association between the task instances distributed from the cloud, the specific physical operating environment, operator identification information, and the generated multimodal file groups at the source of acquisition. This makes it difficult for staff to quickly and accurately determine which task sample a set of files belongs to during subsequent data cleaning and sample management stages, and also makes it impossible to confirm the operating environment of that sample, resulting in inefficient and error-prone dataset processing.

[0004] In scenarios involving joint data acquisition of head-mounted devices and external devices (such as wrist cameras, mechanical grippers, or embedded development boards), existing technologies typically coordinate the work of multiple devices using only simple start / stop commands or time synchronization signals. Each device operates independently, lacking a mechanism to confirm the consistency of the "currently jointly executed task instance" before recording begins. Therefore, if the head-mounted device and external devices acquire different tasks due to network latency, command retries, or differences in cache state, the data generated by the two sets of devices will ultimately belong to different task samples, resulting in a severe sample mismatch. This mismatch directly provides the robot model with incorrect vision-motion mappings, and is extremely difficult to detect and correct in the post-processing stage.

[0005] Furthermore, in most existing solutions, the input of physical environment information (such as room type, workstation number, scene layout, etc.) still relies on manual recording by operators or post-event annotation, and cannot be automatically bound to the task and data file when the collection action is initiated. This disconnect means that the training data often lacks key physical scene semantics, limiting the evaluation and improvement of the model's cross-environment generalization ability. At the same time, in the final stage of data upload and cloud task completion confirmation, the existing process lacks robustness. Conditions such as weak network, device restart, or multi-user rotation often lead to problems such as local files being uploaded but the cloud task status not being closed, and inconsistencies between local and cloud records, affecting the reliability of data production and engineering efficiency.

[0006] In summary, improving the accuracy and consistency of the correlation between tasks, scenarios, devices, and operators during multimodal data acquisition, and achieving a reliable closed loop across the entire data acquisition process from acquisition to the cloud, are pressing technical challenges that need to be addressed. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a technology that can improve the accuracy and consistency of the association between tasks, scenarios, devices and operators in the multimodal acquisition process, and realize a reliable closed loop of the entire link from acquisition to the cloud.

[0008] The objective of this invention is achieved through the following technical solution: Therefore, according to one aspect of the present invention, a method for training data acquisition and task binding applied to an XR head-mounted device is provided, characterized by comprising the following steps: acquiring operator project binding information and physical scene binding information, and generating pre-acquisition access information; wherein, the physical scene binding information includes a scene identifier; acquiring a collection task list based on the pre-acquisition access information, and confirming the requested collection task; acquiring the task identifier corresponding to the collection task, and generating session binding information based on the task identifier and scene identifier; performing cloud task verification based on the session binding information; when the acquisition mode is joint acquisition, performing consistency confirmation on the external device based on the task identifier in the session binding information, and forming a corresponding synchronization record; creating local acquisition resources, and performing multimodal recording on the head-mounted device and the external device; after stopping recording, generating a metadata file containing the task identifier and scene identifier, and writing the metadata file, the recorded video file, and the tracking data file into the local task group record.

[0009] Specifically, based on the task identifier in the session binding information, consistency verification is performed on the external device, which includes the following steps: clearing the synchronization record of the previous acquisition session corresponding to the external device; sending a synchronization request to the external device, and generating a synchronization record corresponding to the current acquisition session based on the sending time and receiving acknowledgment time of the synchronization request; sending a task confirmation request carrying the task identifier to the external device, receiving the confirmation information returned by the external device for the task confirmation request, and determining whether the confirmation identifier in the confirmation information is consistent with the task identifier; if the confirmation identifier is consistent with the task identifier, the consistency verification of the external device is deemed to have passed.

[0010] Another specific step involves obtaining operator project binding information and physical scene binding information, and generating pre-collection access information. This includes the following steps: obtaining operator login information and project selection information, and generating operator project binding information based on these information; obtaining scene codes, parsing the scene codes, and generating corresponding scene identifiers; validating the scene identifiers, and generating corresponding physical scene binding information after successful validation; and generating corresponding pre-collection access information based on the operator project binding information and physical scene binding information.

[0011] More specifically, the process of obtaining a collection task list based on pre-collection access information and confirming the requested collection task includes the following steps: generating a cloud task object request based on the operator's project binding information, physical scene binding information, and input task filtering information; wherein, the task filtering information includes task name or task type; obtaining the collection task list corresponding to the cloud task object request; and in response to the selection operation of the target task in the collection task list, confirming the corresponding target task as the requested collection task.

[0012] Another specific method is to perform cloud task verification based on session binding information, which includes the following steps: sending a cloud task verification request carrying session binding information and receiving the returned verification result; wherein, the verification result is generated based on whether the task identifier and scene identifier are consistent with the task identifier and scene identifier recorded in the cloud.

[0013] The above process involves the following steps: After recording stops, a metadata file containing task identifiers and scene identifiers is generated. The metadata file, along with the recorded video file and tracking data file, is then written to the local task group record. Specifically, this includes the following steps: After recording stops, the recording start time and recording end time corresponding to this recording are obtained; a metadata file is generated based on session binding information and synchronization records; the corresponding local task group record is located using the task identifier in the metadata file, and the scene identifier is written to the local task group record; the metadata file, video file, and tracking data file are written to the local task group record respectively.

[0014] More specifically, after writing the metadata file, video file, and tracking data file as a file group corresponding to the same task identifier and scene identifier into the local task group record, the following steps are also included: reading the metadata file in the local task group record and obtaining the task identifier and scene identifier in the metadata file; determining whether the task identifier and scene identifier in the metadata file are consistent with the task identifier and scene identifier corresponding to the local task group record; if consistent, submitting the metadata file, video file, and tracking data file corresponding to the local task group record as a file group of the same acquisition task to the cloud; sending a task completion request carrying the task identifier and scene identifier to the cloud and receiving the returned completion confirmation information.

[0015] According to another aspect of the present invention, a training data acquisition and task binding device for an XR head-mounted device is provided, comprising: an information acquisition module for acquiring operator project binding information and physical scene binding information, and generating pre-acquisition access information; a task confirmation module for acquiring a list of acquisition tasks based on the pre-acquisition access information, and confirming the requested acquisition tasks; a task binding and verification module for acquiring the task identifier corresponding to the acquisition task, and generating session binding information based on the task identifier and scene identifier; and performing cloud task verification based on the session binding information; a consistency verification module for performing consistency verification on external devices based on the task identifier in the session binding information when the acquisition mode is joint acquisition, and forming a corresponding synchronization record; an execution module for creating local acquisition resources and performing multimodal recording on the head-mounted device and external devices; a local storage module for generating a metadata file containing the task identifier and scene identifier after recording stops, and writing the metadata file, the recorded video file, and the tracking data file into a local task group record; and a data upload module for submitting the metadata file, video file, and tracking data file corresponding to the local task group record as a file group of the same acquisition task to the cloud, and sending a task completion request carrying the task identifier and scene identifier to the cloud.

[0016] According to another aspect of the present invention, a computing device is provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor. When the processor executes the instructions, it implements the steps of the training data acquisition and task binding method applied to an XR head-mounted device as described above.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided that stores computer instructions, which, when executed by a processor, implement the steps of the above-described method for training data acquisition and task binding applied to an XR head-mounted device.

[0018] The beneficial effects of this invention are as follows: By binding the physical scene before data acquisition and using it together with the task identifier as the acquisition admission condition, and by sending a confirmation request carrying the task identifier to an external device for consistency verification in the joint acquisition mode, recording can only start after the confirmation is passed. This moves the discovery point of multi-device sample mismatch from the post-processing stage to the acquisition admission stage, avoiding irreparable sample mismatch from the source. By using the task identifier as the core to manage video, tracking data, and metadata as the same file group and performing consistency verification before uploading, the consistency and integrity of the multimodal file group are guaranteed. The task identifier and scene identifier run through the entire process of pre-acquisition verification, joint acquisition confirmation, metadata generation, local record writing, file upload, and cloud request completion, realizing a strong dual binding of task semantics and physical scene semantics and an end-to-end full-link traceability closed loop. This significantly reduces the cost of data cleaning and quality inspection, while improving the system robustness and automation efficiency of subsequent data processing under complex conditions such as weak network and device restart. Attached Figure Description

[0019] The invention can be better understood by describing exemplary embodiments disclosed herein in conjunction with the accompanying drawings, in which: Figure 1 The diagram shown is a schematic flowchart of a training data acquisition and task binding method for an XR head-mounted device according to Embodiment 1 of the present invention. Figure 2 The diagram shown is a schematic of a program module for a training data acquisition and task binding device for an XR head-mounted device according to a first embodiment of the present invention. Figure 3 The diagram shown is a hardware structure schematic of a computing device according to Embodiment 1 of the present invention. Detailed Implementation

[0020] The following describes specific embodiments of the present invention. It should be noted that, in order to provide a concise description, this specification cannot exhaustively describe all features of the actual embodiments. It should be understood that, in the actual implementation of any embodiment, just as in any engineering or design project, various specific decisions are often made to achieve the developer's specific goals and to meet system-related or business-related constraints, and this can change from one embodiment to another. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this invention, some design, manufacturing, or production modifications based on the technical content disclosed in this invention are merely conventional technical means and should not be construed as insufficient content of this invention.

[0021] Unless otherwise defined, the technical or scientific terms used in the claims and description shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in the patent application description and claims of this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. The terms "an" or "a" and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms "comprising" or "including" and similar terms mean that the element or object preceding "comprising" or "including" encompasses the element or object listed following "comprising" or "including" and its equivalents, and do not exclude other elements or objects. The terms "connected" or "linked" and similar terms are not limited to physical or mechanical connections, nor are they limited to direct or indirect connections.

[0022] Example 1 Please see Figure 1 This embodiment proposes a method for training data acquisition and task binding applied to XR head-mounted devices, characterized by including the following steps S1 to S8: S1: Obtain operator project binding information and physical scene binding information, and generate pre-collection access information; wherein, the physical scene binding information includes scene identifier; specifically including the following steps S11 to S14: S11: Obtain operator login information and project selection information, and generate operator project binding information based on operator login information and project selection information.

[0023] In this embodiment, the operator login information includes the operator identifier, operator name, and body parameters related to the operator. Body parameters may include height, wingspan, leg length, or other data used to interpret human operational demonstration postures. Project selection information includes the project identifier, task type, acquisition mode, and candidate task range. After generating operator project binding information based on the above information, subsequent task filtering, metadata generation, local task group recording, and cloud task completion requests can all be traced back to the same operator and the same project.

[0024] S12: Obtain the scene code, parse the scene code, and generate the corresponding scene identifier.

[0025] In this embodiment, the scene code can be a QR code. When parsing the scene code, the scene identifier can be obtained, as well as physical scene information such as environmental identifier, latitude and longitude, coordinate system, robot number, workstation number, lighting conditions, object set version, calibration board number, maximum number of scene acquisitions, or safety limits.

[0026] S13: Perform validity verification on the scene identifier, and generate the corresponding physical scene binding information after the verification is successful.

[0027] In this embodiment, if the scene code is empty, the scene code format is non-standard, the scene identifier is missing, or the scene identifier cannot match the collectable scene of the current project, the validity check is deemed to have failed, and the process of proceeding to the subsequent task assignment process is blocked. If the scene identifier passes the check, physical scene binding information is generated. The physical scene binding information includes the scene identifier, and in some embodiments, it also includes the environment identifier, latitude and longitude, and coordinate system.

[0028] S14: Generate corresponding pre-collection access information based on the operator's project binding information and physical scene binding information.

[0029] In this embodiment, the pre-collection admission information is used to indicate that the current collection process has both operator project binding relationship and physical scene binding relationship.

[0030] If the operator's project binding information does not exist, the physical scene binding information does not exist, or the scene identifier verification fails, the pre-collection access information will not be generated, and the user will be prompted to log in again, reselect the project, or re-obtain the scene code.

[0031] By establishing a binding between the operator, project, and physical scene in step S1 before data collection, this invention enables subsequent data collection tasks to not rely solely on device serial numbers or timestamps, but to have access conditions in both the operator and scene dimensions before recording, thereby reducing the risk of unclear sample attribution or missing scene labels in the future.

[0032] S2: Obtain the collection task list based on the pre-collection access information and confirm the requested collection task; specifically including the following steps S21 to S23: S21: Generate a cloud task object request based on the operator's project binding information, physical scene binding information, and input task filtering information; wherein, the task filtering information includes task name or task type.

[0033] In this embodiment, the task filtering information is input by the operator and also determined by the current project configuration, acquisition mode, or physical scene binding information. The scene identifier or environment identifier in the physical scene binding information can be used to limit the visible task range. For example, when the cloud task is configured with an applicable environment type, the environment type corresponding to the current physical scene is matched with the task configuration; if the task configuration is not applicable to the current physical scene, the task is removed from the acquisition task list or its display priority is reduced.

[0034] S22: Get the list of collection tasks corresponding to the cloud task object request.

[0035] The task list includes at least one requestable task. Each task includes a task name, task description, task type, applicable scenario scope, data collection status, and corresponding cloud task information. If no task is returned from the cloud, the task list is empty, or the task list does not match the current operator's project binding information, a prompt will appear to retrieve the task list again.

[0036] S23: In response to the selection operation of the target task in the collection task list, the corresponding target task is confirmed as the requested collection task.

[0037] Specifically, once a target task is confirmed, subsequent tasks will only acquire task identifiers and execute session bindings for that target task. If the operator switches projects, rescans scene codes, or the acquisition mode changes, task filtering and confirmation will be re-executed to prevent tasks confirmed in old projects or old scenes from being misused in new acquisition processes.

[0038] In step S2, the physical scene binding information of this invention is not only written into the metadata, but also participates in the task acquisition and task confirmation process. This can reduce the probability of operators receiving tasks in the wrong scene and make the subsequent task identifier and scene identifier have a clear source relationship.

[0039] S3: Obtain the task identifier corresponding to the data collection task, and generate session binding information based on the task identifier and scene identifier; specifically including the following steps S31 to S33: S31: Obtain the cloud task information corresponding to the requested collection task.

[0040] Specifically, cloud task information may include task name, task description, task identifier, cloud job identifier, case object, project identifier, and the scope of scenarios that the task is allowed to be bound to. If the cloud task information is empty, the task link has expired, or the task has been occupied by another data collection session, the current task confirmation will be aborted.

[0041] S32: Extract the task identifier corresponding to the data collection task from the task information in the cloud.

[0042] Specifically, the task identifier is used to uniquely identify the task or training sample in this data collection task. The task identifier serves as a consistency anchor in subsequent external device confirmations, metadata files, local task group records, and cloud task completion requests.

[0043] In some implementations, the task identifier corresponds to the identifier of a human operation case in the cloud.

[0044] In other implementations, the task identifier corresponds to a cloud job identifier, a sample identifier, or a combination thereof.

[0045] S33: Generate session binding information based on task identifier and scene identifier.

[0046] Specifically, the session binding information includes task identifier and scene identifier, as well as operator project binding information, physical scene binding information, acquisition mode, device identifier, and session creation time. After the session binding information is generated, subsequent cloud task verification requests, external device task confirmation requests, metadata file generation, and local task group records will all be based on this session binding information.

[0047] In step S3, this invention combines task identifiers and scene identifiers into session binding information, enabling a dual-axis binding of task semantics and physical scene semantics. Compared to recording only task identifiers or only scene identifiers, this process allows the training platform to confirm "who performed which task in which scene".

[0048] S4: Perform cloud task verification based on session binding information; specifically including the following steps S41 to S42: S41: Send a cloud task verification request carrying session binding information.

[0049] S42: Determine whether the task identifier and scene identifier are consistent with the task identifier and scene identifier recorded in the cloud, and generate the corresponding verification result.

[0050] S43: And receive the returned verification result; wherein, the verification result is generated based on whether the task identifier and scene identifier are consistent with the task identifier and scene identifier recorded in the cloud.

[0051] By confirming in step S4 that the local session binding information is consistent with the cloud task record before recording, this invention can prevent the terminal from mistakenly entering old scenes, old tasks or cached tasks into the acquisition process, thereby blocking the incorrect binding of tasks and scenes before recording.

[0052] S5: When the acquisition mode is joint acquisition, based on the task identifier in the session binding information, a consistency confirmation is performed on the external device, and a corresponding synchronization record is generated; specifically, this includes the following steps S51 to S53: S51: Clear the synchronization record of the previous acquisition session corresponding to the external device.

[0053] Before sending this synchronization request, clear the synchronization record corresponding to the previous acquisition session so that the synchronization record written to the metadata file in the future corresponds only to the current acquisition session. This is to prevent the clock synchronization round-trip delay of the previous task from being mixed into the current acquisition task.

[0054] S52: Send a synchronization request to an external device, and generate a synchronization record corresponding to the current acquisition session based on the sending time and receiving receipt time of the synchronization request.

[0055] Specifically, the synchronization record includes the synchronization request sending time, the external device acknowledgment receiving time, and the round-trip delay calculated from both.

[0056] In some implementations, multiple synchronization requests can be sent consecutively, and the round-trip delays can be used to form a synchronization record list for subsequent multi-device time alignment, data trimming, and anomaly troubleshooting.

[0057] In some implementations, external device communication can employ web sockets, Bluetooth, message queues, Universal Serial Bus, LAN Transmission Control Protocol, or hardware synchronization lines. External device consistency verification can also be extended from a single task verification to multi-stage verification, for example, carrying a task identifier at each stage: task verification, start recording, stop recording, and file submission. If the verification identifier at any stage is inconsistent with the task identifier, the corresponding acquisition task is marked as abnormal, and the local task group record is retained.

[0058] S53: Send a task confirmation request carrying a task identifier to an external device, receive confirmation information returned by the external device in response to the task confirmation request, and determine whether the confirmation identifier in the confirmation information is consistent with the task identifier.

[0059] If the confirmation identifier matches the task identifier, the external device consistency confirmation is deemed successful.

[0060] If the confirmation identifier and / or task identifier are inconsistent, joint recording will not proceed. External devices will only be allowed to participate in subsequent multimodal recording if both the confirmation identifier and task identifier are consistent.

[0061] The task confirmation request carries a task identifier and a project identifier. When sending a task confirmation request, the request time, request content, number of resends, and expected task identifier are registered for the current task confirmation request.

[0062] In this embodiment, after receiving the confirmation information returned by the external device, the corresponding task confirmation request is located based on the request time or request context, and the confirmation identifier in the confirmation information is read. If the confirmation identifier is empty, the confirmation information times out, communication fails, or the confirmation information characterizes a failure, it is determined that the external device has not passed the consistency confirmation.

[0063] By using step S5, the present invention uses the confirmation of the same task identifier by external devices as a condition for joint recording, which can bind head-mounted side video, head-mounted side tracking data, external device data and metadata files to the same acquisition task, thus avoiding external devices from recording data of another acquisition task due to network timing, cache status or independent task retrieval.

[0064] S6: Create local acquisition resources and perform multimodal recording on the head-mounted device and external devices; specifically including the following steps S61 to S64: S61: Create a local acquisition directory based on session binding information, and generate video file paths, tracking data file paths, and metadata file paths.

[0065] The local acquisition directory is generated based on the device identifier, task name, task identifier, acquisition time, or a combination thereof. The video file path is used to store head-mounted video files, the tracking data file path is used to store head-mounted tracking data files, and the metadata file path is used to store the metadata file corresponding to this acquisition. All of these paths are associated with the same session binding information.

[0066] S62: When the acquisition mode is joint acquisition and the consistency of the external device is confirmed, send a start recording command to the external device.

[0067] Specifically, after sending a start recording command to the external device, recording on the head-mounted device begins after a preset delay, ensuring that the data recording start time of the external device is no later than the start time of the head-mounted device video. The preset time can be set according to the startup time of the external device, communication delay, or hardware triggering conditions.

[0068] S63: Start video recording and tracking data logging on the head-mounted side.

[0069] Among them, video recording is used to record the operator's perspective or the perspective of the head-mounted side camera; tracking data recording is used to record the head-mounted device's posture, hand joints, controller status, inertial measurement data, body tracking data, ground height, and input status frame by frame.

[0070] In this embodiment, each frame of tracking data carries a timestamp to align with video files and external device data.

[0071] S64: Generate video files and tracking data files during recording, and associate the video files and tracking data files with session binding information.

[0072] Furthermore, in the joint acquisition mode, external devices can also simultaneously record wrist camera data, development board data, gripper status, force sensor data, tactile sensor data, robot status, or operation intention tags. External device data enters the same acquisition session through task identifier consistency confirmation in step S5.

[0073] In this embodiment, the files obtained from multimodal recording are not limited to video files and tracking data files, but may also include external camera files, wrist camera files, force sensor files, tactile sensor files, voice annotation files, robot status files, or operation intention label files. These extended files can be written together with metadata files, video files, and tracking data files into the same local task group record.

[0074] The present invention establishes local acquisition resources through step S6, and then performs multimodal recording on the head-mounted device and external devices. This enables files of different modalities to have a unified local path and session binding relationship when they are generated, reducing the need for manual grouping based on filenames or timestamps after the acquisition is completed.

[0075] S7: After recording stops, a metadata file containing task identifiers and scene identifiers is generated, and the metadata file, along with the recorded video files and tracking data files, is written to the local task group record. This includes the following steps S71 to S74: S71: Get the recording start time and recording end time corresponding to this recording.

[0076] In response to the stop recording command, video recording and tracking data logging on the head-mounted side are stopped.

[0077] In the joint acquisition mode, a stop recording command is also sent to the external device to stop the external device from recording the data of the current acquisition session.

[0078] The recording start time and recording end time are used to generate metadata files and to calculate video duration or perform multimodal time alignment.

[0079] S72: Generate metadata files based on session binding information and synchronization records.

[0080] The metadata file includes task and scene identifiers, as well as environment identifiers, operator information, project identifiers, device identifiers, acquisition methods, video parameters, recording start time, recording end time, device information, and synchronization records. Operator information includes operator identifier, operator name, and physical parameters. Video parameters include resolution, bitrate, frame rate, and duration. Device information includes device serial number, whether a tracker is used, and communication mode.

[0081] When the acquisition mode is joint acquisition, the metadata file is written to the synchronization record corresponding to the current acquisition session.

[0082] S73: Locate the corresponding local task group record using the task identifier in the metadata file, and write the scene identifier into the local task group record.

[0083] If a local task group record corresponding to the task identifier already exists locally, that record is located; otherwise, a new local task group record is created based on the task identifier. The local task group record is written to the scene identifier, enabling subsequent local retrieval, upload recovery, and cloud completion requests to obtain the task-scene binding relationship from the local task group record.

[0084] S74: Write the metadata file, video file, and tracking data file to the local task group record respectively.

[0085] Metadata files, video files, and tracking data files are written to the local task group record as file groups corresponding to the same task identifier and scene identifier. The local task group record can record file path, file type, task identifier, scene identifier, operator name, expected upload time, valid data marker, joint acquisition marker, upload status, and next task identifier. For joint acquisition mode, the local task group record can also record external device confirmation results or joint acquisition markers.

[0086] In step S7, the task identifier and scene identifier are simultaneously written to the metadata file and the local task group record, enabling the local system to identify which video files, tracking data files, and metadata files belong to the same acquisition task and the same physical scene, even when the network is weak, the upload is delayed, or the device restarts.

[0087] S8: Upload local task group records to the cloud; specifically including the following steps S81 to S83: S81: Read the metadata file in the local task group record and obtain the task identifier and scene identifier in the metadata file.

[0088] Before submitting a local task group record, the metadata file is read and the task identifier and scene identifier in the metadata file are parsed. If the metadata file cannot be read, cannot be parsed, or is missing a task identifier or scene identifier, the local task group record is not submitted, and the corresponding file is retained for repair or regeneration of the metadata file.

[0089] S82: Determine whether the task identifier and scene identifier in the metadata file are consistent with the task identifier and scene identifier corresponding to the local task group record, respectively.

[0090] If the task identifier in the metadata file is inconsistent with the task identifier in the local task group record, or if the scene identifier in the metadata file is inconsistent with the scene identifier in the local task group record, it is determined that the local task group record has a binding anomaly, and the local task group record is retained but not submitted to the cloud as a valid collection task.

[0091] If they match, the metadata file, video file, and tracking data file corresponding to the local task group will be submitted to the cloud as a file group for the same acquisition task.

[0092] Specifically, when submitting to the cloud, metadata files, video files, and tracking data files can be uploaded as a single file group, and the cloud object path corresponding to each file should be recorded. The submission relationship of each file is based on the task identifier and scene identifier recorded in the local task group, and the successful upload of a single file is not the basis for determining the completion of the entire acquisition task.

[0093] S83: Send a task completion request with task identifier and scene identifier to the cloud, and receive the returned completion confirmation information.

[0094] After submitting the same file group, send a task completion request to the cloud. The task completion request carries a task identifier and a scene identifier, and may also include a file list, file paths, video duration, frame rate, capture area, device identifier, or operator information.

[0095] When submission fails, completion confirmation information is missing, the task identifier returned by the cloud is inconsistent, the scenario identifier returned by the cloud is inconsistent, or the network is interrupted, the local task group record and corresponding file will not be deleted. The retained local task group record can continue to be used when the network is restored, manually repaired, or resubmitted.

[0096] By performing consistency verification around task identifiers and scenario identifiers in step S8 before local submission and after completion in the cloud, this invention can avoid situations where files have been uploaded but the cloud task is not closed, or where the cloud completion status corresponds to the wrong task or scenario, thereby improving the integrity and traceability of training data file groups.

[0097] Please continue reading. Figure 2This invention illustrates a training data acquisition and task binding device for an XR head-mounted device. In this embodiment, the device may include or be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the invention and implement the aforementioned training data acquisition and task binding method for an XR head-mounted device. The program module referred to in this invention is a series of computer program instruction segments capable of performing specific functions, and is more suitable than the program itself for describing the execution process of a training data acquisition and task binding device for an XR head-mounted device in a storage medium. The following description will specifically introduce the functions of each program module in this embodiment: The information acquisition module is used to acquire operator project binding information and physical scene binding information, and generate pre-collection access information.

[0098] The task confirmation module is used to obtain a list of collection tasks based on the pre-collection access information and confirm the requested collection tasks.

[0099] The task binding and verification module is used to obtain the task identifier corresponding to the collection task, generate session binding information based on the task identifier and the scene identifier, and perform cloud task verification based on the session binding information.

[0100] The consistency verification module is used to perform consistency verification on external devices based on the task identifier in the session binding information when the acquisition mode is joint acquisition, and to form corresponding synchronization records.

[0101] The execution module is used to create local acquisition resources and perform multimodal recording on the head-mounted side and external devices.

[0102] The local storage module is used to generate a metadata file containing task identifiers and scene identifiers after recording stops, and to write the metadata file, along with the recorded video files and tracking data files, into the local task group record.

[0103] The data upload module is used to submit the metadata files, video files, and tracking data files corresponding to the local task group records as a file group of the same acquisition task to the cloud, and send a task completion request with task identifier and scene identifier to the cloud.

[0104] This embodiment also provides a computing device, such as a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers) capable of executing programs. The computing device 20 in this embodiment includes, but is not limited to, a memory 21 and a processor 22 that can communicate with each other via a system bus. Figure 3As shown. It should be noted that, Figure 3 Only a computing device 20 with components 21-22 is shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0105] In this embodiment, the memory 21 (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 21 may be an internal storage unit of the computing device 20, such as the hard disk or memory of the computing device 20. In other embodiments, the memory 21 may also be an external storage device of the computing device 20, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computing device 20. Of course, the memory 21 may include both the internal storage unit and the external storage device of the computing device 20. In this embodiment, the memory 21 is typically used to store the operating system and various application software installed on the computing device 20, such as the program code of a training data acquisition and task binding device for an XR head-mounted device according to Embodiment 1. In addition, the memory 21 can also be used to temporarily store various types of data that have been output or will be output.

[0106] In some embodiments, processor 22 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor 22 is typically used to control the overall operation of computing device 20. In this embodiment, processor 22 is used to run program code stored in memory 21 or process data, for example, to run a training data acquisition and task binding device for an XR head-mounted device, to implement a training data acquisition and task binding method for an XR head-mounted device according to Embodiment 1.

[0107] This embodiment also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., which stores a computer program. When the program is executed by a processor, it implements the corresponding function. The computer-readable storage medium of this embodiment is used to store a training data acquisition and task binding device for an XR head-mounted device. When executed by a processor, it implements the training data acquisition and task binding method for an XR head-mounted device according to Embodiment 1.

[0108] In summary, according to the exemplary embodiments, the training data acquisition and task binding method and apparatus for XR head-mounted devices of the present invention have the following beneficial effects: 1. By forming session binding information through task identifiers and scene identifiers, the task semantics and physical scene semantics are simultaneously brought forward to the data collection and admission stage, reducing the need for manual supplementation of scene information or manual determination of task ownership after data collection.

[0109] 2. By verifying the consistency of the same task identifier through external devices, external devices cannot participate in recording arbitrarily just by being online. Instead, they must establish a consistent session with the head-mounted acquisition task, thereby reducing the probability of multi-device mismatched samples from the source.

[0110] 3. By writing to both the metadata file and the local task group record, the video file, tracking data file, and metadata file can still be traced back to the same task identifier and scene identifier even in scenarios with weak network conditions, delayed uploads, or device restarts.

[0111] 4. By confirming the consistency of task identifiers and scenario identifiers in the cloud task completion request and completion confirmation information, a closed loop is formed between the cloud task status and the local file group status, reducing the situation where the file group has been submitted but the task status is not completed or has been completed to the wrong task.

[0112] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0113] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0114] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0115] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0116] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0117] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for training data acquisition and task binding applied to XR head-mounted devices, characterized in that, Includes the following steps: Obtain operator project binding information and physical scene binding information, and generate pre-collection access information; wherein, the physical scene binding information includes scene identifier; Based on the pre-collection access information, obtain the collection task list and confirm the requested collection task; Obtain the task identifier corresponding to the data collection task, and generate session binding information based on the task identifier and the scene identifier; Perform cloud task verification based on the session binding information; When the acquisition mode is joint acquisition, based on the task identifier in the session binding information, consistency confirmation is performed on the external device, and a corresponding synchronization record is formed; Create local acquisition resources and perform multimodal recording on the head-mounted device and external devices; After recording stops, a metadata file containing the task identifier and the scene identifier is generated, and the metadata file, along with the recorded video file and tracking data file, is written to the local task group record.

2. The method for training data acquisition and task binding applied to XR head-mounted devices according to claim 1, characterized in that, The process of performing consistency verification on external devices based on the task identifier in the session binding information specifically includes the following steps: Clear the synchronization record of the previous acquisition session corresponding to the external device; send a synchronization request to the external device, and generate a synchronization record corresponding to the current acquisition session based on the sending time and receiving receipt time of the synchronization request; Send a task confirmation request carrying the task identifier to the external device, receive confirmation information returned by the external device in response to the task confirmation request, and determine whether the confirmation identifier in the confirmation information is consistent with the task identifier; If the confirmation identifier matches the task identifier, then the external device consistency confirmation is deemed successful.

3. The method for training data acquisition and task binding applied to XR head-mounted devices according to claim 1, characterized in that, The process of obtaining operator project binding information and physical scene binding information, and generating pre-collection access information, specifically includes the following steps: Obtain operator login information and project selection information, and generate operator project binding information based on the operator login information and project selection information; Obtain the scene code, parse the scene code, and generate the corresponding scene identifier; The validity of the scene identifier is verified, and the corresponding physical scene binding information is generated after the verification is successful. Based on the operator project binding information and the physical scene binding information, corresponding pre-collection access information is generated.

4. The method for training data acquisition and task binding applied to XR head-mounted devices according to claim 3, characterized in that, The process of obtaining the collection task list based on the pre-collection access information and confirming the requested collection task specifically includes the following steps: Based on the operator project binding information, physical scene binding information, and input task filtering information, a cloud task object request is generated; wherein, the task filtering information includes task name or task type; Obtain the list of collection tasks corresponding to the cloud task object request; In response to the selection operation of the target task in the collection task list, the corresponding target task is confirmed as the requested collection task.

5. The method for training data acquisition and task binding applied to XR head-mounted devices according to claim 1, characterized in that, The cloud task verification based on the session binding information specifically includes the following steps: Send a cloud task verification request carrying the session binding information and receive the returned verification result; wherein, the verification result is generated based on whether the task identifier and the scene identifier are consistent with the task identifier and scene identifier recorded in the cloud.

6. A method for training data acquisition and task binding applied to an XR head-mounted device according to any one of claims 1 to 5, characterized in that, After recording stops, a metadata file containing the task identifier and the scene identifier is generated, and the metadata file, along with the recorded video file and tracking data file, is written to the local task group record. This process includes the following steps: After recording stops, obtain the recording start time and recording end time corresponding to this recording session; The metadata file is generated based on the session binding information and the synchronization record; Locate the corresponding local task group record using the task identifier in the metadata file, and write the scene identifier into the local task group record; Write the metadata file, video file, and tracking data file to the local task group record respectively.

7. The method for training data acquisition and task binding applied to an XR head-mounted device according to claim 6, characterized in that, After writing the metadata file, the video file, and the tracking data file into the local task group record as a file group corresponding to the same task identifier and scene identifier, the following steps are also included: Read the metadata file in the local task group record and obtain the task identifier and scene identifier in the metadata file; Determine whether the task identifier and scene identifier in the metadata file are consistent with the task identifier and scene identifier corresponding to the local task group record, respectively. If they match, the metadata file, video file, and tracking data file corresponding to the local task group record will be submitted to the cloud as a file group for the same acquisition task. Send a task completion request carrying a task identifier and a scene identifier to the cloud, and receive the returned completion confirmation information.

8. A training data acquisition and task binding device for an XR head-mounted device, employing the training data acquisition and task binding method for an XR head-mounted device as described in any one of claims 1 to 7, characterized in that, include: The information acquisition module is used to acquire operator project binding information and physical scene binding information, and generate pre-collection access information; The task confirmation module is used to obtain a list of collection tasks based on the pre-collection access information and confirm the requested collection tasks. The task binding and verification module is used to obtain the task identifier corresponding to the collection task, generate session binding information based on the task identifier and the scene identifier, and perform cloud task verification based on the session binding information. The consistency verification module is used to perform consistency verification on external devices based on the task identifier in the session binding information when the acquisition mode is joint acquisition, and to form a corresponding synchronization record. The execution module is used to create local acquisition resources and perform multimodal recording on the head-mounted device and external devices; The local storage module is used to generate a metadata file containing the task identifier and the scene identifier after recording stops, and to write the metadata file, the recorded video file and the tracking data file into the local task group record; The data upload module is used to submit the metadata file, video file and tracking data file corresponding to the local task group record as a file group of the same acquisition task to the cloud, and send a task completion request carrying task identifier and scene identifier to the cloud.

9. A computing device, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that, When the processor executes the instructions, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing computer instructions, characterized in that, When executed by the processor, this instruction implements the steps of the method according to any one of claims 1 to 7.

Images