A data management method, a data management system and an electronic device

Abstract

The application relates to the technical field of data management, in particular to a data management method, a data management system and an electronic device. The method comprises the following steps: acquiring a data collection task; issuing the data collection task to a body intelligent device, the data collection task being used for instructing the body intelligent device to execute a target action and collect sensing data generated in the execution process of the target action; in response to the uploading of the sensing data to a cloud storage space, acquiring the sensing data from the cloud storage space and preprocessing and labeling the sensing data to obtain a data set. It can be seen that the method connects the configuration, issuance, execution, uploading, processing and labeling of the data collection task into an automatic process online, the sensing data does not need to be migrated through a physical medium such as a hard disk in the whole link, the risk of data loss caused by hard disk damage is effectively avoided, the data circulation period is shortened, the manual intervention link is reduced, and therefore the data production efficiency and data quality are improved.

Description

Technical Field

[0001] This application relates to the field of data management technology, and in particular to a data management method, a data management system, and an electronic device. Background Technology

[0002] A dataset is a standardized collection of raw collected data that has been cleaned, aligned, and labeled, and can be applied to scenarios such as model training. In practical applications, the quality of the dataset directly affects the performance and generalization ability of the trained model. Therefore, the integrity, consistency, and traceability of the data are crucial throughout the entire production process, from data collection to final delivery.

[0003] In traditional solutions, data typically relies on physical media such as hard drives for migration across different stages. From the acquisition end to the delivery end, manual copying, transportation, and importing of data are required at each stage. This not only involves a large investment of manpower and a long turnaround time, but also makes hard drives susceptible to physical damage during frequent insertion and removal and transfer between devices, leading to data loss or corruption. This severely restricts data production efficiency and the final data quality. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a data management method, a data management system, and an electronic device to establish a process-oriented management of data and prevent data loss or damage.

[0005] In a first aspect, embodiments of this application provide a data management method applied to a data platform. The method includes: acquiring a data acquisition task; sending the data acquisition task to an embodied intelligent device, wherein the data acquisition task is used to instruct the embodied intelligent device to perform a target action and to collect sensory data generated during the execution of the target action; the sensory data includes visual data, force data, pose data, tactile data and / or environmental data; in response to the sensory data being uploaded to a cloud storage space, retrieving the sensory data from the cloud storage space and preprocessing and labeling the sensory data to obtain a dataset.

[0006] In one possible implementation, the method further includes: acquiring and parsing algorithm requirements, determining data acquisition request parameters, or receiving user-specified data acquisition request parameters; wherein the data acquisition request parameters include at least one of target action, target device information, acquisition data type, acquisition frequency, acquisition duration, and acquisition triggering conditions; the target device information includes the identifier of the embodied intelligent device used to perform the data acquisition task; and generating a data acquisition task based on the data acquisition request parameters.

[0007] In one possible implementation, the data acquisition task is distributed to the embodied smart device, including: distributing the data acquisition task to the embodied smart device through an adapted communication protocol, or distributing the data acquisition task to an edge computing gateway, which is used to publish the data acquisition task to the corresponding embodied smart device.

[0008] In one possible implementation, the method further includes: outputting a data acquisition task; assigning a corresponding working terminal to the data acquisition task; the working terminal includes at least a remote operation device for controlling the embodied intelligent device.

[0009] In one possible implementation, the perceived data is preprocessed and labeled, including: identifying and removing low-quality data from the perceived data, where low-quality data refers to data that does not meet preset quality requirements; aligning visual data, force data, pose data, tactile data, and / or environmental data on the timeline for perceived data collected from the same embodied intelligent device; converting the perceived data into the data format required for model training; calling the labeling model to pre-label the perceived data; synchronously playing the visual data, force data, pose data, tactile data, and / or environmental data in the perceived data on the same timeline; and performing frame-by-frame labeling during playback.

[0010] In one possible implementation, the dataset is used to perform a model training task; the method further includes: displaying a dataset selection interface, which includes multiple datasets; determining a target dataset in response to a user's selection; displaying an algorithm selection interface, which includes multiple algorithms; determining a target algorithm in response to a user's selection; and performing a model training task on the target algorithm using the target dataset in response to a user's triggering operation on a model training control to obtain a pre-trained model.

[0011] In one possible implementation, the method further includes: displaying a training monitoring interface; obtaining training metrics during the training process, the training metrics including at least the loss function value and accuracy; and displaying the training metrics in the training monitoring interface.

[0012] In one possible implementation, the method further includes: during the training process, saving model checkpoints at a preset frequency, whereby the model checkpoints are used for recovery after training interruption and for model version tracking.

[0013] In one possible implementation, in response to a user's triggering operation on the model training control, a model training task is performed based on the target dataset and target algorithm to obtain a pre-trained model. The process further includes: evaluating and verifying the pre-trained model, which includes offline evaluation, simulation verification, and / or real device verification; after the pre-trained model passes the evaluation and verification, the pre-trained model is containerized, packaged, and deployed to the target deployment environment; the target deployment environment includes an edge computing gateway and / or an embodied smart device.

[0014] In one possible implementation, the pre-trained model is containerized, packaged, and deployed to the target deployment environment, including: converting the pre-trained model into an inference format supported by the target deployment environment; packaging the format-converted pre-trained model and its runtime dependencies into a container image; and deploying the container image to the target deployment environment via a canary release method; wherein, the canary release includes: deploying the pre-trained model to some embodied intelligent devices in the target deployment environment, and deploying the pre-trained model to all embodied intelligent devices after satisfying preset stability conditions.

[0015] In one possible implementation, the process of preprocessing and labeling the perceived data to obtain a dataset also includes: establishing a communication connection with a third-party platform through a standardized application programming interface (API) so that the perceived data can be preprocessed and labeled by the third-party platform to obtain a dataset.

[0016] In one possible implementation, the method further includes: sending the data acquisition task to the simulation platform to generate corresponding simulation data; the simulation data includes visual data, force data, pose data, tactile data and / or environmental data generated through simulation; and preprocessing and labeling the simulation data to obtain a dataset.

[0017] In one possible implementation, the method further includes: generating a dataset instance in response to a user's triggering operation on the dataset creation control; obtaining user-specified historical data in response to a user's triggering operation on the data import control and converting it into the target data format corresponding to the dataset instance; and importing the format-converted historical data into the dataset instance to obtain the dataset.

[0018] In one possible implementation, the dataset is used to be published to a data circulation platform for circulation; the method further includes: converting the dataset into a data format supported by the data circulation platform; publishing the dataset to the data circulation platform via a target communication protocol; the data circulation platform is used to perform at least one of the following operations on the dataset: asset registration, quality assessment, pricing, and circulation authorization.

[0019] In one possible implementation, the method further includes: receiving a registration request from an embodied smart device, the registration request carrying device information, the device information including at least one of device identifier, device model, communication protocol and sensor configuration information; authenticating the embodied smart device based on the device information; after successful authentication, assigning a device instance identifier to the embodied smart device, and storing the device information in a device management database after associating it with the device instance identifier.

[0020] Secondly, embodiments of this application also provide a data management method applied to an edge computing gateway. The method includes: receiving a data acquisition task from a data platform; wherein the data acquisition task is used to instruct an embodied intelligent device to perform a target action and to collect sensory data generated during the execution of the target action; the sensory data includes visual data, force data, pose data, tactile data, and / or environmental data; distributing the data acquisition task to the corresponding embodied intelligent device through an adapted communication protocol; receiving the sensory data uploaded by the embodied intelligent device; and uploading the sensory data to a cloud storage space.

[0021] Thirdly, embodiments of this application also provide a data management system, which includes a data platform, an embodied smart device, and an edge computing gateway. The data platform is communicatively connected to the embodied smart device and / or the edge computing gateway, and the edge computing gateway is communicatively connected to the embodied smart device. The data platform is configured to: acquire data collection tasks; distribute data collection tasks to embodied intelligent devices, which are used to instruct the embodied intelligent devices to perform target actions and collect sensory data generated during the execution of the target actions; the sensory data includes visual data, force data, pose data, tactile data and / or environmental data; in response to the sensory data being uploaded to the cloud storage space, the platform retrieves the sensory data from the cloud storage space and preprocesses and labels the sensory data to obtain a dataset; The edge computing gateway is configured to: receive data acquisition tasks from the data platform; wherein the data acquisition tasks are used to instruct the embodied intelligent device to perform a target action and collect the perception data generated during the execution of the target action; the perception data includes visual data, force data, pose data, tactile data and / or environmental data; distribute the data acquisition tasks to the corresponding embodied intelligent device through an adapted communication protocol; receive the perception data uploaded by the embodied intelligent device; and upload the perception data to the cloud storage space.

[0022] In one possible implementation, the data management system also includes a third-party platform and / or a data circulation platform; the data platform communicates with the third-party platform and / or the data circulation platform. The third-party platform is configured to: receive sensing data sent by the data platform; preprocess and label the sensing data to obtain a dataset; and send the dataset to the data platform. The data circulation platform is configured to receive datasets sent by the data platform and perform at least one of the following operations on the datasets: asset registration, quality assessment, pricing, and circulation authorization.

[0023] Fourthly, embodiments of this application also provide an electronic device, including: a memory for storing instructions; and a processor for calling the instructions stored in the memory to execute the methods in the first aspect, the second aspect, and their various implementations described above.

[0024] Fifthly, embodiments of this application also provide a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the methods described in the first aspect, the second aspect, and their various implementations.

[0025] Sixthly, embodiments of this application also provide a computer program product, which, when read and executed by a computer, causes the methods in the first aspect, the second aspect, and their various implementations to be executed.

[0026] As described above, this application provides a data management method, a data management system, and an electronic device. The method includes: acquiring a data acquisition task; distributing the data acquisition task to an embodied intelligent device, wherein the data acquisition task instructs the embodied intelligent device to perform a target action and collects perceptual data generated during the execution of the target action; the perceptual data includes visual data, force data, pose data, tactile data, and / or environmental data; in response to the perceptual data being uploaded to a cloud storage space, retrieving the perceptual data from the cloud storage space and preprocessing and labeling the perceptual data to obtain a dataset. It is evident that this method, by connecting the configuration, distribution, execution, uploading, processing, and labeling of the data acquisition task online into an automated process, eliminates the need for the perceptual data to be migrated through physical media such as hard drives throughout the entire process. This effectively avoids the risk of data loss due to hard drive failure, shortens the data flow cycle, reduces manual intervention, and thus improves data production efficiency and data quality, providing reliable and efficient data support for the model training of embodied intelligent devices. Attached Figure Description

[0027] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of a data management scenario provided in an embodiment of this application; Figure 2 A schematic diagram of the structure of the data management system provided in the embodiments of this application; Figure 3 A flowchart illustrating the first data management method provided in this application embodiment; Figure 4 A flowchart illustrating the process of generating a data acquisition task provided in an embodiment of this application; Figure 5 A flowchart illustrating the preprocessing and annotation steps provided in the embodiments of this application; Figure 6 A flowchart illustrating the model training steps provided in this application embodiment; Figure 7 A flowchart illustrating the second data management method provided in this application embodiment; Figure 8 An interactive schematic diagram of the data management system provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0029] Among them, 100-data platform; 200-embodied intelligent device; 201-end effector; 202-robotic arm; 203-mobile chassis; 204-sensor module; 205-control module; 300-edge computing gateway; 400-teleoperation device; 500-electronic device; 510-memory; 520-processor. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are all within the protection scope of this application.

[0031] In the following description, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0032] Furthermore, in this application, directional terms such as "upper," "lower," "inner," and "outer" are defined relative to the indicated placement of the components in the accompanying drawings. It should be understood that these directional terms are relative concepts, used for relative description and clarification, and can change accordingly depending on the placement of the components in the accompanying drawings.

[0033] In this embodiment, an embodied intelligent device refers to an intelligent device that possesses a physical entity (such as a robotic arm, mobile platform, end effector, or other hardware structure) and achieves perception, understanding, and interactive operation of the physical world through a neural network model. Embodied intelligent devices can autonomously understand the environmental state through multimodal perception data (such as vision, force, and position) and generate operational decisions based on an end-to-end neural network model, thereby completing complex operational tasks in unstructured environments.

[0034] To equip embodied intelligent devices with various capabilities, it is typically necessary to collect corresponding training data for each capability and then train the neural network model within the embodied intelligent device based on the collected training data. However, real-world multimodal perception data collection often faces challenges such as scattered data sources, fragmented collection processes, inconsistent data quality, difficulties in data delivery, and a lack of automated transfer mechanisms between stages.

[0035] In practical applications, data migration issues are particularly prominent. For example, data typically relies on physical media such as hard drives for cross-process migration. From the acquisition end to the delivery end, manual copying, transportation, and importing of data are required at each stage. This not only involves a large investment of manpower and a long turnaround time, but also makes hard drives susceptible to physical damage during frequent insertion and removal and cross-device transfer, leading to data loss or corruption. This severely restricts data production efficiency and final data quality, making it difficult to meet the needs of neural network models for large-scale, high-quality, and diverse training data.

[0036] Based on this, embodiments of this application provide a data management method and a data management system, which can be applied to the entire automated data flow process of data acquisition, data processing, model training and data circulation.

[0037] Figure 1 This is a schematic diagram of a data management scenario provided in an embodiment of this application.

[0038] like Figure 1 As shown, in a data management scenario, this embodiment of the application can first plan data acquisition tasks or model training plans, and assign the tasks to corresponding operators and embodied intelligent devices. The operators manipulate the embodied intelligent devices to perform actions based on the task information. During the execution of the actions, the embodied intelligent devices collect multi-source heterogeneous data such as visual data, force data, pose data, tactile data, and environmental data. Afterward, the multi-source heterogeneous data can be transferred to the backend and processed to form a standard dataset. This dataset can be directly used to train the neural network model of the embodied intelligent device, or published to a data circulation platform for data asset trading.

[0039] In practical applications, after training the neural network model using the dataset provided in the embodiments of this application, the trained model can be deployed to an embodied intelligent device. In this way, the embodied intelligent device can autonomously perform complex tasks such as grasping, placing, assembling, and transporting.

[0040] Figure 2 A schematic diagram of the structure of the data management system provided in the embodiments of this application.

[0041] like Figure 2 As shown, this application embodiment provides a data management system, which includes at least: a data platform 100, an embodied smart device 200, and an edge computing gateway 300. The data platform 100 is communicatively connected to the embodied smart device 200 and / or the edge computing gateway 300, and the edge computing gateway 300 is communicatively connected to the embodied smart device 200.

[0042] Specifically, the data platform 100 can be deployed on a cloud server and can be used to perform one or more operations among task scheduling, data reception, data processing, data storage, data annotation, data delivery, and model training task scheduling. The edge computing gateway 300 can be deployed close to the embodied intelligent device 200 for communication with it. The edge computing gateway 300 has protocol parsing and task forwarding capabilities, and can adapt to the communication protocols used by different brands and models of embodied intelligent devices 200. Through the relay of the edge computing gateway 300, unified scheduling and distribution of data acquisition tasks from multiple embodied intelligent devices 200 can be achieved by connecting a single gateway. Furthermore, the edge computing gateway 300 can also possess edge computing capabilities to reduce the computational burden on the data platform 100 and lower transmission latency. In practical applications, one edge computing gateway 300 can simultaneously establish communication connections with multiple embodied intelligent devices 200 to achieve centralized scheduling and data aggregation of multiple embodied intelligent devices 200.

[0043] In this embodiment, the embodied intelligent device 200 can be a single-arm robot, a dual-arm robot, a mobile robot, a humanoid robot, or a composite robot composed of any combination thereof, which is an intelligent device with a physical entity and can physically interact with the environment. This embodiment does not specifically limit this.

[0044] In this embodiment, the data platform 100 and the embodied intelligent device 200, the data platform 100 and the edge computing gateway 300, and the edge computing gateway 300 and the embodied intelligent device 200 are connected via wired and / or wireless means. For example, wired means may include Ethernet, fiber optic communication or serial communication, etc., and wireless means may include Wireless Fidelity (Wi-Fi), 4G / 5G, Bluetooth, ZigBee or Ultra Wideband (UWB), etc. This embodiment does not limit the specific communication method.

[0045] In some implementations, the data management system may also include a remote control device 400, which may include, for example, a virtual reality (VR) head-mounted display, a force feedback handle, a data glove, a joystick, a haptic feedback device, a master-slave control handle, a master-slave robotic arm, etc., for operators to remotely control the embodied intelligent device 200 to perform target actions, such as moving props.

[0046] In this embodiment, the data platform 100 can be used to manage the remote operation device 400, including registering and authenticating the remote operation device 400, monitoring its status (such as online / offline status, device power, communication latency, etc.), configuring its binding relationship with the embodied intelligent device 200, and recording operation logs, so as to ensure that the remote operation device 400 can safely and reliably access the data management system and establish a stable control link with the corresponding embodied intelligent device 200, providing a device foundation for data acquisition in remote operation mode.

[0047] In some implementations, embodiments of this application can also establish communication connections between the data platform 100 and third-party platforms through adapted communication protocols. These third-party platforms include at least one of a third-party annotation platform, model training platform, tool platform, data circulation platform, simulation platform, and upper-layer business system, to achieve cross-platform transfer of datasets, cross-platform collaboration of model training tasks, and cross-platform import and generation of simulation data. Specifically, the data platform 100 can interact with the aforementioned platforms through standardized interfaces (such as Representational State Transfer (RESTful) or Message Queuing Telemetry Transport (MQTT) interfaces). Embodiments of this application do not specifically limit the type of adapted communication protocol or platform.

[0048] Figure 3 This is a flowchart illustrating the first data management method provided in an embodiment of this application. Figure 3As shown, this application embodiment provides a data management method applied to a data platform 100, namely... Figure 3 The method shown is executed by data platform 100, and includes: S100: Data acquisition task.

[0049] Specifically, the data acquisition task refers to a set of information used to instruct the embodied intelligent device 200 to perform a target action and collect relevant data. Furthermore, the data acquisition task may carry data acquisition request parameters, which may include at least one of the following: target action, target device information, data type, acquisition frequency, acquisition duration, and acquisition triggering conditions.

[0050] Specifically, the target action can refer to the action performed by the various actuators of the embodied intelligent device 200. In practical applications, the embodied intelligent device 200 typically includes multiple actuators, such as an end effector 201, a robotic arm 202, and a mobile chassis 203. Correspondingly, the target action can include a combination of one or more of the following: end effector action, robotic arm action, and / or mobile chassis action.

[0051] An end effector 201 refers to the actuator in an embodied intelligent device 200 that directly contacts and interacts with the work object or environment. Examples include grippers, dexterous hands, suction cups, and tool changers. End effector actions can include at least: grasping actions, i.e., controlling the end effector 201 to close to grip or absorb a target object; releasing actions, i.e., controlling the end effector 201 to open to release the target object; and rotational actions, i.e., controlling the end effector 201 to rotate around its own axis to adjust its working posture.

[0052] The robotic arm 202 refers to the articulated boom structure in the embodied intelligent device 200 that connects to and provides motion support for the end effector 201, and may include, for example, a single arm or two arms. The robotic arm's movements may include at least: joint rotation movements, i.e., controlling at least one joint of the robotic arm 202 to rotate about its joint axis; arm extension movements, i.e., controlling the robotic arm 202 to extend outward to expand the reach of the end effector 201; arm lifting movements, i.e. controlling the robotic arm 202 to lift upward to change the position of the end effector 201 in the vertical direction; and spatial pose adjustment movements, i.e., controlling the joints of the robotic arm 202 to coordinate and adjust the position and orientation of the end effector 201 in three-dimensional space.

[0053] The mobile chassis 203 refers to the bottom motion platform in the embodied intelligent device 200 used to support the robotic arm 202 and realize overall movement. For example, it may include a wheeled chassis, a tracked chassis, or a footed chassis. The mobile chassis movements may include at least: traveling movements, i.e., controlling the mobile chassis 203 to move forward, backward, or turn on a horizontal plane; parking movements, i.e., controlling the mobile chassis 203 to stop moving and maintain its current position and posture; and lifting movements, i.e., controlling the suspension or lifting mechanism of the mobile chassis 203 to adjust the base height of the robotic arm 202.

[0054] It is understood that the target action can be any one of the aforementioned end effector actions, robotic arm actions, and mobile chassis actions, or it can be a combination of these actions. For example, in a grasping operation scenario, the target action may include the moving action of the mobile chassis 203 (moving the embodied intelligent device 200 to the vicinity of the target object), the arm extension and spatial pose adjustment action of the robotic arm 202 (aligning the end effector 201 with the target object), and the grasping action of the end effector 201 (clamping the target object), with the three working together to complete the entire grasping task. This application embodiment does not limit the specific type and combination of the target actions; those skilled in the art can flexibly configure them according to the actual operating scenario and data acquisition requirements.

[0055] Furthermore, the target device information includes an identifier for the embodied intelligent device 200 used to perform the data acquisition task. This identifier can be one or more of a device ID, device serial number, device network address, or device instance identifier, used to uniquely identify the embodied intelligent device 200 performing the data acquisition task. Further, the target device information can specifically be a target device list, which includes at least one target device identifier, with each target device identifier corresponding to one embodied intelligent device 200.

[0056] Furthermore, the data type to be collected indicates the type of perceptual data that the embodied intelligent device 200 needs to collect during the execution of the target action, such as visual data, pose data, etc. It is understood that the embodied intelligent device 200 may possess various data acquisition-related hardware such as sensor modules 204, which can be used to collect data. In this embodiment, the sensor module 204 includes, but is not limited to: visual sensors, such as red-green-blue (RGB) cameras, depth cameras, event cameras, etc., for collecting visual data; force sensors, such as force sensors, torque sensors, etc., for collecting force data; pose sensors, such as encoders, inertial measurement units (IMUs), etc., for collecting pose data; tactile sensors, such as arrayed tactile sensors, for collecting tactile data; and environmental sensors, such as light detection and ranking (LiDAR), ultrasonic sensors, etc., for collecting environmental data.

[0057] Furthermore, perceived data can include, but is not limited to, one or more of the following types: Joint angle data: Rotation angle or displacement data of each joint of the embodied intelligent device 200 collected by encoders or angle sensors, used to record the motion trajectory and posture changes of each joint of the robotic arm 202 during the execution of the target action.

[0058] Torque data: Torque data collected by torque sensors on each joint or end effector 201 of the embodied intelligent device 200 during the execution of the target action can be used to analyze the force conditions and dynamic characteristics of the robotic arm 202.

[0059] Visual data: Color image data acquired by red-green-blue color space cameras and / or depth image data acquired by depth cameras, used to record visual environment information during the execution of target actions.

[0060] Point cloud data: Three-dimensional spatial point cloud data collected by LiDAR or depth camera, used to record the three-dimensional spatial structure information of target objects, obstacles and working environment.

[0061] Force data: Contact force data between the end effector 201 and the target object or environment collected by force sensors, such as triaxial force, six-dimensional force, etc., used to record force interaction information during the execution of the target action.

[0062] Pose data: Pose data of 200 actuators of the embodied intelligent device, collected by inertial measurement unit or encoder, including spatial position coordinates, Euler angles or quaternions, etc., used to record the real-time pose of each actuator during the execution of the target action.

[0063] Tactile data: Pressure distribution, texture features and / or slip state data of the contact surface between the end effector 201 and the target object, collected by tactile sensors (e.g., array tactile sensors), used to record fine tactile information during the execution of the target action.

[0064] Environmental data: The operational environment perception data collected by environmental sensors (such as lidar, ultrasonic sensors, infrared sensors, temperature and humidity sensors, etc.) is used to record the environmental status information during the execution of target actions.

[0065] It is understood that the above-mentioned data types are merely illustrative examples. This application does not limit the specific selection and combination of sensing data. Those skilled in the art can flexibly configure one or more sensing data types according to actual application scenarios and subsequent model training requirements.

[0066] Furthermore, the acquisition frequency is used to indicate the data acquisition rate of the embodied intelligent device 200 during the execution of the target action. For example, it may include the frame rate of the visual sensor (e.g., 30 frames / second, 60 frames / second, etc.), the sampling rate of the force sensor (e.g., 500Hz, 1000Hz, etc.), the sampling rate of the pose sensor (e.g., 100Hz, 200Hz, etc.), the sampling rate of the tactile sensor (e.g., 200Hz, 500Hz, etc.), and the scanning frequency of the environmental sensor (e.g., 10Hz, 20Hz, etc.). In this application embodiment, the specific value of the acquisition frequency is not limited. It can be flexibly configured according to the execution speed of the target action, the time resolution requirements of the perceived data, and the storage and computing capabilities of the cloud.

[0067] Furthermore, the collection duration is used to instruct the embodied intelligent device 200 to perform the target action and collect perception data for a continuous period of time. For example, it may include the duration of a single collection task (such as 10 seconds, 30 seconds, 60 seconds, etc.) or the number of times the target action is repeated (such as repeating the grasping action 5 times, 10 times, etc.). In this application embodiment, the specific value of the collection duration is not limited. Those skilled in the art can flexibly configure it according to the complexity of the target action and the amount of data required.

[0068] Furthermore, the acquisition trigger conditions are used to indicate the timing of initiating data acquisition. For example, they may include: action trigger conditions, i.e., acquisition is automatically triggered when the embodied intelligent device 200 begins to perform a target action; event trigger conditions, i.e., acquisition is triggered when a preset event is detected (such as the end effector 201 contacting the target object, the contact force detected by the force sensor reaching a preset threshold, the posture sensor detecting that the robotic arm 202 has reached a preset position, etc.); and manual trigger conditions, i.e., acquisition is manually initiated by the operator through remote control commands. This application embodiment does not limit the specific type of acquisition trigger conditions; they can be flexibly configured according to the actual acquisition scenario and requirements.

[0069] Furthermore, embodiments of this application can obtain data collection tasks through user interaction via a visual interface; alternatively, embodiments of this application can generate data collection tasks based on algorithm requirements to achieve the purpose of obtaining data collection tasks; or, embodiments of this application can interact with a third-party platform or system to obtain data collection tasks. The specific steps for generating data collection tasks will be detailed below and will not be repeated here.

[0070] As can be seen, in this embodiment of the application, the data acquisition task generation steps are all completed online, without requiring users to write task scripts offline, and without requiring users to obtain device information of each of the smart devices 200 offline, thus reducing the operational costs and error risks caused by manual intervention.

[0071] S200: Distribute the data collection task to the embodied intelligent device 200.

[0072] Specifically, embodiments of this application can encapsulate data acquisition tasks to generate task configuration files that conform to the communication protocols and / or data formats supported by the target embodied intelligent device 200. For example, embodiments of this application can encapsulate the data into structured data according to a preset encapsulation format, such as JavaScript Object Notation (JSON), Protocol Buffers (Protobuf), or Extensible Markup Language (XML) format.

[0073] Furthermore, in this embodiment, the task configuration file can be distributed to the target embodied smart device 200. Alternatively, in this embodiment, the task configuration file can be distributed to the edge computing gateway 300, which then forwards the task configuration file to the target embodied smart device 200.

[0074] In some implementations, after receiving the data acquisition task (task configuration file), the embodied intelligent device 200 can return a task reception confirmation message to the data platform 100 and / or the edge computing gateway 300 so that the data platform 100 can confirm that the task has been successfully sent. If the data platform 100 does not receive the confirmation message within a preset time, the embodiments of this application can trigger a retransmission mechanism to ensure the reliability of the task sending.

[0075] Simultaneously, the embodied intelligent device 200 can also parse the received data acquisition tasks, extract task information such as data acquisition request parameters, and load them into the cache or memory of the control module 205 for subsequent execution. After parsing, the embodied intelligent device 200 can also verify the legality of the data acquisition task, such as verifying whether the task format is correct, whether the target action is within the device's executable range, and whether the acquisition frequency exceeds the device's capacity limit. If the verification fails, an error report is returned to the data platform 100.

[0076] Furthermore, the embodied intelligent device 200 can associate and store the task identifier from the received data acquisition task with its own device identifier. This allows the task identifier to be uploaded along with the sensing data during subsequent execution of the target action and data acquisition, enabling the data platform 100 to associate and trace the received sensing data with the corresponding data acquisition task. Simultaneously, the embodied intelligent device 200 can switch its current device status from "idle" to "task received" or "pending execution" based on the data acquisition task's reception status, and continuously update the status during task execution, allowing the data platform 100 to monitor the task's progress in real time.

[0077] In some implementations, the embodied smart device 200 can also synchronize its time with the data platform 100 or the edge computing gateway 300 to ensure that the timestamp of the collected sensing data is consistent with the time base of the data platform 100, thus avoiding errors in subsequent data alignment due to time deviation.

[0078] S300: In response to the uploaded sensing data to the cloud storage space, it retrieves the sensing data from the cloud storage space and preprocesses and labels the sensing data to obtain a dataset.

[0079] The cloud storage space can be at least one of Object Storage Service (OSS), a distributed file system, or a data lake architecture, used to receive and persistently store perceived data from the embodied intelligent device 200. In one implementation, the cloud storage space can be configured on a cloud server, for example, deployed on the same cloud server as the data platform 100, or deployed on a different cloud server but interconnected via an internal network or high-speed link.

[0080] To manage data within the cloud storage space, this embodiment of the application can monitor whether new sensing data is being uploaded to the cloud storage space. Specifically, the data platform 100 can acquire sensing data upload events in real time through at least one of a file system monitoring mechanism, a message queue notification mechanism, or a database trigger mechanism. For example, after the embodied smart device 200 or the edge computing gateway 300 uploads sensing data to a designated directory or bucket in the cloud storage space, the file system can generate file creation or modification events. The data platform 100 can directly know that the sensing data has been uploaded by monitoring these events, without needing additional network polling or external notifications.

[0081] In some implementations, the cloud storage space can send an upload completion notification message to the data platform 100 after the data writing is completed. The notification message may include at least one of the following: data identifier, storage path, data size, and upload timestamp, so that the data platform 100 can trigger subsequent data processing procedures based on the notification message.

[0082] In some implementations, the data platform 100 can also be configured with data-aware scripts, which periodically poll or monitor the data writing status in the cloud storage space to proactively acquire data upload events.

[0083] Furthermore, the preprocessing and annotation steps include at least one of the following: data cleaning, data alignment, data format standardization, pre-labeling, data augmentation, fine-grained annotation, and quality assessment. The preprocessing and annotation steps will be detailed below and will not be repeated here.

[0084] In summary, based on the above steps S100-S300, the entire process of data collection, uploading, preprocessing and labeling is completed online automatically. There is no need for manual data transfer through physical media such as hard drives, nor is there a need for manual triggering of the data processing flow step by step, which effectively improves the efficiency and accuracy of data management.

[0085] As can be seen from the above, this application provides a data management method applied to a data platform 100. The method includes: acquiring a data acquisition task; distributing the data acquisition task to an embodied intelligent device 200, the data acquisition task being used to instruct the embodied intelligent device 200 to perform a target action and to collect sensory data generated during the execution of the target action; the sensory data includes visual data, force data, pose data, tactile data, and / or environmental data; in response to the sensory data being uploaded to a cloud storage space, retrieving the sensory data from the cloud storage space and preprocessing and labeling the sensory data to obtain a dataset. It is evident that this method, by connecting the configuration, distribution, execution, uploading, processing, and labeling of the data acquisition task online into an automated process, eliminates the need for the sensory data to be migrated through physical media such as hard drives throughout the entire process. This effectively avoids the risk of data loss due to hard drive failure, shortens the data flow cycle, reduces manual intervention, and thus improves data production efficiency and data quality, providing reliable and efficient data support for the model training of the embodied intelligent device 200.

[0086] Figure 4 This is a flowchart illustrating the process of generating a data acquisition task provided in an embodiment of this application.

[0087] Furthermore, such as Figure 4 As shown, step S100 may include the following steps S101-S102.

[0088] S101: Obtain and parse algorithm requirements, determine data acquisition request parameters, or receive user-specified data acquisition request parameters.

[0089] Specifically, in this embodiment, a connection can also be established between the data platform 100 and the upper-layer business system through an Application Programming Interface (API), and algorithm requirements from the upper-layer business system can be received through the API. For example, the upper-layer business system can submit algorithm requirements to the data platform 100 through the API interface according to its own model training needs or business plans.

[0090] And / or, in automated data production processes, when the data analysis module or model evaluation module of the data platform 100 (which can also be an upper-layer business system and / or model training platform) detects insufficient data or discovers new long-tail scenarios, it can automatically generate algorithm requirements. In practical applications, the working environment faced by the embodied intelligent device 200 is unstructured and diverse. Some scenarios (such as operating scenarios under different lighting conditions, different degrees of occlusion, different object materials, or different environmental noise levels) occur less frequently in actual operation and belong to long-tail scenarios. Because such scenarios are difficult to be fully covered in conventional data collection processes, the performance of neural network models trained based on existing datasets degrades significantly when dealing with long-tail scenarios. The embodiments of this application can analyze the distribution of training data, identify long-tail scenario types with insufficient data coverage, and automatically generate targeted algorithm requirements to supplement the data samples for such scenarios.

[0091] Furthermore, upon receiving the algorithm requirement, this embodiment can invoke a large language model to parse the algorithm requirement and determine the data acquisition request parameters. For example, the algorithm requirement can be a semantic natural language description, such as "collecting multimodal data of a robot grasping a semi-transparent object in a low-light environment to supplement samples in the current training set that are under-lit and lack material coverage." Moreover, this embodiment can utilize a large language model to perform intent recognition and parameter extraction on this semantic description, parsing out data acquisition request parameters such as the target action (grasping), target object attributes (semi-transparent object), environmental conditions (low light), data type to be acquired (visual data, force data, pose data), and expected data volume. This transforms the unstructured algorithm requirement into structured task configuration information that can be executed by the task orchestration engine.

[0092] Furthermore, in this embodiment, a graphical user interface (GUI) can be rendered via a human-machine interface (HMI) rendering module and displayed visually on the data platform 100. The user can then access the data platform 100 and input data acquisition request parameters into the user interface. Subsequently, this embodiment can acquire the data acquisition request parameters and generate a data acquisition task.

[0093] In some implementations, the user interface (such as the first user interface) may include at least one of the following interactive elements: parameter input boxes, drop-down selection boxes, sliders, checkboxes, and button controls. For example, drop-down selection boxes can be used for users to select parameters with a fixed range of options, such as the target action type (e.g., grasping, placing, assembling, transporting, etc.), the data type of data to be acquired (e.g., visual data, force data, pose data, tactile data, environmental data, etc.), and the target embodied intelligent device identifier; parameter input boxes can be used for users to input acquisition frequency (e.g., camera frame rate 30 frames / second, force sensor sampling rate 1000Hz, etc.), acquisition duration (e.g., 10 seconds, 30 seconds, 60 seconds, etc.), and numerical parameters such as motion speed and acceleration; sliders can be used for users to intuitively adjust continuously valued parameters such as acquisition frequency or motion speed; checkboxes can be used for users to select the sensor type to be enabled (e.g., RGB camera, depth camera, force sensor, IMU, LiDAR, etc.) or the acquisition channel to be activated; button controls (e.g., "Submit," "Confirm," "Cancel," etc.) can be used for users to trigger the generation of a data acquisition task. With the graphical user interface described above, users can configure data acquisition request parameters without writing any code or scripts, simply by clicking and inputting visual interactive operations, which greatly reduces the technical threshold and operational complexity of data acquisition task configuration.

[0094] It is understood that the user interface may also include other types of interactive elements as needed, such as date pickers, file upload controls, and switch controls. This application embodiment does not limit the specific type, layout, or number of interactive elements in the user interface. By providing a visual and interactive graphical user interface, this application embodiment enables users to configure and generate data collection tasks in an intuitive and convenient manner, further reducing manual intervention and improving the efficiency of data collection task configuration and user experience.

[0095] It is understood that the specific method for acquiring data collection tasks can be any one of the above methods or a combination of the above methods. This application embodiment does not limit the specific method for acquiring data collection tasks.

[0096] S102: Generate a data acquisition task based on the data acquisition request parameters.

[0097] In this embodiment, the data acquisition task may include high-level semantic information and / or low-level execution instructions to specifically represent task information. The high-level semantic information can be used to describe the overall goal and execution logic of the data acquisition task. For example, it may include data acquisition request parameters (such as target action type, data type, acquisition frequency, acquisition duration, acquisition triggering conditions, etc.), scene description information (such as operating environment type, target object attributes, lighting conditions, etc.), prop information (such as the name, size, weight, material, etc. of the object to be operated), and task execution logic (such as action execution order, conditional branches, loop control, etc.). High-level semantic information has strong readability and interpretability, facilitating understanding and configuration by operators.

[0098] The underlying execution instructions can be used to directly drive the actuators and sensors of the embodied intelligent device 200. For example, they may include at least one of the following: motion control instructions, sensor control instructions, input / output (I / O) control instructions, trigger control instructions, and data feedback configuration. Specifically, motion control instructions may include joint angle sequences, end effector pose parameters, motion speed, and acceleration, used to drive the embodied intelligent device 200 to complete the target action; sensor control instructions may include sensor activation indicators, acquisition frequency, and acquisition resolution, used to control each sensor to perform data acquisition according to the configured parameters; input / output control instructions may include end effector opening / closing control and tool switching control, used to control the end effector or operating tool to perform corresponding physical operations; trigger control instructions indicate the triggering timing of data acquisition (e.g., triggering at the start of an action, triggering upon contact with a target object); and data feedback configuration indicates the data feedback method (e.g., real-time streaming or batch uploading).

[0099] In some implementations, embodiments of this application can utilize a Large Language Model (LLM) to perform semantic understanding and task parsing of high-level semantic information, automatically converting the high-level semantic information into low-level execution instructions. Specifically, the data platform 100 can input high-level semantic information into the LLM, which then parses the high-level semantic information, identifies key elements such as target actions, scene conditions, and execution logic, and automatically generates corresponding low-level execution instruction sequences based on preset instruction generation templates or rules. Through the semantic understanding capabilities of the LLM, automatic conversion from natural language or structured semantic information to low-level control instructions can be achieved without manually writing low-level code, significantly reducing the complexity and threshold of task configuration.

[0100] In other implementations, embodiments of this application can utilize the conversion capabilities of a task orchestration engine to convert high-level semantic information into low-level execution instructions, enabling the embodied intelligent device 200 to directly execute target actions and collect sensory data based on these low-level execution instructions. The task orchestration engine has a built-in task parsing and instruction generation module, which can automatically break down the target action type, execution logic, and acquisition configuration from the high-level semantic information into multiple sub-tasks and generate a corresponding sequence of low-level execution instructions for each sub-task. Simultaneously, the task orchestration engine can also adapt and generate low-level execution instructions that conform to the communication protocol and data format of the target embodied intelligent device 200 based on its model and capability information, ensuring compatibility of instructions across different devices. Through the conversion capabilities of the task orchestration engine, embodiments of this application achieve automated conversion from high-level semantics to low-level execution instructions, eliminating the need for manual configuration of low-level instructions for each data acquisition task.

[0101] In some implementations, embodiments of this application can also use the edge computing gateway 300 to call a large language model to perform the conversion from high-level semantic information to low-level execution instructions. Specifically, the data platform 100 encapsulates the data acquisition task containing high-level semantic information into a task configuration file and sends it to the edge computing gateway 300. After receiving the task configuration file containing high-level semantic information, the edge computing gateway 300 can call the API interface of the large language model deployed on the edge or the cloud-based large language model to input the high-level semantic information into the large language model. The large language model then parses the high-level semantic information and generates corresponding low-level execution instructions. The edge computing gateway 300 then sends the generated low-level execution instructions to the target embodied intelligent device 200. By calling the large language model from the edge, instruction conversion can be completed closer to the device, reducing cloud processing pressure, lowering network transmission latency, and improving the real-time performance of task delivery.

[0102] It is understood that the large language model can be deployed on the data platform 100, the edge computing gateway 300, or an independent model service node. This application embodiment does not limit the specific deployment location of the large language model.

[0103] In some other implementations, embodiments of this application may also call a large language model through the embodied intelligent device 200 to perform the conversion of high-level semantic information to low-level execution instructions, and embodiments of this application do not specifically limit this.

[0104] In some implementations, embodiments of this application can also directly obtain data collection tasks through a third-party platform. Specifically, the data platform 100 can provide an API interface for third-party platforms (such as model training platforms, data analysis platforms, data trading platforms, or upper-layer business systems) to call, so as to directly submit data collection tasks to the data platform 100 or query existing data collection tasks. Third-party platforms can transmit data collection tasks corresponding to their own business needs to the data platform 100 through the API interface. The data platform 100, in response to the API call request from the third-party platform, can include the data collection task in the task scheduling queue. Through the above methods, embodiments of this application can achieve open integration with external systems, realize external triggering and cross-platform collaboration of data collection tasks, and further expand the source channels and applicable scenarios of data collection tasks. Embodiments of this application do not limit the specific type of third-party platform or the specific implementation method of the API interface, as long as cross-platform acquisition of data collection tasks can be achieved.

[0105] In some implementations, when a data acquisition task includes both high-level semantic information and low-level execution instructions, the embodied intelligent device 200 can directly parse and execute the low-level execution instructions to drive various actuators and sensors to perform target actions and acquire data. Simultaneously, it monitors and verifies the execution process based on the high-level semantic information. For example, the embodied intelligent device 200 can determine whether the execution progress of the low-level execution instructions meets expectations based on the execution logic in the high-level semantic information, or automatically adjust the execution plan according to the exception handling strategy in the high-level semantic information when anomalies occur during execution. Through this method, even if the low-level execution instructions deviate during conversion or anomalies occur during execution, the embodied intelligent device 200 can perform self-verification and correction based on the high-level semantic information, thereby improving the fault tolerance and reliability of task execution.

[0106] Through the above-mentioned various implementation methods, the embodiments of this application can flexibly select the conversion method, deployment location and execution strategy of high-level semantic information and low-level execution instructions according to actual application scenarios and user needs, thereby taking into account the ease of use of task configuration, the accuracy of instruction generation and the reliability of task execution, and providing users with a more flexible and efficient data acquisition task configuration and distribution experience.

[0107] Furthermore, step S200 may include the following step S201: sending the data acquisition task to the embodied smart device 200 through an adapted communication protocol, or sending the data acquisition task to the edge computing gateway 300, the edge computing gateway 300 being used to publish the data acquisition task to the corresponding embodied smart device 200.

[0108] It is understood that the embodiments of this application support two delivery methods: direct delivery and forwarding via edge computing gateway 300.

[0109] In the direct delivery method, this embodiment of the application can directly deliver the data acquisition task to the embodied intelligent device 200 through an adapted communication protocol. The adapted communication protocol refers to a protocol supported by the target embodied intelligent device 200 and with which a communication connection has been established between it and the data platform 100. Examples include, but are not limited to: Message Queuing Telemetry Transmission Protocol, Open Platform Communications Unified Architecture (OPCUA) protocol, Hypertext Transfer Protocol (HTTP) / Hypertext Transfer Protocol Secure (HTTPS) protocol, WebSocket protocol, etc. In the direct delivery method, the data platform 100, based on the device identifier or device network address carried in the target device information, directly sends the encapsulated data acquisition task (e.g., task configuration file) to the corresponding embodied intelligent device 200 through the aforementioned adapted communication protocol. This method is suitable for scenarios where the embodied smart device 200 has the ability to communicate directly with the cloud, such as when the device has a built-in 4G / 5G communication module, Wi-Fi communication module, or Ethernet interface, and the network environment allows the device to establish a direct connection with the cloud data platform 100. The direct delivery method has the advantages of short link and low latency, and is suitable for data acquisition tasks with high real-time requirements or deployment scenarios with relatively simple network environments.

[0110] In the edge computing gateway forwarding method, this embodiment of the application can distribute the data acquisition task to the edge computing gateway 300. After receiving the data acquisition task, the edge computing gateway 300 publishes the data acquisition task to the corresponding embodied intelligent device 200. Specifically, the data platform 100 and the edge computing gateway 300 can establish a communication connection through standardized Internet protocols (such as MQTT, HTTP / HTTPS, WebSocket, etc.). The data platform 100 first sends the data acquisition task to the edge computing gateway 300. After receiving the data acquisition task, the edge computing gateway 300 parses the target device information (such as device identifier, device network address, etc.) carried in the data acquisition task, and forwards or publishes the data acquisition task to the corresponding embodied intelligent device 200 according to the target device information and through the communication protocol adapted to the target embodied intelligent device 200.

[0111] The edge computing gateway 300 is deployed close to the embodied intelligent device 200 (e.g., within the same local area network or on the same field edge node), and has multi-protocol adaptation and task forwarding capabilities. Specifically, the edge computing gateway 300 can convert standardized task instructions issued by the data platform 100 into a proprietary communication protocol format that the target embodied intelligent device 200 can recognize, thereby achieving unified access and task distribution across brands and models of devices. This approach is suitable for the following scenarios: the embodied intelligent device 200 does not have the ability to directly access the cloud (e.g., the device only supports local area network communication or only supports specific industrial fieldbus protocols), or the field network environment is complex and requires protocol conversion by the edge node, or the edge node needs to perform unified task scheduling and management for multiple embodied intelligent devices 200.

[0112] It is understood that the embodiments of this application can flexibly select the above two distribution methods according to the actual deployment scenario and network environment.

[0113] In some implementations, in scenarios where data is forwarded through the edge computing gateway 300, after receiving a data collection task, if the edge computing gateway 300 detects that the intelligent device 200 corresponding to the target device information is offline or busy, the edge computing gateway 300 can also cache the data collection task locally and forward it when the device returns to online or idle, ensuring that the data collection task is not lost due to network fluctuations or device offline during the distribution process, thereby improving the reliability of the distribution.

[0114] Through the above steps S201, the embodiments of this application can adapt to different network environments, different communication capabilities and different brand and model of intelligent devices 200, realize the reliable distribution of data collection tasks from the data platform 100 to the target device, without the need to develop different task distribution interfaces for different devices, reduce the development and maintenance costs of the system, and improve the flexibility and reliability of data collection task distribution.

[0115] It should be noted that the data management method provided in this application embodiment may further include the following steps S401-S403.

[0116] S401: Receives a registration request from the embodied smart device 200, the registration request carrying device information.

[0117] Specifically, before registration, the embodied smart device 200 is an "unknown device" to the data platform 100. This embodiment of the application can achieve unified management of numerous embodied smart devices 200 through the proactive registration of the embodied smart device 200.

[0118] In this embodiment, the device information may include at least one of the following: device identifier, device model, communication protocol, and sensor configuration information. The device identifier uniquely identifies a smart device 200, and the data platform 100 can use the device identifier to identify and locate the target device. The device model indicates the product model or type of the smart device 200. The device model may include device manufacturer information, product serial number, and specific model version. The device model information can be used for the following purposes: Firstly, the data platform 100 can match the corresponding device driver or communication protocol adapter plugin according to the device model, enabling devices of different brands and models to access the data platform 100 in a unified manner; secondly, the data platform 100 can obtain the default capability parameters (such as the number of joints, maximum load, working radius, end effector type, etc.) of the device model, providing a reference for configuring subsequent data acquisition tasks.

[0119] The communication protocol is used to indicate the type of communication protocol supported by the embodied smart device 200. In practical applications, different brands and models of embodied smart devices 200 may use different communication protocols. By reporting the communication protocol type, the data platform 100 or the edge computing gateway 300 can use the protocol format that the device can recognize and parse when subsequently issuing data collection tasks, ensuring that the task instructions can be correctly received and understood by the device.

[0120] Sensor configuration information indicates the types and parameters of sensors configured in the embodied intelligent device 200. The embodied intelligent device 200 is typically equipped with multiple sensors to achieve multimodal perception of the physical world. Sensor configuration information may include sensor type, number of sensors (e.g., 2 RGB cameras, 1 depth camera, 6 force sensors), sensor installation location (e.g., cameras mounted at the end of a robotic arm, force sensors mounted at various joints), sensor range (e.g., force sensor range 0-100N), and sensor accuracy (e.g., force sensor accuracy ±0.1N). By reporting sensor configuration information, the data platform 100 can understand the data acquisition capabilities of the device, and thus, when configuring data acquisition tasks, rationally allocate the types of data to be acquired and the acquisition parameters based on the actual sensor configuration of the device, avoiding issuing task instructions that exceed the device's acquisition capabilities.

[0121] In some implementations, device information may also include one or more of the following: a digital certificate issued by a trusted Certificate Authority (CA), signature information or a token generated based on a preset key, device firmware version, device computing power level (e.g., low, medium, high), device storage capacity (e.g., memory size, storage space size), device current geographical location (e.g., latitude and longitude coordinates or deployment site number), and device group information (e.g., which acquisition cluster or production line it belongs to). This information can be used as supplementary data for subsequent device management, task scheduling, and data analysis. This application does not limit the specific content of the device information; it can be flexibly configured according to actual application scenarios and management needs.

[0122] It should also be noted that, in this embodiment of the application, the registration request can be initiated by the embodied smart device 200 after power-on, or it can be initiated in batches by multiple embodied smart devices 200 within its jurisdiction by the edge computing gateway 300. This embodiment of the application does not limit the triggering method of the registration request.

[0123] S402: Authenticate the embodied smart device 200 based on device information.

[0124] Specifically, after receiving a registration request, this application embodiment can extract the device information carried in the registration request and perform legality authentication on the embodied smart device 200. The authentication method may include at least one of the following: Device Identification Authentication: In this embodiment, the device identifier carried in the registration request can be compared with a preset device whitelist. If the device identifier exists in the device whitelist, the device identifier authentication is deemed successful; if it does not exist in the device whitelist, the authentication is deemed unsuccessful, and the registration request for the device is rejected. The device whitelist can be pre-configured by the system administrator and stored in the device management database of the data platform 100 to limit the range of legitimate devices allowed to access the data platform 100.

[0125] Digital certificate authentication: The smart device 200 can also carry a digital certificate issued by a trusted certificate authority in the registration request. In this embodiment of the application, the digital certificate can be verified to check the integrity of the certificate chain, the validity period of the certificate, and whether the certificate has been revoked. If the verification is successful, the digital certificate authentication is deemed to be successful.

[0126] Key authentication: The smart device 200 can also carry signature information or token generated based on a preset key in the registration request. In this embodiment of the application, a pre-shared key or public key can be used to verify the signature information or token. If the verification is successful, the key authentication is determined to be successful.

[0127] In one implementation, the above-mentioned authentication methods can be used in combination, such as simultaneously employing device identification authentication and digital certificate authentication for dual verification to improve authentication security. When any of the above authentication methods fails, this embodiment can return a registration failure response to the embodied smart device 200 and record authentication failure log information (including the reason for failure, failure time, and requester device information, etc.) for subsequent security auditing and anomaly tracing. When all the above authentication methods pass, this embodiment can determine that the embodied smart device 200 is a legitimate device, allowing it to access the data platform 100 and perform subsequent data collection tasks.

[0128] S403: After successful authentication, assign a device instance identifier to the embodied smart device 200, and store the device information in the device management database after associating it with the device instance identifier.

[0129] Specifically, after confirming the authentication of the embodied smart device 200, this embodiment of the application can assign a unique device instance identifier to the embodied smart device 200. This device instance identifier is used to uniquely identify the current registration instance of the embodied smart device 200 in the data platform 100. It is understood that the device instance identifier is different from the device identifier in the device information. The device identifier is the inherent identity information of the device itself (such as the factory serial number), while the device instance identifier is a temporary or persistent identifier assigned by the data platform 100 for each registration, used for device management and data association within the data platform 100. By introducing the device instance identifier, when the same embodied smart device 200 is re-registered due to firmware upgrades, configuration changes, or network reconnection, the device identifier remains unchanged, but the data platform 100 can assign a new device instance identifier for each registration, thereby distinguishing the device status of different registration periods and facilitating device lifecycle management and historical record tracing.

[0130] In one implementation, the device instance identifier can be a unique number generated by the data platform 100 according to preset rules, such as a combination of registration timestamp, random number, and device type code, to ensure uniqueness within the global scope of the data platform 100. In another implementation, the device instance identifier can also be directly used as the device instance identifier, that is, no additional identifier is assigned. This application embodiment does not limit the specific generation method of the device instance identifier.

[0131] Furthermore, in this embodiment, device information can be associated with an assigned device instance identifier, and the associated information can be stored in a device management database. The device management database can be deployed in the storage of the data platform 100, or it can be deployed in an independent cloud database service (such as a relational database service (RDS) or a key-value store database, etc.) to persistently store information of all registered devices to support subsequent operations such as device query, status monitoring, task scheduling, and data tracing.

[0132] Furthermore, after storing the device information in the device management database, the data platform 100 can also initialize the device status of the embodied intelligent device 200, such as setting the device status to "registered" or "online," and recording information such as the registration timestamp and registration network address (e.g., IP address and port number). Simultaneously, the data platform 100 can return a registration success response to the embodied intelligent device 200, which carries the assigned device instance identifier, enabling the embodied intelligent device 200 to carry this device instance identifier in subsequent communications, facilitating the data platform 100's identification of the data source.

[0133] In some implementations, after the device instance identifier is allocated and stored, the data platform 100 can also automatically generate a data acquisition capability profile of the device based on the sensor configuration information in the device information, including the data types supported by the device, the upper limit of the acquisition frequency of each sensor, and the acquisition accuracy range, so that when configuring data acquisition tasks later, the actual acquisition capability of the device can be reasonably scheduled, avoiding the issuance of data acquisition tasks that exceed the device's capability range.

[0134] As can be seen, the embodiments of this application can realize the automated registration, authentication, and access management of embodied intelligent devices 200, enabling embodied intelligent devices 200 of different brands, models, and communication protocols to uniformly access the data platform 100, providing a standardized device management foundation for subsequent data collection task distribution, sensing data uploading, and device status monitoring. Simultaneously, by associating device information with device instance identifiers and storing it in the device management database, centralized management and persistent storage of device information are achieved, facilitating subsequent operations such as device traceability, data association, and task scheduling. The distinction between device instance identifiers and device identifiers supports the differentiation of the same device's status and the tracing of historical records at different registration periods. Furthermore, the automatic generation of device data collection capability profiles and the automatic matching of device drivers further reduce the complexity of device access and management.

[0135] Furthermore, the data management method provided in this application embodiment may also include the following step S202.

[0136] S202: Output data acquisition task.

[0137] Specifically, after the data platform 100 generates a data collection task and / or before or after it is sent to the embodied smart device 200, the embodiments of this application can also output relevant information about the data collection task to the operator in various ways so that the operator can know the specific content of the data collection task for confirmation, adjustment or intervention.

[0138] In one implementation, embodiments of this application can output data acquisition tasks through a visual interface. For example, in the graphical user interface (such as a second user interface) of the data platform 100, various information about the data acquisition tasks, including but not limited to the aforementioned high-level semantic information, is displayed in the form of a task list, task cards, or task details page. After logging into the data platform 100, operators can view a list of all current data acquisition tasks on the task management page. Clicking on any task will take them to a details page to view the complete configuration information and execution status of that task. Furthermore, operators can also receive data acquisition tasks through the user interface to further operate the embodied intelligent device 200. Through the output of the visual interface, operators can intuitively and comprehensively understand the specific content of the data acquisition tasks, facilitating task verification, progress tracking, and anomaly troubleshooting.

[0139] In another implementation, this embodiment of the application can also output data acquisition tasks via voice. For example, the data platform 100 can use a text-to-speech (TTS) engine to convert key information of the data acquisition task (such as "Please confirm: Device number 001 is about to execute a data capture task, with a collection duration of 30 seconds") into voice, and play it through the operator's headphones, on-site speakers, or the voice broadcast module built into the smart device 200. The voice output method is suitable for scenarios where it is inconvenient for the operator to look at the screen, such as when the operator is remotely operating the device or when their hands are occupied, allowing them to promptly learn the task content through voice.

[0140] In another implementation, this embodiment of the application can also output the data acquisition task via projection. For example, the data platform 100 can project the data acquisition task information onto a wall, floor, or workbench near the embodied smart device 200, or overlay the task information onto the operator's field of vision using an augmented reality (AR) device. Projection or AR output methods are suitable for industrial sites or laboratory scenarios. Operators can intuitively see the content of the data acquisition task that needs to be performed or is about to be performed without leaving their current workstation or changing their viewpoint. This achieves a fusion presentation of task information and the physical working environment, thereby reducing the time operators spend interrupting their work due to viewing the screen and improving work efficiency and continuity.

[0141] Through step S202, the embodiments of this application enable operators to obtain the specific content and execution status of the data acquisition task in a timely and accurate manner, which facilitates the operator to confirm, supervise and intervene in the task, enhances the human-machine collaboration capability in the data acquisition process, and further improves the transparency and controllability of the data acquisition task.

[0142] Furthermore, the data management method provided in this application embodiment may also include the following step S203.

[0143] S203: Assign a corresponding work terminal to the data acquisition task; the work terminal shall include at least a remote operation device for controlling the embodied intelligent device 200.

[0144] This application embodiment can also assign corresponding workstations to data acquisition tasks. Specifically, a workstation refers to a set of workstations or devices used by operators to perform remote data acquisition, including at least a remote control device 400 for remotely controlling the embodied intelligent device 200.

[0145] It is worth noting that the embodiments of this application can allocate the work terminal based on factors such as the target action type of the data acquisition task, the required skill level, the current workload of the operator, and the availability status of the teleoperated device 400. For example, the embodiments of this application can query the currently idle teleoperated devices 400 and their bound operators in the device management database, select the teleoperated device 400 and operator that match the data acquisition task, and establish the association between the task and the work terminal. In another implementation, the allocation of the work terminal can also be manually specified by the operator or task administrator through the graphical user interface (such as a third user interface) provided by the data platform 100, for example, by selecting the specified teleoperated device 400 or operator from the drop-down list on the task configuration page. The embodiments of this application do not limit the specific allocation method of the work terminal.

[0146] After assigning the corresponding work terminal to the data acquisition task, both the embodied intelligent device 200 and the work terminal have completed the corresponding execution preparations. Specifically, on the embodied intelligent device 200 side, it has received the data acquisition task and completed the following preparations: parsing the task configuration file, extracting target action information and acquisition configuration parameters (including at least the data acquisition request parameters); loading the corresponding action execution library or control strategy model according to the target action type; completing the initialization configuration of each sensor according to the acquisition configuration parameters (e.g., setting the camera frame rate, force sensor sampling rate, etc.), and waiting to receive control commands from the work terminal. On the work terminal side, the remote operation device 400 has completed the following preparations: completing device self-tests (e.g., communication link checks, battery level checks, sensor calibration, etc.); establishing a control link binding relationship with the corresponding embodied intelligent device 200; and waiting for the operator to begin operation.

[0147] Once both the embodied intelligent device 200 and the work terminal are ready, the operator can directly control the embodied intelligent device 200 to perform target actions via the remote operation device 400 on the work terminal, thus achieving sensory data acquisition in remote operation mode. For example, the operator can operate the remote operation device 400 (e.g., push a joystick, turn a handle, wear data gloves to make gestures, etc.). The remote operation device 400 converts the operator's actions into control commands and transmits them to the embodied intelligent device 200 in real time. After receiving the control commands, the embodied intelligent device 200 drives its various actuators (such as the robotic arm 202, end effector 201, mobile chassis 203, etc.) to perform corresponding actions in real time. During the execution of the target action, the embodied intelligent device 200 can simultaneously activate various sensors to collect sensory data such as visual data, force data, pose data, tactile data, and / or environmental data according to the acquisition parameters configured in the data acquisition task. Throughout the process, the operator can perceive the status and environmental information of the embodied smart device 200 in real time through the feedback mechanism of the remote control device 400 (such as tactile feedback provided by the force feedback handle, visual feedback provided by the VR head-mounted display, etc.), thereby achieving immersive remote control data acquisition.

[0148] In some implementations, the operating end may also include a display device, which is used to display visual data and / or device status information collected by the embodied intelligent device 200 in real time, so that the operator can adjust the control strategy in real time according to the displayed content.

[0149] It should be further noted that the embodiments of this application can also achieve interconnection between the data platform 100 and the item platform through an adapted communication protocol. For example, the data platform 100 can query the item inventory or send an item allocation request to the item platform. The item platform automatically allocates the required items according to the task requirements and returns the allocation results (such as item number and storage location) to the data platform 100. The data platform 100 can also display the item allocation status and retrieval information in a user interface (such as a fourth user interface) for operators to view and directly retrieve, without the need for additional communication or application. Through the above methods, automated linkage between the data collection task and the item preparation stage is achieved, improving task execution efficiency.

[0150] As can be seen, this embodiment of the application realizes the automatic association and allocation of data acquisition tasks with the operating terminal, and completes the pre-execution preparation work for the embodied intelligent device 200 and the operating terminal. This allows operators to directly start remote data acquisition when both the equipment and the control terminal are ready, without the need for additional manual operations such as equipment configuration, link establishment, and parameter setting. This further reduces manual intervention and improves the automation level and execution efficiency of the data acquisition process. At the same time, through the allocation and management of the operating terminal, the data platform 100 can uniformly schedule multiple operating terminals and operators to achieve concurrent remote data acquisition of multiple tasks, making full use of human and equipment resources.

[0151] In some implementations, after data collection is completed, the embodied intelligent device 200 can automatically upload the collected sensing data to the cloud storage space without manual intervention. In other implementations, the operator can manually trigger the data upload by clicking the "Upload" button in the mobile terminal application of the embodied intelligent device 200. For example, after completing a remote data collection operation, the operator can click the "Confirm Completion and Upload" button on the task details page of the mobile terminal application, and the embodied intelligent device 200 will respond to the upload command and upload the collected sensing data to the cloud storage space. This application embodiment does not limit the specific method of triggering the upload of sensing data and can be flexibly configured according to the compliance requirements and operation procedures of the actual application scenario.

[0152] In some implementations, the embodied intelligent device 200 can access cloud storage space and automatically upload sensing data. In other implementations, the embodied intelligent device 200 can upload the sensing data to the edge computing gateway 300, which in turn uploads the sensing data to the cloud storage space. This application embodiment does not specifically limit this implementation.

[0153] In some implementations, after the sensing data is successfully uploaded to the cloud storage space, the embodiments of this application can also automatically update the status of the data collection task (for example, update the task status from "collecting" or "pending upload" to "uploaded" or "completed"), and record the relevant metadata information of the data upload (such as upload timestamp, data size, upload time, etc.) for subsequent data traceability, quality assessment and task statistics.

[0154] It is worth noting that before the sensed data is uploaded to the cloud storage space, this embodiment of the application can also process the sensed data through the edge computing gateway 300 before uploading it to the cloud. Specifically, the edge computing gateway 300 can receive the raw data packets uploaded by the embodied smart device 200, and parse the raw data packets through an adapted communication protocol to extract multi-source heterogeneous sensed data. On this basis, the edge computing gateway 300 can also perform edge preprocessing on the multi-source heterogeneous sensed data, which includes at least one of format conversion, cleaning, statistical aggregation, and compression processing.

[0155] Regarding format conversion, since different brands and models of embodied smart devices 200 may use their own proprietary communication protocols and data formats, the raw data from these different sources will have format differences. The edge computing gateway 300 can perform protocol parsing and data extraction on the raw data packets according to the communication protocols used by each device, and convert the raw data of different formats into a standardized data format, such as JSON or Protobuf. Through format conversion, heterogeneous data reported by different devices is converted into a unified data format, which facilitates unified storage, management and analysis in the cloud data lake.

[0156] In terms of data cleaning, the edge computing gateway 300 can clean the parsed data, including removing outliers (such as sensor readings outside the reasonable range), filling in or removing missing values ​​(such as data points not collected by the sensor), and filtering and denoising the data to eliminate the impact of sensor noise and environmental interference on data quality. Through edge-side data cleaning, most obviously invalid or low-quality data can be removed before it is sent to the cloud, reducing the amount of invalid data occupying cloud storage space and improving the efficiency of subsequent data processing.

[0157] In terms of statistical aggregation, the Edge Computing Gateway 300 can aggregate and statistically analyze frequently collected data, such as calculating the mean, maximum, minimum, and standard deviation over a period of time, and then upload the aggregated statistical data to the cloud instead of the original high-frequency data. Through statistical aggregation, the amount of data that needs to be uploaded and stored can be significantly reduced while preserving the statistical characteristics of the data, thereby lowering transmission bandwidth consumption and cloud storage costs.

[0158] In terms of data compression, the edge computing gateway 300 can also compress data by using compression algorithms suitable for time-series data or sensor data before uploading it to the cloud, further reducing the amount of data transmitted and the storage space occupied.

[0159] Through the aforementioned edge preprocessing operations, the edge computing gateway 300 completes a series of processing tasks before data is sent to the cloud, including protocol parsing, format unification, quality cleaning, statistical aggregation, and compression optimization. On the one hand, this converts heterogeneous data from different sources into a unified standardized format before being sent to the cloud, facilitating unified management and analysis in the cloud. On the other hand, it effectively reduces the amount of data that needs to be transmitted and stored, lowering transmission bandwidth and cloud storage costs. At the same time, after data cleaning is completed at the edge, the data uploaded to the cloud is of higher quality, reducing the burden on subsequent cloud data processing stages.

[0160] Furthermore, after the data is fed into the cloud, the embodiments of this application can preprocess and label the data to obtain a high-quality dataset. The preprocessing and labeling steps are described in detail below.

[0161] Figure 5 This is a flowchart illustrating the preprocessing and annotation steps provided in the embodiments of this application.

[0162] like Figure 5 As shown, step S300 may include the following steps S301-S306.

[0163] S301: Identify and remove low-quality data from the perceived data. Low-quality data refers to data that does not meet the preset quality requirements.

[0164] In actual data acquisition, the collected sensor data may contain low-quality data due to factors such as sensor noise, environmental interference, communication packet loss, and equipment jitter. These low-quality data can interfere with subsequent data labeling and model training. Therefore, it is necessary to identify and remove them during the data preprocessing stage.

[0165] Specifically, embodiments of this application can perform quality assessment on each or multiple frames of perceived data according to preset quality evaluation standards (i.e., preset quality requirements) to determine whether they meet the preset quality requirements. The specific manifestations of low-quality data may include, but are not limited to, at least one of the following: Blurred image data: For example, blurred frames caused by rapid camera movement, inaccurate focus, or lens damage can be identified by calculating image sharpness evaluation metrics (such as Laplacian variance) and comparing them with preset sharpness thresholds.

[0166] Missing data frames: For example, data frames with missing data types or incomplete data fields due to communication packet loss or sensor acquisition failure.

[0167] Data with out-of-time sequence: such as abnormal timestamps or incorrect data frame order caused by sensor clock asynchrony or out-of-order data transmission.

[0168] Outlier data: For example, force or pose data may contain outliers that exceed the sensor's range, physical limits, or show significant jumps from previous or subsequent frames.

[0169] Data with excessive noise: For example, sensor data with a signal-to-noise ratio (SNR) lower than a preset threshold.

[0170] This application does not limit the specific identification criteria for low-quality data. In practice, the corresponding quality evaluation indicators and judgment thresholds can be flexibly configured according to the actual data source and application scenario.

[0171] S302: For the sensory data collected by the same embodied smart device, align the visual data, force data, pose data, tactile data and / or environmental data on the time axis.

[0172] Understandably, when the embodied intelligent device 200 performs a target action, it simultaneously collects multimodal perception data through multiple sensors. Due to differences in the sampling frequency, startup time, and clock source of different sensors (e.g., the frame rate of the vision sensor is 30 frames / second, the sampling rate of the force sensor is 1000 Hz, and the sampling rate of the pose sensor is 200 Hz), the data collected by each sensor are not naturally aligned in time, and there may be time offset between different types of data collected at the same time.

[0173] This application embodiment can further perform time axis alignment on multimodal sensing data from the same embodied intelligent device 200. Specifically, this application embodiment can match the timestamps of visual data, force data, pose data, tactile data, and environmental data within the same time moment or time window, based on the timestamp information carried in each sensor data packet and using a unified time reference. For sensor data with different sampling frequencies, the data platform 100 can use interpolation methods (such as linear interpolation or spline interpolation) to interpolate the time of low-frequency data, so that high-frequency data and low-frequency data have corresponding data values ​​at the same point in time.

[0174] By aligning the timeline, the embodiments of this application can ensure that all modal data at the same time can be correctly correlated, thereby providing time- and space-consistent training samples for subsequent multimodal data fusion and model training.

[0175] S303: Convert the perceived data into the data format required for model training.

[0176] It is understandable that different model training frameworks and algorithms may have different requirements for the format of input data. For example, deep learning-based vision models may require image data in tensor format as input, imitation learning models may require structured data sequences containing state-action pairs as input, and vision-language-action models (VLA) may require a joint data format containing images, text instructions, and action labels. Embodiments of this application can convert the aligned perceptual data from its original format to the standard data format required for training the target model, based on the target algorithm framework for subsequent model training. Format conversion may include, but is not limited to, the following operations: Image data is converted from its original encoding format (such as Joint Photographic Experts Group (JPEG) or Portable Network Graphics (PNG)) to the tensor format required for model input (such as TensorFlow Record (TFRecord) or Hierarchical Data Format Version 5 (HDF5)). Temporal data such as force and pose data are organized into arrays or matrices required for model input. Multimodal sensing data is packaged into a unified data structure according to model input requirements, and metadata information such as corresponding label fields, timestamp fields, and task identifier fields are added to each data sample. This embodiment of the application does not specifically limit the specific implementation of these steps.

[0177] It is understood that the embodiments of this application can automatically complete the format conversion operation according to the preset conversion rules or the conversion configuration specified by the user, without the need for manual intervention.

[0178] S304: Call the annotation model to pre-annotate the perceptual data.

[0179] After the data format conversion is completed, this embodiment of the application can call a pre-trained or preset annotation model to automatically pre-annotate the perceived data, providing preliminary labels for subsequent manual fine-grained annotation. Specifically, this embodiment of the application can input the converted perceived data into the annotation model, which will automatically identify and annotate elements such as target objects, action types, and keyframes in the perceived data. The annotation model may include, but is not limited to: deep learning-based object detection models (used to identify the location and category of target objects in images), action recognition models (used to identify the type and key stages of target actions), and pose estimation models (used to identify the poses of robotic arms and end effectors), etc.

[0180] Pre-labeling can generate initial labels, significantly reducing the workload and time required for manual labeling and improving the overall efficiency of data labeling.

[0181] S305: Synchronously play visual data, force data, pose data, tactile data and / or environmental data in the perception data and simulation data on the same timeline.

[0182] In order to achieve fine-grained frame-level annotation of multimodal sensing data, embodiments of this application can synchronously play multimodal sensing data (including visual data, force data, pose data, tactile data and / or environmental data) collected by the same smart device 200 on the same timeline.

[0183] In practical applications, embodiments of this application can provide an annotation interface, which can be displayed on the data platform 100. The annotation interface may include a video playback area, a time-series curve display area, and an annotation information input area. The video playback area is used to play visual data (RGB images, depth images, etc.), allowing annotators to intuitively see the process of the embodied intelligent device 200 performing the target action; the time-series curve display area is used to simultaneously display curves showing the changes of time-series data such as force data, pose data, and tactile data over time, allowing annotators to observe the data change trends and key event moments; the annotation information input area is used for annotators to input or select the label information for the current frame.

[0184] It should be further explained that "synchronous playback" refers to aligning and simultaneously playing perceptual data from different modalities, such as visual data, force data, pose data, tactile data, and / or environmental data, on the same timeline. When playback reaches a certain moment t, the video playback area of ​​the annotation interface can display the image at that moment, while the time-series curve display area can synchronously highlight the corresponding data points such as the force curve and pose curve at that moment, and the annotation information input area displays the existing annotation information at that moment. Through "synchronous playback," annotators can simultaneously observe multi-dimensional information such as "what the robot sees" (vision), "how much force it feels" (force), and "its position and posture" (pose) at the same point in time, thereby accurately understanding the robot's complete state at that moment and making more accurate annotation judgments.

[0185] S306: Perform frame-by-frame annotation during playback.

[0186] During the synchronized playback of data, this application embodiment can call the annotation model to perform frame-by-frame annotation, or it can use manual annotation to perform frame-by-frame annotation in order to generate high-precision annotation results.

[0187] In practical applications, the annotation content may include, but is not limited to, at least one of the following: Target object annotation: Marking the bounding box, category, and instance segmentation of objects appearing in each frame of the image; Action phase annotation: Annotating the execution phase of the target action, such as distinguishing between "approach phase", "grasping phase", "transportation phase", "release phase", etc.; Key point annotation: Annotating key points such as the joint position of the robotic arm and the pose of the end effector; Event annotation: Annotating special events (such as object slippage, sudden change in contact force, abnormal state, etc.); Semantic annotation: Annotating high-level semantic information such as scene type, object attributes, and operation intention.

[0188] During frame-by-frame annotation, annotators can pause, rewind, or fast-forward in the playback interface to accurately annotate any frame of data.

[0189] As can be seen, the embodiments of this application can realize the fully automated processing of perceptual data from low-quality removal, timeline alignment, format conversion, pre-annotation to synchronized playback and fine frame-by-frame annotation, effectively improving the efficiency and accuracy of data annotation and providing a high-quality labeled dataset for subsequent model training.

[0190] In some implementations, step S300 may also include the following step S307.

[0191] S307: Establish a communication connection with a third-party platform through a standardized application programming interface to preprocess and label the perceived data to obtain a dataset.

[0192] Specifically, the third-party platform refers to a third-party annotation platform. This embodiment of the application can establish a communication connection with the third-party annotation platform via an API interface, sending the cleaned, aligned, and format-converted perceptual data, or perceptual data obtained from cloud storage, to the third-party annotation platform. The third-party annotation platform then uses its annotation tools and personnel to complete the data annotation and returns the annotated dataset to the data platform 100. By introducing a third-party annotation platform, this embodiment of the application can reuse the annotation capabilities and human resources of professional annotation platforms, making it suitable for scenarios requiring large-scale annotation or with special requirements for annotation quality.

[0193] Furthermore, in the embodiments of this application, step S300 may also include the following steps S308-S309.

[0194] S308: Send the data acquisition task to the simulation platform to generate the corresponding simulation data; the simulation data includes visual data, force data, pose data, tactile data and / or environmental data generated through simulation.

[0195] Specifically, in this embodiment, a communication connection can be established with the simulation platform through an adapted communication protocol. The simulation platform then generates simulation data corresponding to the data acquisition task to achieve data coverage of long-tail scenarios and obtain sufficient training data. After receiving the data acquisition task, the simulation platform can construct a corresponding virtual scene in the simulation environment based on the task information, drive the robot model to perform target actions in the simulation environment, and synchronously generate multimodal simulation data through virtual sensors, including visual data (such as simulated RGB images and depth images), force data (such as simulated contact force and torque), pose data (such as simulated joint angles and end-effector poses), tactile data (such as simulated tactile pressure distribution), and environmental data (such as simulated LiDAR point clouds). After generating the simulation data, the simulation platform returns the simulation data to the data platform 100.

[0196] Understandably, simulation data can serve as an effective supplement to real-world perception data, especially suitable for generating data for long-tail scenarios (such as extreme lighting, rare objects, dangerous operations, etc.). It can quickly expand the scale of the dataset without relying on the real environment and equipment, thereby improving the generalization ability of the model.

[0197] S309: Preprocess and label the simulation data to obtain the dataset.

[0198] The steps for preprocessing and labeling the data can be referred to in steps S301-S306 above, and will not be repeated here. In some implementations, the embodiments of this application may process the simulation data by at least one of the steps S301-S306 above, and the embodiments of this application do not specifically limit this.

[0199] In some implementations, embodiments of this application can perform a unified preprocessing and / or annotation process on real perceived data and simulation data without the need to configure separate processing logic, thereby simplifying data processing operations.

[0200] And / or, the data management method provided in the embodiments of this application may further include the following steps S310-S312.

[0201] S310: In response to a user's triggering action on the dataset creation control, generate a dataset instance.

[0202] Specifically, embodiments of this application may provide a user interface (such as a fifth user interface), which can be displayed on the data platform 100. This user interface may include a dataset creation control (e.g., a "New Dataset" button). After the user clicks this control, embodiments of this application may, in response to the triggering operation, generate an empty dataset instance in the background and assign a unique dataset ID to the dataset instance for subsequent association with data and metadata.

[0203] S311: In response to the user's triggering operation on the data import control, retrieve the historical data specified by the user and convert it into the target data format corresponding to the dataset instance.

[0204] Furthermore, the user interface can also include data import controls. After the dataset instance is created, users can specify the historical data to be imported through data import controls in the user interface (such as the "Import Data" button or the file drag-and-drop area). Historical data can be existing data files stored locally by the user, historical data exported from other platforms, or publicly available datasets provided by third-party platforms. After the data platform 100 obtains the historical data specified by the user, it can perform format conversion on the historical data according to the target data format requirements corresponding to the dataset instance. Format conversion can include at least one of the following operations: data field mapping, data type conversion, data encoding conversion, and data structure reorganization, to ensure that the historical data can maintain consistency with the data format of the dataset instance.

[0205] S312: Import the converted historical data into the dataset instance to obtain the dataset.

[0206] After the format conversion is completed, this embodiment of the application imports the converted historical data into the dataset instance to form a complete dataset. After the import is completed, this embodiment of the application can also generate corresponding version metadata for the dataset, recording information such as data source (marked as "historical import"), import time, original data format and conversion rules, so as to facilitate subsequent traceability.

[0207] As can be seen, the embodiments of this application provide a flexible historical data import capability. Users do not need to re-collect data or re-label it. They can simply import historical data through the user interface and automatically convert it into a dataset version that meets the platform's format requirements. This effectively solves the technical problems of difficult migration of existing data and the need for repeated labeling, realizes the reuse of existing data assets, and reduces the cost of repeated investment in data collection.

[0208] Furthermore, step S300 may also include the following steps S313-S314.

[0209] S313: Collect the labeled perception data according to at least one dimension of task type, scene type, and modality type to obtain a dataset.

[0210] Specifically, after the data platform 100 or a third-party annotation platform completes the data annotation, this embodiment of the application can classify and aggregate the annotated perceptual data according to preset organizational dimensions (e.g., by calling a large model) to form a structured standard dataset. Task types can include different action categories such as grasping, placing, assembling, and transporting; scene types can include environmental change dimensions such as different lighting conditions, different degrees of occlusion, and different object materials; modal types can include different sensor modalities such as visual data, force data, pose data, and tactile data. By aggregating the data according to the above dimensions, a high-quality dataset with strong targeting and high coverage can be provided for subsequent model training.

[0211] S314: Record version metadata of the dataset, including at least one of data source, processing flow, annotation information and quality score, so that the dataset can be traced back to the embodied intelligent device that collected the data.

[0212] Specifically, after the dataset is generated, this embodiment can generate corresponding version metadata for each dataset and store the version metadata associated with the dataset. The data source can include the device identifier and device instance identifier of the embodied intelligent device 200 that collected the data; the processing flow can include data cleaning parameters, alignment methods, format conversion rules, and pre-labeled model information; the annotation information can include the annotator identifier, annotation time, and annotation rule version; the quality score can include quality assessment indicators such as annotation accuracy and data completeness. By recording the version metadata, this embodiment can ensure that each dataset has a complete traceability chain, that is, it can be traced step-by-step from the final training dataset back to the specific embodied intelligent device 200, the specific data collection task, and the specific processing stage, providing a reliable traceability basis for data quality auditing, model iteration analysis, and problem troubleshooting. At the same time, recording the version metadata also supports multi-version management of datasets, facilitating the selection of appropriate versions for different model training needs.

[0213] Furthermore, embodiments of this application can further manage the dataset, such as using the dataset to perform model training tasks, publishing the dataset to a distribution platform for distribution, etc., and embodiments of this application do not specifically limit this.

[0214] The model training process will be described in detail below.

[0215] Figure 6 This is a flowchart illustrating the model training steps provided in an embodiment of this application.

[0216] like Figure 6 As shown, the data management method provided in this application embodiment may further include the following steps S501-S505.

[0217] S501: Displays the dataset selection interface, which includes multiple datasets.

[0218] To enable model training, embodiments of this application may also provide a data selection interface, which can be displayed on a data platform 100 or a model training platform. The dataset selection interface may include multiple datasets so that users can select generated datasets for model training.

[0219] S502: In response to the user's selection action, determine the target dataset.

[0220] In this embodiment, the dataset selection interface may include dataset selection controls, such as checkboxes, radio buttons, or "Select" buttons for each dataset. Users can select one or more datasets as training data by clicking the corresponding controls. After the user completes the selection and clicks the "Confirm" or "Next" button, this embodiment can respond to the user's selection operation and determine the selected dataset as the target dataset for subsequent model training.

[0221] S503: Displays the algorithm selection interface, which includes a variety of algorithms.

[0222] After determining the target dataset, this embodiment of the application can display an algorithm selection interface for the user to choose the algorithm framework to be used for this model training. The algorithm selection interface can include a variety of selectable algorithms, such as Imitation Learning (IL) algorithms, Vision-Language-Action Model (VLA) algorithms, Behavior Cloning (BC) algorithms, and Reinforcement Learning (RL) algorithms. Each algorithm can display auxiliary information such as the algorithm name, description of applicable scenarios, and recommendations to help the user select a suitable algorithm based on the characteristics of the target dataset and the training objectives.

[0223] S504: In response to the user's selection operation, determine the target algorithm.

[0224] Users can select the target algorithm for this training using selection controls (such as radio buttons or drop-down lists) in the algorithm selection interface. After the user completes the selection, this embodiment of the application can, in response to the user's selection operation, determine the algorithm selected by the user as the target algorithm.

[0225] S505: In response to the user's trigger operation on the model training control, perform model training tasks on the target algorithm using the target dataset to obtain a pre-trained model.

[0226] After selecting the target dataset and target algorithm, the user can trigger model training by clicking the "Start Training" or "Submit Training Task" controls in the algorithm selection interface. In response to the user's triggering of the model training controls, the data platform 100 uses the target dataset to perform iterative training on the target algorithm, optimizing model parameters to obtain a pre-trained model. During model training, the data platform 100 can call upon background computing resources (such as a Graphics Processing Unit (GPU) cluster) to execute training tasks.

[0227] In some implementations, the data management method provided in this application embodiment may further include S506-S508.

[0228] S506: Displays the training monitoring interface.

[0229] This application embodiment can also provide a training monitoring interface, which can be displayed on the data platform 100 or the model training platform so that users can view the training progress and training status in real time.

[0230] S507: Obtain training metrics during the training process.

[0231] During model training, this embodiment can obtain training metrics from the training framework in real time. These metrics include at least the loss function value and accuracy. The loss function value measures the difference between the model's predictions and the true labels; a smaller value indicates a better fit. Accuracy measures the model's prediction accuracy on the training dataset; a higher value indicates better model performance. Besides the loss function value and accuracy, training metrics may also include the loss value on the validation set, validation set accuracy, learning rate, and training speed (e.g., samples processed per second), etc. This embodiment does not specifically limit these aspects.

[0232] S508: Displays training metrics in the training monitoring interface.

[0233] This application embodiment can display training metrics in real time on the training monitoring interface. For example, the training monitoring interface may include one or more charts to dynamically display the changing trends of loss function values ​​and accuracy with training epochs or iteration steps. This application embodiment does not specifically limit this.

[0234] In some implementations, the data management method provided in this application embodiment may further include S509.

[0235] S509: During training, save model checkpoints at a preset frequency. Model checkpoints are used for recovery after training is interrupted and for model version tracking.

[0236] As we can understand, a model checkpoint is a snapshot of the model parameters at a specific point in the training process, including information such as model weights, optimizer state, and training epochs at that moment. By periodically saving model checkpoints, when training is unexpectedly interrupted (e.g., due to computational resource failure, network timeout), training can be resumed from the most recently saved checkpoint without having to retrain from scratch, thus effectively saving training time and improving training efficiency. Simultaneously, model checkpoints also record the parameter states of the model at different stages of training, supporting model version tracking and rollback, facilitating subsequent model performance comparison and analysis.

[0237] In some implementations, step S305 may be followed by steps S510-S511.

[0238] S510: Evaluate and validate the pre-trained model, including offline evaluation, simulation validation, and / or real machine validation.

[0239] After model training is completed, in order to evaluate whether the model performance meets the expected requirements, embodiments of this application can also evaluate and verify the pre-trained model. Evaluation and verification can be performed using at least one of the following three methods: Offline evaluation: The pre-trained model is evaluated offline using a reserved test dataset (i.e., data that was not used in the training process). Quantitative metrics such as success rate, precision, and recall of the model on the test set are calculated to objectively measure the model's generalization ability.

[0240] Simulation verification: The pre-trained model is deployed to a simulation environment for virtual verification. The simulation platform simulates the task scenarios in the real environment, evaluates the model's performance in the simulation scenario, and verifies the model's execution effect in diverse scenarios.

[0241] Real-device verification: The pre-trained model is deployed on 200 real embodied intelligent devices for field verification. The performance and success rate of the model in the actual environment are recorded to verify the usability and robustness of the model in real-world scenarios.

[0242] The three evaluation methods described above can be flexibly selected or combined according to actual needs. For example, offline evaluation can be performed first, followed by simulation verification, and finally real-machine verification. This phased and gradual approach to approximating the real environment verifies the model's performance and reduces the security risks associated with direct machine verification.

[0243] S511: After the pre-trained model passes evaluation and verification, the pre-trained model is containerized, packaged, and deployed to the target deployment environment; the target deployment environment includes the edge computing gateway 300 and / or the embodied smart device 200.

[0244] Once the pre-trained model passes the evaluation and verification of S510 (e.g., offline evaluation metrics reach preset thresholds, simulation verification success rate reaches preset standards, and real-device verification execution effect meets expected requirements), this embodiment of the application can deploy the pre-trained model to the target deployment environment. Specifically, the data platform 100 can containerize and package the pre-trained model, and publish the packaged model to the edge computing gateway 300 and / or the embodied intelligent device 200.

[0245] In some implementations, step S511 may include the following steps S5111-S5113.

[0246] S5111: Convert the pre-trained model into an inference format supported by the target deployment environment.

[0247] It is understandable that different target deployment environments may support different model inference formats. For example, an edge computing gateway 300 or an embodied intelligent device 200 may have strict requirements on model size and inference speed, requiring the model to be converted to an optimized inference format. Embodiments of this application can convert the pre-trained model to the corresponding inference format according to the type and requirements of the target deployment environment. For example, the model can be converted from the native PyTorch or TensorFlow format of the training framework to the Open Neural Network Exchange (ONNX) format, and further converted to the TensorRT or OpenVINO format, etc., to improve the inference efficiency of the model at the edge or endpoint.

[0248] S5112: Package the pre-trained model after format conversion and its runtime dependencies into a container image.

[0249] Specifically, embodiments of this application can package the format-converted pre-trained model and its runtime dependencies (such as inference engine libraries, system dependencies, configuration files, etc.) into a container image (such as a Docker image). Containerization allows the model to run consistently in different deployment environments, avoiding compatibility issues caused by environmental differences and achieving standardization of model delivery.

[0250] S5113: Deploy the container image to the target deployment environment through canary release; wherein, canary release includes: deploying the pre-trained model to some of the embodied intelligent devices 200 in the target deployment environment, and deploying the pre-trained model to all embodied intelligent devices 200 after the preset stability conditions are met.

[0251] When deploying the model to the embodied intelligent devices 200, this embodiment of the application can adopt a canary release strategy. Specifically, this embodiment of the application can first deploy the pre-trained model to a portion of the embodied intelligent devices 200 in the target deployment environment (e.g., devices accounting for 10% of the total number of devices or devices in a specific area), run it in a real-world scenario for a period of time, and continuously monitor its performance. When it is confirmed that the model runs stably on these devices, its performance meets expectations, and no anomalies occur (e.g., the success rate is higher than a preset threshold, no system crashes or abnormal alarms occur, etc.), the deployment scope is gradually expanded, and finally the pre-trained model is deployed to all embodied intelligent devices 200. Through the canary release method, this embodiment of the application can effectively reduce the risks brought about by the deployment of new models. Even if the new model performs poorly in a real-world scenario, its impact is limited to a small number of devices, avoiding large-scale production accidents caused by model problems.

[0252] Through the above steps S501 to S511, the embodiments of this application realize the full-link automated management from dataset selection, algorithm selection, model training, training monitoring, model evaluation and verification to model containerization deployment, providing complete and efficient technical support for the iterative update of the model of the embodied intelligent device 200.

[0253] The following section details the steps involved in the data transfer process.

[0254] S512: Convert the dataset into a data format supported by the data circulation platform.

[0255] After the dataset is constructed, if users need to share the dataset externally or conduct commercial data transactions, this embodiment can convert the dataset into a data format supported by the data circulation platform. Specifically, this embodiment can adapt and convert the dataset's storage format, data encoding method, metadata structure, etc., according to the interface specifications and data format requirements of the data circulation platform. For example, the dataset can be converted from the Protobuf or JSON format used internally by the platform to the Parquet, Avro, or CSV formats required by the data circulation platform. Simultaneously, the annotation information and metadata information in the dataset are reorganized and encapsulated according to the standards of the data circulation platform to ensure that the dataset can be correctly identified, parsed, and used by the platform after it is published. Through the above format conversion, this embodiment achieves data format compatibility between the dataset and the data circulation platform, laying the foundation for secure data circulation and compliant sharing.

[0256] S513: Publish the dataset to the data circulation platform via the target communication protocol.

[0257] After the format conversion is complete, this embodiment of the application can publish the dataset to the data circulation platform via the target communication protocol (e.g., HTTPS, File Transfer Protocol (FTP), or a dedicated data exchange protocol). Upon receiving the dataset, the data circulation platform can perform a series of management operations on it, specifically including at least one of the following: Asset Registration: Data sets are registered as assets, each assigned a unique DataAsset ID. Basic information such as data name, data type, data volume, modality type, and collection scenario is recorded, along with data source information (e.g., the smart device identifier and task identifier for the collection task) and data rights information (e.g., ownership, usage rights, and revenue rights). This creates a complete data asset ledger for subsequent data asset management, auditing, and traceability. Quality Assessment: Automated quality assessments are conducted based on dimensions such as dataset quality score, annotation accuracy, data integrity, and data distribution diversity. Quality assessment reports are generated to provide objective data quality references for potential data requesters. Pricing: Pricing suggestions or auxiliary pricing are provided for datasets based on factors such as data volume, annotation accuracy, scenario coverage, collection difficulty, quality assessment results, and market demand. Circulation Authorization: The circulation scope and usage rights of datasets are managed. Smart contracts or Digital Rights Management (DRM) technologies are used to authorize and control data use, ensuring that datasets are accessed and used only within authorized limits, preventing unauthorized access or misuse by third parties.

[0258] In summary, the data management method provided in this application eliminates the fragmentation between subsystems in the data acquisition and management process through the interconnection and collaboration between multiple platforms, and opens up a fully automated process from data acquisition, processing, labeling, training to circulation.

[0259] It should also be noted that, for data acquisition tasks, this embodiment of the application can use "tasks" as the basic management unit, flexibly performing operations such as task allocation, scheduling, pausing, resuming, and adjustment. For example, when a task cannot continue due to equipment failure or insufficient resources, other tasks are unaffected and can continue to execute normally; when a task needs to adjust its acquisition parameters, it is not necessary to pause all tasks (or task blocks composed of some tasks), but only the task itself needs to be adjusted. This decoupled design effectively avoids the problem of all acquisition tasks stopping due to the abnormality of a single task, improves the flexibility and fault tolerance of the data acquisition process, and also makes the allocation and scheduling of acquisition tasks more refined and efficient.

[0260] As can be seen from the above, the embodiments of this application provide a data management method. This method can be applied to a data platform 100. Through standardized communication and collaborative cooperation between the data platform 100 and multiple platforms such as the embodied intelligent device 200, edge computing gateway 300, data circulation platform, simulation platform, third-party annotation platform, and prop platform, the creation, distribution, execution, uploading, processing, annotation, training, deployment, and circulation of data acquisition tasks are connected online into an end-to-end automated process. This effectively reduces manual intervention, lowers the risk of data loss or damage caused by the transfer of data through physical media such as hard drives, shortens the data transfer cycle, improves data production efficiency and final data quality, and provides an efficient, reliable, and complete data support system for the model training and iterative upgrade of the embodied intelligent device 200.

[0261] Figure 7 This is a flowchart illustrating a second data management method provided in an embodiment of this application.

[0262] like Figure 7 As shown in the embodiment of this application, a data management method is also provided. This method can be applied to an edge computing gateway 300, that is, the execution subject of this method is the edge computing gateway. This method may include the following steps S601-S604.

[0263] S601: Receives data acquisition tasks from the data platform.

[0264] The data acquisition task is used to instruct the embodied intelligent device 200 to perform a target action and to collect the sensory data generated during the execution of the target action; the sensory data includes visual data, force data, pose data, tactile data and / or environmental data. In this embodiment, the edge computing gateway 300 can specifically receive the data acquisition task issued by the data platform 100, and can cache and parse the task to prepare for subsequent task distribution.

[0265] S602: Distribute the data acquisition task to the corresponding embodied smart device 200 through the adapted communication protocol.

[0266] Specifically, the edge computing gateway 300 can distribute the data acquisition task to the corresponding embodied smart device 200 by adapting to the communication protocol supported by the target device, based on the target device information carried in the data acquisition task.

[0267] S603: Receives sensing data uploaded by the embodied smart device 200.

[0268] Specifically, the edge computing gateway 300 can receive raw data packets uploaded by the embodied smart device 200, and parse the raw data packets through an adapted communication protocol to extract multi-source heterogeneous sensing data.

[0269] S604: Upload the sensed data to the cloud storage space.

[0270] Specifically, the edge computing gateway 300 can upload the parsed perception data to the cloud storage space for subsequent data processing and model training.

[0271] Before step S604, the following step S605 may also be included: performing edge preprocessing on the perceived data, wherein edge preprocessing includes at least one of format conversion, cleaning, statistical aggregation and compression processing.

[0272] Before uploading the sensed data to the cloud, the edge computing gateway 300 can also perform local preprocessing on the sensed data at the edge, including converting data of different formats into a standardized format, removing outliers and noise, aggregating and statistically analyzing high-frequency data, and compressing the data to reduce the amount of data transmitted, reduce bandwidth consumption and storage costs.

[0273] As can be seen, the method provided in this application embodiment can achieve efficient collaboration between the data platform 100 and the edge computing gateway 300. The edge computing gateway 300, as an intermediate layer, undertakes both the distribution and dissemination of data acquisition tasks, as well as the reception, parsing, preprocessing, and uploading of perceived data. This effectively reduces the computational load and storage pressure on the data platform 100, lowers network bandwidth usage and transmission latency, and realizes a cloud-edge-device collaborative data acquisition and processing architecture.

[0274] Figure 8 This is an interactive diagram of the data management system provided in an embodiment of this application.

[0275] like Figure 8 As shown, the data management system provided in this application embodiment includes a data platform 100, an embodied smart device 200, and an edge computing gateway 300. The data platform 100 is communicatively connected to the embodied smart device 200 and / or the edge computing gateway 300, and the edge computing gateway 300 is communicatively connected to the embodied smart device 200. Data platform 100 is configured to perform the following steps: S01: Obtain data acquisition task; S02: The data acquisition task is sent to the embodied intelligent device 200. The data acquisition task is used to instruct the embodied intelligent device 200 to perform the target action and to collect the sensory data generated during the execution of the target action. The sensory data includes visual data, force data, pose data, tactile data and / or environmental data. S03: In response to the uploaded sensing data to the cloud storage space, retrieve the sensing data from the cloud storage space and preprocess and label the sensing data to obtain the dataset; The edge computing gateway 300 is configured to perform the following steps: S04: Receive a data acquisition task from the data platform; wherein, the data acquisition task is used to instruct the embodied intelligent device 200 to perform a target action and to collect the sensory data generated during the execution of the target action; the sensory data includes visual data, force data, pose data, tactile data and / or environmental data; S05: Distribute the data acquisition task to the corresponding embodied smart device 200 through the adapted communication protocol; S06: Receive sensory data uploaded by the embodied smart device 200; S07: Upload the sensed data to the cloud storage space.

[0276] Furthermore, the embodied intelligent device 200 is configured to perform the following steps: S08: Receive and parse data acquisition tasks; S09: Execute the target action and collect the perception data generated during the execution of the target action; S10: Upload the sensed data to the edge computing gateway 300 and / or cloud storage space.

[0277] In some implementations, the data management system may also include a third-party platform and / or a data circulation platform; the data platform communicates and connects with the third-party platform and / or the data circulation platform. S11: The third-party platform is configured to perform the following steps: S12: Receive sensing data sent by the data platform; S13: Preprocess and label the perceived data to obtain the dataset; S14: Send the dataset to the data platform; The data circulation platform is configured to perform the following steps S15: receiving a dataset sent by the data platform to perform at least one of the following operations on the dataset: asset registration, quality assessment, pricing, and circulation authorization.

[0278] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0279] like Figure 9 As shown in the illustration, this application also provides an electronic device 500, including a memory 510 and a processor 520. The memory 510 is used to store instructions. The processor 520 is used to call the instructions stored in the memory 510 to execute the above-described data management method.

[0280] This application also provides a computer-readable storage medium storing a computer program, which, when part or all of the computer program is executed by a computer, causes the above-described data management method to be executed.

[0281] This application also provides a computer program product, which includes a computer program that, when executed by a computer, causes the aforementioned data management method to be executed.

[0282] It should be noted that, after considering the specification and practicing the content disclosed in this application, those skilled in the art will readily conceive of other embodiments of this application. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary technical means in the art that are not disclosed in this application.

[0283] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The true scope is indicated by this application.

Claims

1. A data management method, characterized in that, Applied to a data platform, the method includes: Data acquisition tasks; The data acquisition task is sent to the embodied intelligent device. The data acquisition task is used to instruct the embodied intelligent device to perform a target action and to collect the sensory data generated during the execution of the target action. The sensory data includes visual data, force data, pose data, tactile data and / or environmental data. In response to the uploaded sensing data to the cloud storage space, the sensing data is retrieved from the cloud storage space and preprocessed and labeled to obtain a dataset.

2. The data management method according to claim 1, characterized in that, The method further includes: The algorithm requirements are acquired and parsed to determine the data acquisition request parameters, or the data acquisition request parameters specified by the user are received; wherein, the data acquisition request parameters include at least one of the target action, target device information, data acquisition type, acquisition frequency, acquisition duration, and acquisition triggering conditions; the target device information includes the identifier of the embodied intelligent device used to perform the data acquisition task; The data acquisition task is generated based on the data acquisition request parameters.

3. The data management method according to claim 1, characterized in that, The step of sending the data collection task to the smart device includes: The data acquisition task is sent to the embodied smart device through an adapted communication protocol, or the data acquisition task is sent to an edge computing gateway, which is used to publish the data acquisition task to the corresponding embodied smart device; Furthermore, the method further includes: Output the data acquisition task; Assign a corresponding working terminal to the data acquisition task; the working terminal includes at least a remote operation device for controlling the embodied intelligent device.

4. The data management method according to claim 1, characterized in that, The preprocessing and annotation of the perceived data includes: Identify and remove low-quality data from the perceived data, where low-quality data refers to data that does not meet preset quality requirements; For the sensory data collected by the same embodied intelligent device, the visual data, force data, pose data, tactile data and / or environmental data are aligned on the time axis. Convert the perceived data into the data format required for model training; The perception data is pre-labeled using a labeling model; The visual data, force data, pose data, tactile data, and / or environmental data in the sensory data are played synchronously on the same timeline; Frame-by-frame annotation is performed during playback.

5. The data management method according to claim 1, characterized in that, The dataset is used to perform model training tasks; the method further includes: Display a dataset selection interface, which includes multiple datasets; In response to the user's selection action, determine the target dataset; The display algorithm selection interface includes a variety of algorithms; In response to the user's selection, determine the target algorithm; In response to the user's trigger operation on the model training control, the target algorithm is trained using the target dataset to obtain a pre-trained model; And / or, the method further includes: Display the training monitoring interface; Acquire training metrics during the training process, wherein the training metrics include at least the loss function value and accuracy; The training metrics are displayed on the training monitoring interface; And / or, the method further includes: During training, model checkpoints are saved at a preset frequency. These checkpoints are used for recovery after training is interrupted and for tracing model versions.

6. The data management method according to claim 5, characterized in that, After responding to the user's trigger operation on the model training control, and performing the model training task based on the target dataset and the target algorithm to obtain the pre-trained model, the method further includes: The pre-trained model is evaluated and verified, including offline evaluation, simulation verification, and / or real machine verification. After the pre-trained model passes evaluation and verification, the pre-trained model is containerized, packaged, and deployed to the target deployment environment; the target deployment environment includes an edge computing gateway and / or the embodied smart device.

7. The data management method according to claim 6, characterized in that, The step of containerizing and packaging the pre-trained model and deploying it to the target deployment environment includes: The pre-trained model is converted into an inference format supported by the target deployment environment; The pre-trained model after format conversion and its runtime dependent environment are packaged into a container image; The container image is deployed to the target deployment environment via a canary release method; wherein, the canary release includes: deploying the pre-trained model to some embodied intelligent devices in the target deployment environment, and deploying the pre-trained model to all embodied intelligent devices after satisfying preset stability conditions.

8. The data management method according to any one of claims 1-7, characterized in that, The process of preprocessing and labeling the perceived data to obtain a dataset also includes: A communication connection is established with a third-party platform through a standardized application programming interface, so that the perceived data can be preprocessed and labeled by the third-party platform to obtain the dataset.

9. The data management method according to any one of claims 1-7, characterized in that, The method further includes: The data acquisition task is sent to the simulation platform to generate corresponding simulation data; the simulation data includes visual data, force data, pose data, tactile data and / or environmental data generated through simulation. The simulation data is preprocessed and labeled to obtain the dataset; And / or, the method further includes: In response to the user's triggering action on the dataset creation control, a dataset instance is generated; In response to the user's triggering operation on the data import control, the system retrieves the historical data specified by the user and converts it into the target data format corresponding to the dataset instance. Import the converted historical data into the dataset instance to obtain the dataset.

10. The data management method according to any one of claims 1-7, characterized in that, The dataset is used to be published to a data circulation platform for circulation; the method further includes: Convert the dataset into a data format supported by the data circulation platform; The dataset is published to the data circulation platform via the target communication protocol; the data circulation platform is used to perform at least one of the following operations on the dataset: asset registration, quality assessment, pricing, and circulation authorization.

11. The data management method according to any one of claims 1-7, characterized in that, The method further includes: Receive a registration request from the embodied smart device, the registration request carrying device information, the device information including at least one of device identifier, device model, communication protocol and sensor configuration information; The embodied smart device is authenticated based on the device information; After successful authentication, a device instance identifier is assigned to the embodied smart device, and the device information is associated with the device instance identifier and stored in the device management database.

12. A data management method, characterized in that, Applied to an edge computing gateway, the method includes: Receive data acquisition tasks from a data platform; wherein, the data acquisition tasks are used to instruct the embodied intelligent device to perform a target action and collect sensory data generated during the execution of the target action; the sensory data includes visual data, force data, pose data, tactile data and / or environmental data; The data acquisition task is distributed to the corresponding embodied smart device through an adapted communication protocol; Receive the sensing data uploaded by the embodied intelligent device; The sensed data is uploaded to the cloud storage space.

13. A data management system, characterized in that, The data management system includes a data platform, an embodied intelligent device, and an edge computing gateway. The data platform is communicatively connected to the embodied intelligent device and / or the edge computing gateway, and the edge computing gateway is communicatively connected to the embodied intelligent device. The data platform is configured as follows: Data acquisition tasks; The data acquisition task is sent to the embodied intelligent device. The data acquisition task is used to instruct the embodied intelligent device to perform a target action and to collect the sensory data generated during the execution of the target action. The sensory data includes visual data, force data, pose data, tactile data and / or environmental data. In response to the uploaded sensing data to the cloud storage space, the sensing data is retrieved from the cloud storage space and preprocessed and labeled to obtain a dataset; The edge computing gateway is configured as follows: Receive data acquisition tasks from a data platform; wherein, the data acquisition tasks are used to instruct the embodied intelligent device to perform a target action and collect sensory data generated during the execution of the target action; the sensory data includes visual data, force data, pose data, tactile data and / or environmental data; The data acquisition task is distributed to the corresponding embodied smart device through an adapted communication protocol; Receive the sensing data uploaded by the embodied intelligent device; The sensed data is uploaded to the cloud storage space.

14. The data management system according to claim 13, characterized in that, The data management system also includes a third-party platform and / or a data circulation platform; the data platform is communicatively connected to the third-party platform and / or the data circulation platform. The third-party platform is configured as follows: Receive the sensed data sent by the data platform; The perceived data is preprocessed and labeled to obtain the dataset; Send the dataset to the data platform; The data circulation platform is configured to receive the dataset sent by the data platform, and to perform at least one of the following operations on the dataset: asset registration, quality assessment, pricing, and circulation authorization.

15. An electronic device, characterized in that, include: Memory, used to store instructions; And a processor for invoking instructions stored in the memory to execute the method of any one of claims 1 to 12.

16. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 12.

17. A computer program product, characterized in that, When the computer reads and executes the computer program product, the method as described in any one of claims 1 to 12 is performed.

Images