A body intelligent integrated machine, system and running method

Abstract

The application provides a body intelligent integrated machine, system and operation method, and belongs to the technical field of industrial automation and robot control system, wherein the integrated machine comprises an automation system IDE module, an automation system driving module, a robot body driving module and a function supply module, wherein the function supply module is used for providing the robot body driving module with time-synchronized data acquisition function, simulation task execution function, model training function and model reasoning function according to the control commands that can be executed by the robot; the body intelligent integrated machine solves the time synchronization problem between the integrated machine and the robot, between the acquisition module and the robot and between the multi-frequency sensor data, and through precise timestamp management and multi-level time synchronization mechanism, it is ensured that the data of each module can be aligned in millisecond level in the process of sensor data acquisition, model reasoning and task execution, and the control error caused by time drift and data delay is effectively avoided.

Description

Technical Field

[0001] This invention belongs to the field of industrial automation and robot control system technology, specifically relating to an integrated intelligent machine, system and operating method. Background Technology

[0002] Current industrial automation systems generally adopt a distributed control architecture centered on a PLC (Programmable Logic Controller). Typically, automated processes and robotic operating systems are designed and deployed independently, often using customized communication protocols for interfacing and coordination. This architecture leads to high system integration complexity and long deployment cycles, especially when rapid adaptation to heterogeneous equipment or dynamic adjustment of task flows is required, highlighting the system's lack of flexibility. Furthermore, most current robot control systems are primarily local and closed-loop, lacking effective data feedback loops. This hinders continuous feedback and optimization of task status, equipment status, and perceived information, significantly limiting the system's application potential in flexible production and intelligent collaboration.

[0003] In recent years, embodied intelligence (EI) technology has begun to be gradually applied to industrial robot control systems. Its basic concept is to enhance the robot system's understanding and adaptability to complex task environments through closed-loop interaction between perception, cognition, and action execution. This technology is generally based on data-driven policy models such as reinforcement learning, using perceived data and environmental states as input to autonomously generate action planning and decisions. However, the practical application of embodied intelligence in the industrial field still faces many bottlenecks, specifically:

[0004] There are common problems such as difficulty in time synchronization, data delay and clock drift between existing industrial automation systems and robot systems, as well as between robot systems and data acquisition modules. This time inconsistency is particularly prominent in scenarios involving the fusion of data from multiple sensors with different frequencies, which significantly affects the accuracy of collaborative operations between robots and equipment.

[0005] Asynchronous sampling is a common problem in industrial field sensor data. The input data required for model inference often suffers from timing misalignment or data fluctuation. The control commands output by the model usually lack smoothing of indicators such as speed, acceleration, and torque, causing discontinuity and jitter in robot motion execution, which seriously affects the stability of task execution and control accuracy.

[0006] In industrial automation and communication systems, time synchronization technology has been widely applied to multi-device collaborative control, sensor data consistency processing, and task scheduling coordination. Currently, the mainstream time synchronization methods mainly include the IEEE 1588 Precision Time Protocol (PTP), the Network Time Protocol (NTP), and the EtherCAT (Ethernet for Control Automation Technology) distributed clock mechanism (DC). PTP can provide sub-microsecond clock alignment capabilities without relying on GPS (Global Positioning System) and is widely used for precise synchronization between industrial Ethernet, motion controllers, and embedded systems. NTP, as a relatively universal time synchronization protocol, is often used for periodic time reference calibration in non-real-time applications, while EtherCAT DC achieves microsecond-level synchronization control in a master-slave structure through master station frame broadcasting. In addition, some distributed control architectures also employ middleware-coordinated logical clock mechanisms, such as timestamp control and delay modeling in ROS (Robot Operating System) / OPC UA (OPC Unified Architecture), to achieve soft synchronization between devices. These methods have become standardized in industrial systems, but they are mostly designed for stable topologies and periodic task control scenarios, making it difficult to adapt to the highly heterogeneous and dynamically changing data interaction needs of embodied intelligent systems.

[0007] Compared to industrial systems, current mainstream embodied intelligence systems remain relatively weak in terms of time synchronization mechanisms. Most systems employ soft timestamp mechanisms based on platforms such as ROS, Habitat, and Isaac Sim, relying on timestamp records for each sensor message and fixed-frequency sampling windows to achieve coarse-grained time alignment. For example, in ROS / ROS2 systems, time synchronization is typically managed through timestamp fields within nodes, lacking high-precision alignment strategies and drift correction mechanisms. Simulation platforms such as Habitat and HomeRobot often use frame numbers as the basic unit for task time progression, without involving real-time clock modeling. Policy model training systems (such as VIMA, RoboNet, and BEHAVIOR) generally construct multimodal input sequences based on fixed time windows, without dynamic alignment or delay compensation for asynchronous sampled data. Even in industrial-grade platforms that support hardware time synchronization, such as NVIDIA Isaac Sim, data flow synchronization between simulation and physical entities still requires manual management and calibration by developers, lacking a general automated time synchronization mechanism. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this application proposes an all-in-one intelligent machine, system, and operating method.

[0009] Firstly, this application proposes an embodied intelligent all-in-one machine, comprising:

[0010] The automation system IDE module, linked to the automation system driver module, is used to receive industrial tasks provided by the process package, break down the industrial tasks into the smallest task units, convert the smallest task units into structured scheduling instructions, and send the structured scheduling instructions to the automation system driver module.

[0011] The automation system drive module is connected to both the robot body drive module and the factory automation system. It is used to convert structured scheduling instructions into control commands that the robot can execute, receive data information generated by the factory automation system, and send the control commands that the robot can execute and the data information generated by the factory automation system to the robot body drive module.

[0012] The robot body drive module is connected to various robot bodies and is used to control the corresponding robot body to perform corresponding actions according to the control commands that the robot can execute. At the same time, according to the request of the function supply module or the automation system drive module, it receives various data of the corresponding robot body in real time and sends the various data of the robot body to the function supply module or the automation system drive module.

[0013] The function supply module, connected to the robot body drive module, is used to provide the robot body drive module with time-synchronized data acquisition, simulation task execution, model training, and model inference functions according to the control commands that the robot can execute.

[0014] The functional supply module includes:

[0015] The data acquisition module is connected to both the database and the robot body drive module. It is used to collect data from various robot bodies in real time, process the collected data from various robot bodies in real time using a time synchronization method, and send the time-synchronized robot body data to the database. The time synchronization method includes: timestamp alignment, timestamp correction, time window aggregation, and delay compensation.

[0016] The model training module connects to the database and the model library respectively. It is used to call the robot body data collected in real time from the database and the data generated by simulation for model training and continuous optimization, and save the optimized model in the model library.

[0017] The model reasoning module is connected to the model library and the robot body driving module respectively. It is used to generate control commands that the robot can execute in real time based on the optimized model stored in the model library during the execution of the smallest task unit, and send the control commands that the robot can execute to the robot body driving module.

[0018] The data simulation module, connected to the robot body drive module, is used to dynamically construct a simulation environment based on real-time collected robot body data, simulate the execution process of the smallest task unit, and verify the training and optimization effects of the trained and optimized models in the simulation environment.

[0019] The timestamp alignment is performed by using hardware timestamps to align the real-time collected robot body data when the sampling frequency of the robot body's sensors is greater than a first frequency threshold, and by using software timestamps to align the real-time collected robot body data when the sampling frequency of the robot body's sensors is less than or equal to the first frequency threshold.

[0020] The hardware timestamp alignment includes:

[0021] When the data acquisition module sends an acquisition request, a start timestamp is generated using the current system time;

[0022] Based on the start timestamp, the robot's sensors record hardware timestamps at the physical layer;

[0023] The collected robot body data, along with a hardware timestamp, is sent to the data acquisition module.

[0024] The data acquisition module synchronizes data time according to the start timestamp and the hardware timestamp.

[0025] The software timestamp alignment includes:

[0026] When the data acquisition module sends an acquisition request, a start timestamp is generated using the current system time;

[0027] Based on the starting timestamp, the first timestamp correction is performed to obtain a dynamic estimate of the network round-trip time delay;

[0028] Based on the dynamic estimate of the network round-trip time delay, the receiving time of the robot body data is corrected, and the time drift of the network round-trip time delay is corrected a second time at a specified frequency to obtain the dynamic estimate of the network round-trip time delay after the second correction.

[0029] The data acquisition module synchronizes data time according to the start timestamp and the dynamic estimate of the network round-trip time delay after the second correction.

[0030] The method for correcting the robot's data reception time based on a dynamic estimate of the network round-trip time delay is as follows:

[0031] ;

[0032] in, The time of receiving the corrected robot body data. d0 represents the time it takes to receive robot body data, and d0 represents a dynamic estimate of the network round-trip time delay.

[0033] The timestamp correction includes:

[0034] The sensors of all robot bodies are synchronized in time periodically via NTP or PTP protocol. The first timestamp correction is performed for the network round-trip time delay, and the deviation coefficient of each correction is corrected a second time.

[0035] The first timestamp correction is calculated as follows:

[0036] ;

[0037] in, Let N be the measured network round-trip time delay for the i-th time, and N be the number of time synchronization attempts. The moving average estimate of the network round-trip time delay is used as the compensation reference after the first correction. That is, the dynamic estimate of the network round-trip time delay. If the fluctuation of the dynamic estimate of the network round-trip time delay is less than the preset fluctuation threshold, the robot body data collected in this round passes the first correction. If the fluctuation of the dynamic estimate of the network round-trip time delay is greater than or equal to the preset fluctuation threshold, the robot body data collected in this round is automatically discarded and the process of aligning the robot body data with the timestamp is re-executed.

[0038] The second timestamp correction is calculated as follows:

[0039] ;

[0040] Where P is the deviation coefficient for each correction. Let be the measured value of the network round-trip time delay for the i-th time. The compensation reference value after the first correction is used. If the deviation coefficient of each correction is within the set range, the robot body data collected this time will be corrected for the second time. If the deviation coefficient of the correction exceeds the set range, the compensation reference value after the first correction will be stopped from being updated further.

[0041] The time window aggregation involves statistically analyzing or integrating the continuous sampled values ​​of the sensors within the time window when the sampling frequency of the robot's sensors exceeds a first frequency threshold, in order to form an input feature within the time window; and dynamically adjusting the length of the time window based on the sensor's sampling frequency and response time.

[0042] The length of the time window is calculated as follows:

[0043] ;

[0044] Where f is the sampling frequency of the robot's sensors, k is the window coefficient, and W is the length of the time window.

[0045] The window coefficient is calculated as follows:

[0046] ;

[0047] Where τ is the response time of the robot's sensors, and f is the sampling frequency of the robot's sensors. The redundancy safety factor is ceil(), which is the ceil function.

[0048] The delay compensation includes static time compensation and dynamic time compensation;

[0049] The static time compensation refers to the static delay compensation performed on the robot's sensors before the robot's data is transmitted back, given the known network round-trip time delay.

[0050] The dynamic time compensation refers to the dynamic time compensation of the robot body data by using dynamic time warping when the difference between the sampling frequency of the first sensor and the sampling frequency of the second sensor is greater than a preset frequency threshold.

[0051] The static time compensation is calculated as follows:

[0052] ;

[0053] in, This refers to the time of data acquisition for the robot body after static time compensation. This is the measured value of the network round-trip time delay for the current round. For the time it takes to receive robot body data, This refers to the time when the robot's body data is sent.

[0054] The dynamic time compensation includes:

[0055] Based on the order of the start timestamp, time series X and Y are constructed using robot body data collected by the first sensor and robot body data collected by the second sensor.

[0056] Calculate the distance between time series X and Y at every time step, and construct a distance matrix;

[0057] The FastDTW algorithm, which employs a path constraint strategy that optimizes bandwidth, searches for the minimum path in the distance matrix.

[0058] Calculate the time compensation amount based on the time length corresponding to the shortest path;

[0059] Add time compensation to the robot body data collected by the second sensor to obtain dynamically time-compensated robot body data.

[0060] The distance matrix is ​​calculated as follows:

[0061] ;

[0062] in, The element in the i-th row and j-th column of the distance matrix is... and Distance between them This refers to the robot body data collected by the first sensor in the i-th iteration. This refers to the robot body data collected by the second sensor for the i-th time. for and Distance between them for and Distance between them for and The distance between them.

[0063] The path constraint strategy for optimizing bandwidth, and the formula for calculating the optimized bandwidth are as follows:

[0064] ;

[0065] in, The sampling frequency of the first sensor. The sampling frequency of the second sensor. For bandwidth control parameters, This represents the optimized bandwidth.

[0066] Secondly, this application proposes an embodied intelligence system, comprising:

[0067] The first aspect describes the embodied intelligent all-in-one machine, the factory automation system, and the bodies of various robots;

[0068] The factory automation system is used to receive sensor data, engine data, mechanical device data, execution equipment data and manufacturing execution system data in the factory automation, and send the received sensor data, engine data, mechanical device data, execution equipment data and manufacturing execution system data in the factory automation to the automation system drive module;

[0069] The various robot bodies are used to transmit various data of the robot bodies to the robot body drive package.

[0070] Thirdly, this application proposes a method for operating an embodied intelligent all-in-one machine, including:

[0071] The automation system IDE module receives industrial tasks provided by the process package, breaks down the industrial tasks into the smallest task units, converts the smallest task units into structured scheduling instructions, and sends the structured scheduling instructions to the automation system driver module.

[0072] The automation system drive module converts structured scheduling instructions into control commands that the robot can execute, receives data information generated by the factory automation system, and sends the control commands that the robot can execute and the data information generated by the factory automation system to the robot body drive module.

[0073] The robot body drive module controls the corresponding robot body to perform corresponding actions according to the control commands that the robot can execute. At the same time, according to the request of the function supply module or the automation system drive module, it receives various data of the corresponding robot body in real time and sends the various data of the robot body to the function supply module or the automation system drive module.

[0074] The functional supply module provides the robot body drive module with time-synchronized data acquisition, simulation task execution, model training, and model inference functions based on the control commands that the robot can execute.

[0075] A method for operating an embodied intelligent all-in-one machine further includes:

[0076] The data acquisition module collects data from multiple robot bodies in real time and processes the collected data in real time using a time synchronization method. The time-synchronized robot body data is then sent to the database. The time synchronization method includes timestamp alignment, timestamp correction, time window aggregation, and delay compensation.

[0077] The model training module retrieves real-time robot body data and simulation-generated data from the database to train and continuously optimize the model, and saves the optimized model in the model library.

[0078] During the execution of the smallest task unit, the model reasoning module generates control commands that the robot can execute in real time based on the optimized model stored in the model library, and sends the control commands that the robot can execute to the robot body drive module.

[0079] The data simulation module dynamically constructs a simulation environment based on real-time collected robot body data, simulates the execution process of the smallest task unit, and verifies the training and optimization effects of the trained and optimized models in the simulation environment.

[0080] Beneficial effects:

[0081] This application proposes an embodied intelligent all-in-one machine, system, and operating method. Through a data closed-loop design involving multiple modules and workflows, it enhances the generalization ability and robustness of the embodied intelligent all-in-one machine in complex and dynamic industrial environments, thereby achieving stable operation and continuous iterative optimization of the embodied intelligent control strategy in industrial automation scenarios. It solves the time synchronization problem between the all-in-one machine and the robot, between the acquisition module and the robot, and between multi-frequency sensor data. Through precise timestamp management and a multi-level time synchronization mechanism, it ensures that the data of each module can achieve millisecond-level time alignment during sensor data acquisition, model inference, and task execution, effectively avoiding control errors caused by time drift and data delay. Attached Figure Description

[0082] Figure 1 A schematic diagram of a smart all-in-one machine according to an embodiment of this application;

[0083] Figure 2 A flowchart of an operation method for a smart all-in-one machine according to an embodiment of this application. Detailed Implementation

[0084] The specific implementation methods of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0085] Example 1:

[0086] This embodiment proposes a smart all-in-one machine, such as... Figure 1 As shown, it includes:

[0087] The IDE (Integrated Development Environment) module of the automation system is linked with the automation system driver module. It is used to receive industrial tasks provided by the process package, break down the industrial tasks into the smallest task units, convert the smallest task units into structured scheduling instructions, and send the structured scheduling instructions to the automation system driver module.

[0088] In this embodiment, an embodied intelligent all-in-one machine is equipped with an embodied intelligent operating system (EOS), which includes: an automation system IDE module, an automation system driver module, a robot body driver module, and a function supply module. The modules work together efficiently to achieve data closure and real-time interaction between the various functional modules, and enable rapid deployment of models and adaptation to multiple robot bodies.

[0089] The automation system IDE module receives specific industrial task definitions provided by the process package, including but not limited to industrial operation processes such as handling, helium testing, tool inspection, and gluing. It defines, edits, and issues industrial task processes, then breaks down the industrial task processes into the smallest task units, converts the smallest task units into structured scheduling instructions through standardized interfaces, and sends them to the automation system driver module.

[0090] The automation system drive module is connected to both the robot body drive module and the factory automation system. It is used to convert structured scheduling instructions into control commands that the robot can execute, receive data information generated by the factory automation system, and send the control commands that the robot can execute and the data information generated by the factory automation system to the robot body drive module.

[0091] In this embodiment, the automation system drive module serves as the task execution engine. On the one hand, it connects with the robot body drive module to transform the upper-level defined task control logic into control commands that the robot can execute. On the other hand, it connects with the sensors, execution devices, and manufacturing execution system (EMS) in the factory automation system to achieve the execution coordination and information communication between the robot operation and the process equipment.

[0092] The robot body drive module is connected to various robot bodies and is used to control the corresponding robot body to perform corresponding actions according to the control commands that the robot can execute. At the same time, according to the request of the function supply module or the automation system drive module, it receives various data of the corresponding robot body in real time and sends the various data of the robot body to the function supply module or the automation system drive module.

[0093] In this embodiment, the robot body driving module receives control commands generated by the model inference module and drives the robot body to perform corresponding actions. Simultaneously, it collects operational information such as the robot body's state and environmental perception data, and sends this operational information back to the data acquisition module. The robot body driving module is also responsible for synchronously sending control commands to the data simulation module or receiving simulation feedback from the data simulation module, thereby supporting virtual-real comparison and strategy verification of control commands and constructing a closed-loop data path for simulation-real-machine collaboration.

[0094] The function supply module, connected to the robot body drive module, is used to provide the robot body drive module with time-synchronized data acquisition, simulation task execution, model training, and model inference functions according to the control commands that the robot can execute.

[0095] In this embodiment, it can be understood that the connection between the robot body drive module and the robot body can use various media or physical protocols, including but not limited to: CAN / Ethernet / RS485.

[0096] The functional supply module includes:

[0097] The data acquisition module is connected to both the database and the robot body drive module. It is used to collect data from various robot bodies in real time, process the collected data from various robot bodies in real time using a time synchronization method, and send the time-synchronized robot body data to the database. The time synchronization method includes: timestamp alignment, timestamp correction, time window aggregation, and delay compensation.

[0098] The model training module connects to the database and the model library respectively. It is used to call the robot body data collected in real time from the database and the data generated by simulation for model training and continuous optimization, and save the optimized model in the model library.

[0099] In this embodiment, it can be understood that the simulation-generated data is written to the database via the data acquisition module, and the model training module then uniformly retrieves the simulation-generated data from the database.

[0100] The model reasoning module is connected to the model library and the robot body driving module respectively. It is used to generate control commands that the robot can execute in real time based on the optimized model stored in the model library during the execution of the smallest task unit, and send the control commands that the robot can execute to the robot body driving module.

[0101] The data simulation module, connected to the robot body drive module, is used to dynamically construct a simulation environment based on real-time collected robot body data, simulate the execution process of the smallest task unit, and verify the training and optimization effects of the trained and optimized models in the simulation environment.

[0102] In this embodiment, the functional supply module mainly focuses on the unified management of four core functions: simulation, data acquisition, model training, and inference, to support the intelligent decision-making process of industrial tasks. The system defines and divides the task flow through the automation system IDE module, and the automation system driver package acts as the execution engine responsible for task issuance and status feedback. Based on this, it interacts with the robot body driver package through standardized interfaces to realize model inference execution and training optimization, forming an intelligent control closed loop in the task execution process.

[0103] The model library is used to uniformly manage the storage and retrieval of strategy models, supports model version updates, structural encapsulation, and data integration with other modules, and is an important intermediary for the system to achieve intelligent deployment and dynamic optimization.

[0104] The data acquisition module is responsible for collecting the robot's status, task execution data, and environmental perception information in real time.

[0105] The model training module trains and continuously optimizes the strategy model based on the actual data provided by the data acquisition module and the data generated by the data simulation module.

[0106] The data simulation module uses actual collected data to dynamically construct a simulation environment, simulate the execution process of industrial tasks, and verify the model training effect in the simulation environment.

[0107] The model reasoning module generates control decisions and action commands in real time during the actual task execution, driving the robot body to complete the corresponding actions.

[0108] The database is used to uniformly store multi-source data from the acquisition module, simulation module, and task execution process. This module can interface with the model training module and data simulation module, supporting continuous optimization of the strategy model and dynamic reconstruction of the virtual environment, forming the core link of the system's data-driven closed loop.

[0109] This embodiment proposes an embodied intelligent all-in-one machine, mainly comprising a process package and an embodied intelligent operating system, forming a complete intelligent control closed loop. The process package is used to define specific industrial tasks such as handling, inspection, and assembly, supporting modular editing and standardized output of tasks; the embodied intelligent operating system provides four core functions: simulation, data acquisition, model training, and inference, realizing real-time control and data closed loop for task execution. Based on virtual-physical collaboration, the system supports full-process management from task definition to model optimization, significantly improving the flexibility, adaptability, and task generalization ability of industrial operations.

[0110] This embodiment of the embodied intelligent all-in-one machine supports the decomposition of complex industrial tasks into finer-grained atomic tasks and generates structured task instruction code through modular compilation. The system completes the definition and compilation of task flows in the automation system IDE and realizes real-time scheduling and status feedback of task instructions through the automation system driver package. The automation system driver package can adaptively optimize the task path based on sensor data, equipment status, and real-time feedback, forming an efficient data closed loop in the task execution process, significantly improving the scheduling efficiency and adaptability of complex tasks.

[0111] To address the relatively weak time synchronization mechanisms in current mainstream embodied intelligent systems, this application proposes a collaborative control method for embodied intelligent all-in-one machines and robot systems based on time synchronization algorithms. This method solves the time synchronization problems between the all-in-one machine and the robot, between the data acquisition module and the robot, and between data from multi-frequency sensors. In embodied intelligent systems, there are numerous sensors with significant frequency differences, making it impossible to use a single method for time synchronization. However, using different methods for time synchronization makes it difficult to guarantee universality. Therefore, this embodiment proposes an effective time synchronization scheme. Through precise timestamp management and a multi-level time synchronization mechanism, it ensures that the data from each module can achieve millisecond-level time alignment during sensor data acquisition, model inference, and task execution, effectively avoiding control errors caused by time drift and data delay.

[0112] Time synchronization solutions include:

[0113] (1) Timestamp alignment:

[0114] In this embodiment, for high-frequency sensors, a dedicated timestamp chip is used for hardware timestamp recording, that is, hardware timestamps are used to align the real-time robot body data; for low-frequency sensors, the system records time in the data acquisition module, using kernel timestamps or application layer timestamps, that is, software timestamps are used to align the real-time robot body data.

[0115] Specifically, when the sampling frequency of the robot's sensors is greater than a first frequency threshold, hardware timestamps are used to align the real-time collected robot body data; when the sampling frequency of the robot's sensors is less than or equal to the first frequency threshold, software timestamps are used to align the real-time collected robot body data.

[0116] The hardware timestamp alignment process is as follows:

[0117] (1) When the data acquisition module sends an acquisition request, a start timestamp is generated using the current system time;

[0118] In this embodiment, the starting timestamp is obtained by the data acquisition module at the moment of sending the acquisition request, using the result read from the local system clock as the "starting timestamp" of the current period. This timestamp serves as the reference time for the entire acquisition batch and is used to construct a unified time base for the sampled data. This starting timestamp is used for alignment with the hardware timestamp recorded by the sensor. (2) According to the starting timestamp, the robot's sensors record the hardware timestamp at the physical layer; ensuring that the data transmission delay is minimized;

[0119] (3) Send the collected robot body data along with a hardware timestamp to the data acquisition module for precise alignment;

[0120] (4) The data acquisition module performs data time synchronization according to the start timestamp and the hardware timestamp.

[0121] In this embodiment, the starting timestamp is generated by the data acquisition module at the moment it sends the acquisition request, using the local system clock as a reference to mark the unified time base for the current acquisition cycle. The data acquisition module uses this starting timestamp as the starting point of the time window for this data acquisition, and combines it with the hardware timestamp returned by the sensor to calculate the relative time offset of each sampled data, thereby mapping all sensor data to a unified time axis and achieving time alignment. The software timestamp alignment process is as follows:

[0122] (1) When the data acquisition module sends an acquisition request, a start timestamp is generated using the current system time;

[0123] In this embodiment, a starting timestamp is generated at the start of each data acquisition and serves as a unified reference time point. All subsequent sensor sampling data is compared with this starting timestamp (in conjunction with a hardware timestamp) and mapped to a unified timeline for time alignment, model training, and data fusion.

[0124] (2) Perform the first timestamp correction according to the start timestamp to obtain the dynamic estimate of the network round-trip time delay;

[0125] (3) Correct the network round-trip time delay based on the dynamic estimate of the network round-trip time delay and the receiving time of the robot body data, and perform a second timestamp correction on the time drift of the network round-trip time delay at a specified frequency to obtain the dynamic estimate of the network round-trip time delay after the second correction.

[0126] (4) The data acquisition module performs data time synchronization according to the start timestamp and the dynamic estimate of the network round-trip time delay after the second correction. In this embodiment, the correction is completed through a two-stage mechanism: the first stage calculates the moving average of RTT (Round-Trip Time) for static compensation, and the second stage monitors the deviation coefficient, dynamically updates the average value and corrects the timestamp in real time. The whole process is executed at a fixed period to ensure that the delay drift is continuously tracked and eliminated.

[0127] The robot's data reception time is corrected based on network latency, and the time drift of network round-trip time delay is timestamped at a specified frequency to ensure data consistency with hardware timestamps and reduce long-term error accumulation.

[0128] The method for correcting the robot's data reception time based on a dynamic estimate of the network round-trip time delay is as follows:

[0129] ;

[0130] in, The time of receiving the corrected robot body data. d0 is the receiving time of the robot body data, and d0 is the dynamic estimate of the network round-trip time delay, which is calculated by moving average of delay samples measured multiple times and is used as the compensation amount for the current receiving time correction.

[0131] The timestamp correction includes:

[0132] The sensors of all robots are synchronized regularly via NTP or PTP protocols. The network round-trip time delay is corrected for the first time, and the deviation coefficient of each correction is corrected for the second time.

[0133] In this embodiment, the system periodically synchronizes the sensor nodes using NTP (Network Time Protocol) or PTP (Precision Time Protocol) to reduce time deviations caused by network latency and jitter.

[0134] The first timestamp correction is calculated as follows:

[0135] ;

[0136] in, Let N be the measured value of the network round-trip time delay for the i-th time. This value is dynamic, and N is the number of time synchronization attempts. The moving average estimate of the network round-trip time delay is used as the compensation reference after the first correction. That is, the dynamic estimate of the network round-trip time delay. If the fluctuation of the dynamic estimate of the network round-trip time delay d0 is less than the preset fluctuation threshold (e.g., 5%), the robot body data collected this time passes the first correction. If the fluctuation of the dynamic estimate of the network round-trip time delay is greater than or equal to the preset fluctuation threshold, it indicates that there is a significant fluctuation in the current network state. The system will determine that the data collected this time is not usable, and will automatically discard the robot body data collected in this round and re-execute the process of aligning the robot body data with the timestamp to ensure the time consistency and correction effectiveness of subsequent data.

[0137] The second timestamp correction is calculated as follows:

[0138] ;

[0139] Where P is the deviation coefficient for each correction. Let be the measured value of the network round-trip time delay for the i-th time. If the deviation coefficient of each correction is within the set range, the robot body data collected in this instance will undergo a second correction. If the deviation coefficient exceeds the set range, it indicates that the network is unstable, and the system can pause or trigger the correction. Updates are made to ensure the robustness and timeliness of the calibration process.

[0140] The time window aggregation involves statistically analyzing or integrating the continuous sampled values ​​of the sensors within the time window when the sampling frequency of the robot's sensors exceeds a first frequency threshold, in order to form an input feature within the time window; and dynamically adjusting the length of the time window based on the sensor's sampling frequency and response time.

[0141] In this embodiment, for high-frequency sensors, the system aggregates data within a short time window to ensure accurate time alignment. Here, "aggregation" refers to statistically analyzing or integrating continuous sampled values ​​from the high-frequency sensor within a time window to form a representative input feature. Common aggregation methods include: average value, extreme value, sliding weighting, and context stitching after timestamp alignment. The aim is to reduce noise interference caused by high-frequency jitter while ensuring real-time response and achieving time alignment with other asynchronous signals.

[0142] This method can be used for multimodal data alignment. The system aligns data within a short time window based on data timestamps and sensor characteristics, ensuring that data from different frequencies can be synchronously fused within the same time frame. For example, in robot grasping tasks, the system calculates the average data value within a time window when aligning data from the torque sensor and the vision sensor to reduce timing errors.

[0143] The length of the time window is calculated as follows:

[0144] ;

[0145] Where f is the sampling frequency of the robot's sensors, k is the window coefficient, and W is the length of the time window.

[0146] The sliding step size S determines the window update frequency. In this embodiment, the sliding step size S is set to be less than the length of the time window, which results in an overlapping sliding window, which is beneficial to enhance data continuity.

[0147] The response time τ serves as a constraint for window adjustment, dynamically adjusting the size of k to ensure that W does not exceed the system's acceptable response delay. The system dynamically adjusts the window coefficient k of the aggregation window based on the sensor response time τ and the current sampling frequency f, calculated as follows: Where τ is the response time of the robot's sensors, and f is the sampling frequency of the robot's sensors. The redundancy safety factor is used to compensate for sensor jitter and system scheduling delay. ceil() is the ceil function, a mathematical function used to round up floating-point numbers.

[0148] The delay compensation includes static time compensation and dynamic time compensation;

[0149] The static time compensation refers to the static delay compensation performed on the robot's sensors before the robot's data is transmitted back, given the known network round-trip time delay.

[0150] The dynamic time compensation refers to the use of dynamic time warping to dynamically compensate for the time of the robot body data when the difference between the sampling frequency of the first sensor and the sampling frequency of the second sensor is greater than a preset frequency threshold, thereby reducing the time misalignment caused by the sensor frequency difference.

[0151] The static time compensation is calculated as follows:

[0152] ;

[0153] in, This refers to the time of data acquisition for the robot body after static time compensation. This is a measurement of the network round-trip time delay for the current round. This measurement is either a constant or a known fixed delay parameter used to compensate for communication delays during the return transmission. For the time it takes to receive robot body data, This refers to the time when the robot's body data is sent.

[0154] The dynamic time compensation includes:

[0155] (1) Based on the order of the start timestamp, the robot body data collected by the first sensor and the robot body data collected by the second sensor are used to construct time series X and Y;

[0156] In this embodiment, the starting timestamp refers to the starting timestamp generated by the system when the robot body data is sent, i.e. Used for receiving time The network round-trip delay is calculated together and used as a time reference point for delay compensation.

[0157] (2) Calculate the distance between time series X and Y at each time step and construct a distance matrix;

[0158] (3) The FastDTW algorithm, which employs a path constraint strategy with optimized bandwidth, searches for the minimum path in the distance matrix;

[0159] (4) Calculate the time compensation amount based on the time length corresponding to the minimum path;

[0160] (5) Add time compensation to the robot body data collected by the second sensor to obtain the robot body data after dynamic time compensation.

[0161] In this embodiment, the FastDTW (Fast Dynamic Time Warping) algorithm with optimized bandwidth path constraint strategy is used instead of the traditional DTW (Dynamic Time Warping) algorithm. In order to improve real-time performance and reduce the computational complexity of constructing the distance matrix, a path constraint strategy is preferably introduced in the dynamic time compensation to limit the shortest path search area, thereby reducing time complexity.

[0162] The path constraint strategy is the Sakoe-Chiba region constraint, which restricts the shortest path to a band-shaped region with a bandwidth of δ on both sides of the main diagonal, and performs dynamic programming only within this region. To enhance the system adaptability of the path constraint, the following adaptive bandwidth setting method is further proposed: the bandwidth parameter δ is dynamically set according to the sampling frequency difference between the first sensor and the second sensor, specifically as follows:

[0163] ;

[0164] in, The sampling frequency of the first sensor. The sampling frequency of the second sensor. This is a bandwidth control parameter, an empirically set constant (e.g., 100) used to increase or decrease the allowable path offset range. This represents the optimized bandwidth.

[0165] Based on this adaptive bandwidth constraint, the Sakoe–Chiba path search region will automatically expand or shrink according to sensor characteristics. When frequency differences are large, the path matching space will be automatically widened to ensure alignment accuracy; when frequencies are close, the path range will be automatically narrowed to reduce computational complexity. Through this path constraint mechanism, the time complexity of dynamic time-warped path search can be reduced from O(n²) to O(n) or O(n log n), ensuring good real-time performance and engineering feasibility in high-frequency sampling and resource-constrained scenarios. This effectively reduces computational overhead and guarantees the real-time performance and stability of the system in high-frequency sampling and online compensation scenarios. The distance matrix is ​​calculated as follows:

[0166] ;

[0167] in, The element in the i-th row and j-th column of the distance matrix is... and Distance between them This refers to the robot body data collected by the first sensor in the i-th iteration. This refers to the robot body data collected by the second sensor for the i-th time. for and Distance between them for and Distance between them for and The distance between them.

[0168] In this embodiment, hardware time synchronization uses IEEE 1588 PTP for physical layer time synchronization, ensuring that data from different sensors can be time-aligned within sub-milliseconds. For software time synchronization, NTP is used for time alignment of sensors that do not support hardware time synchronization, and a software timestamp is appended during data encapsulation. The system also employs a hierarchical time synchronization algorithm, such as using Kalman filtering in a multi-sensor network to dynamically correct timestamps, reducing data misalignment caused by time drift.

[0169] This embodiment proposes an embodied intelligent all-in-one machine. The difficulty in constructing such a machine lies in its ability to collect and transmit various data from the factory automation system and the robot itself. Within this vast system, various sensors and data acquisition devices exist, with vastly different acquisition frequencies, making synchronous acquisition difficult. Furthermore, the modular configuration of the embodied intelligent all-in-one machine is highly flexible, making it difficult to implement traditional data synchronization methods in a fixed manner. Additionally, the embodied intelligent all-in-one machine needs to consider real-time performance, ensuring that data synchronization does not compromise real-time accuracy. To address these challenges, this application proposes a time synchronization algorithm where various parameters can be automatically adjusted to adapt to flexible module configurations. The data from each module can achieve millisecond-level time alignment, effectively avoiding control errors caused by time drift and data delay. This time synchronization algorithm guarantees the real-time performance of the embodied intelligent all-in-one machine, enabling accurate data acquisition.

[0170] Example 2:

[0171] This embodiment proposes an embodied intelligence system, including:

[0172] The first aspect describes the embodied intelligent all-in-one machine, the factory automation system, and the bodies of various robots;

[0173] The factory automation system is used to receive sensor data, engine data, mechanical device data, execution equipment data and manufacturing execution system data in the factory automation, and send the received sensor data, engine data, mechanical device data, execution equipment data and manufacturing execution system data in the factory automation to the automation system drive module;

[0174] The various robot bodies are used to transmit various data of the robot bodies to the robot body drive package.

[0175] Example 3:

[0176] This embodiment proposes a method for operating a smart all-in-one machine, such as... Figure 2 As shown, it includes:

[0177] Step S1: The automation system IDE module receives the industrial task provided by the process package, breaks down the industrial task into the smallest task unit, converts the smallest task unit into a structured scheduling instruction, and sends the structured scheduling instruction to the automation system driver module.

[0178] Step S2: The automation system drive module converts the structured scheduling instructions into control commands that the robot can execute, receives data information generated by the factory automation system, and sends the control commands that the robot can execute and the data information generated by the factory automation system to the robot body drive module.

[0179] Step S3: The robot body drive module controls the corresponding robot body to perform corresponding actions according to the control commands that the robot can execute. At the same time, according to the request of the function supply module or the automation system drive module, it receives various data of the corresponding robot body in real time and sends the various data of the robot body to the function supply module or the automation system drive module.

[0180] Step S4: The function supply module provides the robot body drive module with time-synchronized data acquisition, simulation task execution, model training, and model inference functions according to the control commands that the robot can execute.

[0181] A method for operating an embodied intelligent all-in-one machine further includes:

[0182] Step S4.1: The data acquisition module collects data from multiple robot bodies in real time, processes the collected data from multiple robot bodies in real time using a time synchronization method, and sends the time-synchronized robot body data to the database. The time synchronization method includes: timestamp alignment, timestamp correction, time window aggregation, and delay compensation.

[0183] Step S4.2: The model training module retrieves real-time robot body data and simulation-generated data from the database to train and continuously optimize the model, and saves the optimized model in the model library;

[0184] Step S4.3: During the execution of the smallest task unit, the model reasoning module generates control commands that the robot can execute in real time based on the optimized model stored in the model library, and sends the control commands that the robot can execute to the robot body drive module.

[0185] Step S4.4 The data simulation module dynamically constructs a simulation environment based on the real-time collected robot body data, simulates the execution process of the smallest task unit, and verifies the training and optimization effects of the trained and optimized model in the simulation environment.

[0186] Based on the aforementioned operating method of the embodied intelligent all-in-one machine, this application summarizes the following workflows, which are described in detail below:

[0187] 1. Automated system integration workflow:

[0188] (1) Task instruction issuance: The process package is imported into the automation system IDE module in a modular form. Users can edit and define the task flow in the automation system IDE module to form structured task instructions. The automation system IDE module sends the structured task instructions to the automation system driver module. The automation system driver module will decompose the industrial task, break down the high-level task into finer-grained atomic tasks, and generate structured task instruction code through modular compilation.

[0189] (2) Task Status Feedback: During task execution, the automation system drive module will transmit the real-time status data of the equipment (such as position, speed, and sensor feedback) back to the automation system IDE module to realize task status monitoring and process control. Sensor data will be structured and encapsulated before transmission and parsed after arriving at the automation system IDE module to ensure data integrity and consistency.

[0190] 2. Automated system operation workflow:

[0191] (1) Task instruction issuance: After receiving the task instructions issued by the automation system IDE module, the automation system drive module will convert these instructions into scheduling instructions for equipment control. These instructions not only include basic instructions for action execution, but also include equipment status monitoring and task path optimization information to ensure that the task can be executed efficiently in complex industrial environments. These task instructions are sent to the robot body drive package through a standard interface to achieve precise control of the robot body.

[0192] (2) Task status feedback: During task execution, the robot body drive module will transmit the real-time status of the device (such as position, speed, joint angle, and sensor feedback) back to the automation system drive module. These statuses are not only used for task status monitoring and path optimization, but are also synchronously updated to the automation system IDE module for task flow control and progress management.

[0193] (3) Factory equipment control: In addition to robot control, the automation system drive module is also responsible for forwarding the equipment control signals in the task instructions to the factory production line equipment, realizing the operation triggering and status management of the process links. For example, in the assembly process, the automation system drive module will control the start and stop of equipment such as conveyor belts, servo motors, and cylinders according to the task progress to ensure the synchronous coordination and precise execution of the task.

[0194] (4) Status data feedback: During task execution, the operating status, execution feedback and sensor data generated by various devices and robots will be fed back to the automation system drive module in real time, and further synchronized to the automation system IDE module. This data is not only used for task status monitoring, but also stored in the data module as data support for model optimization and long-term task scheduling, forming a task status closed loop.

[0195] 3. Data Acquisition Workflow (Entity):

[0196] (1) Data Acquisition Request Sending: The data acquisition module first sends a data acquisition request to the robot body drive module. This request includes key information such as the required status items, data frequency, and time synchronization requirements. To ensure data consistency and time accuracy, the system performs precise management of timestamps when sending the data acquisition request, ensuring that the sensor data is time-aligned within milliseconds. For example, in a multi-sensor data fusion task, the system performs time window alignment and time drift compensation for data sources from different sensors to ensure the consistency of data from each module.

[0197] (2) Data Acquisition and Time Synchronization: After receiving the acquisition request, the robot body drive module initializes the data acquisition process and configures the data frequency, sampling duration, and time synchronization mechanism according to the acquisition instructions. Since industrial robots are usually equipped with multimodal sensors (such as lidar, cameras, torque sensors, etc.), the sampling frequency, data transmission delay, and time stability of these sensors are different. Therefore, precise time synchronization is required during the data acquisition process to ensure the consistency of data timing and control accuracy.

[0198] (3) Data backhaul and encapsulation: During task execution, the robot body drive module receives real-time running status data from the robot body, including multimodal information such as pose, velocity, joint angle, force feedback, and visual images. This data is structured and encapsulated before transmission and timestamped to ensure accurate matching after the data is back to the acquisition module, avoiding data misalignment caused by time drift.

[0199] (4) Data caching and preprocessing: After receiving data from the robot body drive module, the data acquisition module performs structured parsing and preprocessing on the data to ensure data integrity and consistency. During this process, the system performs time alignment and data filtering on the multimodal data to ensure that the data from different sensors can be synchronized within a millisecond time window.

[0200] (5) Data storage and retrieval: Preprocessed data will be stored in the database, serving as an important data source for subsequent model training, simulation construction, and strategy optimization. The database not only supports batch data storage but also enables time-based indexing and rapid retrieval of historical task data, ensuring high-quality training data during model optimization and task scheduling. In addition, the system supports archiving and long-term analysis of data from long-term tasks, providing a reliable data foundation for task path optimization and continuous model improvement.

[0201] 4. Data Acquisition Workflow (Simulation):

[0202] (1) Simulation Acquisition Request Sending: The data acquisition module first sends a simulation acquisition request to the robot body drive module. This request includes the acquisition task type, required state items, data frequency, and simulation environment parameters. For example, in the simulation of an assembly task, the acquisition module may need to acquire end pose, joint angles, force feedback, and visual data. These data need to be reproduced with high precision in the simulation environment.

[0203] (2) Simulation Environment Construction: After receiving the simulation acquisition request, the robot body drive module forwards the relevant control parameters to the data simulation module. The data simulation module constructs a virtual task scenario based on the task type and control parameters, and initializes the virtual robot and environment model. During this process, the system performs detailed modeling of the task scenario, including object position, joint constraints, collision detection, and force feedback models, to ensure that the simulation environment can accurately reproduce the physical characteristics of real industrial tasks.

[0204] (3) Simulation Task Execution and Status Acquisition: After the data simulation module is built, it will perform virtual actions according to task instructions, such as handling, assembly, or welding. During the simulation, the system will collect multimodal data in real time, including end-effector pose, joint status, environmental response, contact force, and visual images. This data is structured and encapsulated in the simulation module and timestamped to ensure accurate alignment during data transmission. In addition, the system also supports delay compensation and time alignment of simulation data to ensure that the status data in the virtual environment is synchronized with the operating status of the real robot.

[0205] (4) Data feedback and standardization: After receiving the simulation status data returned by the simulation module, the robot body drive package will perform structured encapsulation and timestamp correction on the data to ensure that the data can be accurately matched when it is fed back to the acquisition module.

[0206] (5) Data storage: After receiving the simulation data sent by the data simulation module, the data acquisition module will perform structured parsing and preprocessing on the data to ensure its integrity and consistency. This data will then be stored in the data module as a data source for subsequent model training and simulation optimization.

[0207] 5. Model training workflow:

[0208] (1) Data Reading and Preprocessing: The model training module first reads historical task data from the database, including multimodal data such as perception information, control actions, and task feedback during robot execution. During the data reading process, the system performs timestamp correction, data completion, and outlier filtering on the sensor data to ensure that the model can receive high-quality training data. In addition, the system performs time alignment and structured encapsulation on data from different sources to ensure that multimodal data can be accurately aligned within the time window.

[0209] (2) Model Training and Parameter Optimization: After data preprocessing, the model training module extracts features and augments samples based on a unified model structure, and then performs model training. The training process includes feature extraction, data augmentation, gradient optimization, and model parameter updates to ensure that the model can achieve efficient control and stable execution in actual industrial tasks. The trained model is output through a standard interface as the basis for the inference of the model inference module, used for control decisions in actual tasks.

[0210] 6. Model Inference Workflow (Entity):

[0211] (1) Model loading and inference initialization: The model inference module first loads the corresponding policy model, including model parameters, weight files and inference configuration, and enters a standby state. After loading, the inference module initializes the inference context according to the task type and execution requirements to ensure real-time response during task execution.

[0212] (2) Real-time status feedback: During the task operation, the real-time operating status of the robot body is first transmitted back to the robot body drive package through the robot body's sensor system. The drive package performs state analysis and structured encapsulation on the raw sensor data, and extracts multimodal information such as joint angles, speeds, end effector forces, and visual features.

[0213] (3) State analysis and inference input: After receiving the state feedback from the robot body, the robot body drive module will perform multimodal feature extraction and time alignment on the original sensor data to ensure that the dynamic features of the task can be accurately captured before the model forward calculation. This is the key input stage of model inference, which directly determines the accuracy and real-time performance of action decisions.

[0214] (4) Control command generation: After receiving the structured state data, the model inference module will use this data as the forward input of the policy model and perform action decision calculation in combination with the task context. During this process, the system will perform feature fusion and trajectory optimization on the multimodal data to ensure the smoothness and stability of the control commands.

[0215] (5) Command execution: After generating low-level control signals, the robot body drive module forwards these signals to the robot body to drive the joint motors and actuators to complete the actual actions. These control signals include pose adjustment, speed control and joint coordination, ensuring that the robot can complete the specific task according to the motion path calculated by the strategy model.

[0216] 7. Model Inference Workflow (Simulation):

[0217] (1) Model loading and inference initialization: The model inference module first loads the trained policy model parameters, including model weights, network structure and optimization policy, and initializes the inference context. This is the policy model loading stage, which ensures that the decision logic of the model can be accurately reproduced before the simulation task begins.

[0218] (2) Simulation data transmission: After initialization, the data simulation module generates virtual state data according to the task instructions, such as joint angles, end poses, sensor simulation values ​​and environmental feedback. This simulation data is first transmitted to the robot body drive package, and is standardized and structured during the transmission process to ensure rapid matching in subsequent inference calculations.

[0219] (3) State Analysis and Inference Input: After receiving the virtual state data transmitted by the data simulation module, the robot body drive module performs structured encapsulation and multimodal feature extraction on this data to ensure that the dynamic features of the task can be accurately captured before the model's forward computation. In this process, the system first decodes and extracts the joint angles, velocities, end-effector forces, and visual features to ensure that the data from each sensor can be synchronously aligned on the time axis. In addition, the system performs time window alignment on the asynchronously sampled multimodal data. At the same time, in order to reduce data misalignment caused by sensor latency and network jitter, the system also performs short-term prediction and dynamic compensation on historical state data before inference to ensure the time consistency and data integrity of the model input.

[0220] (4) Control Command Generation: After receiving the structured state data, the model inference module uses this data as the forward input to the policy model and calculates action decisions in conjunction with the task context. During this process, the system performs feature fusion and trajectory optimization on the multimodal data to ensure the smoothness and stability of the control commands. Specific implementations include trajectory optimization and output smoothing, such as reducing velocity abrupt changes and path oscillations through multi-segment spline interpolation to ensure the continuity and stability of task execution. To reduce jitter in the actuator during command conversion, the system also performs velocity filtering and acceleration smoothing on the output control signal to ensure that torque abrupt changes and joint jitter can be effectively suppressed during joint control and end-effector execution.

[0221] (5) Command Execution: After receiving the control commands generated by the inference module, the robot body drive module forwards the commands to the simulation module, driving the virtual robot to perform the corresponding actions. After executing the control commands, the simulation module generates virtual task feedback data, such as end-effector pose, joint state, torque feedback, and environmental response. This feedback data is structured and calibrated in the simulation module and used to update the task state in the simulation environment, ensuring time alignment and data consistency during the simulation process.

[0222] Through the aforementioned multi-workflow data closed-loop design, the system improves the model's generalization ability and robustness in complex and dynamic industrial environments, thereby enabling the stable operation and continuous iterative optimization of embodied intelligent control strategies in industrial automation scenarios.

[0223] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0224] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of equivalent technology of this disclosure, then the intent of this disclosure also includes such modifications and variations.

Claims

1. A personal intelligent all-in-one machine, characterized in that, include: The automation system IDE module, linked to the automation system driver module, is used to receive industrial tasks provided by the process package, break down the industrial tasks into the smallest task units, convert the smallest task units into structured scheduling instructions, and send the structured scheduling instructions to the automation system driver module. The automation system drive module is connected to both the robot body drive module and the factory automation system. It is used to convert structured scheduling instructions into control commands that the robot can execute, receive data information generated by the factory automation system, and send the control commands that the robot can execute and the data information generated by the factory automation system to the robot body drive module. The robot body drive module is connected to various robot bodies and is used to control the corresponding robot body to perform corresponding actions according to the control commands that the robot can execute. At the same time, according to the request of the function supply module or the automation system drive module, it receives various data of the corresponding robot body in real time and sends the various data of the robot body to the function supply module or the automation system drive module. The functional supply module, connected to the robot body drive module, is used to provide the robot body drive module with time-synchronized data acquisition, simulation task execution, model training, and model inference functions according to the control commands that the robot can execute. The functional supply module includes: The data acquisition module is connected to both the database and the robot body drive module. It is used to collect data from various robot bodies in real time, process the collected data from various robot bodies in real time using a time synchronization method, and send the time-synchronized robot body data to the database. The time synchronization method includes: timestamp alignment, timestamp correction, time window aggregation, and delay compensation. The timestamp alignment is performed by using hardware timestamps to align the real-time collected robot body data when the sampling frequency of the robot body's sensors is greater than a first frequency threshold, and by using software timestamps to align the real-time collected robot body data when the sampling frequency of the robot body's sensors is less than or equal to the first frequency threshold. The time window aggregation involves statistically analyzing or integrating the continuous sampled values ​​of the sensors within a time window when the sampling frequency of the robot's sensors exceeds a first frequency threshold, in order to form an input feature within the time window; and dynamically adjusting the length of the time window based on the sensor's sampling frequency and response time. The length of the time window is calculated as follows: ； Where f is the sampling frequency of the robot's sensors, k is the window coefficient, and W is the length of the time window; The window coefficient is calculated as follows: ； Where τ is the response time of the robot's sensors, and f is the sampling frequency of the robot's sensors. The redundancy safety factor is ceil(), which is the ceil function.

2. The all-in-one intelligent machine according to claim 1, characterized in that, The functional supply module includes: The model training module connects to the database and the model library respectively. It is used to call the robot body data collected in real time from the database and the data generated by simulation for model training and continuous optimization, and save the optimized model in the model library. The model reasoning module is connected to the model library and the robot body driving module respectively. It is used to generate control commands that the robot can execute in real time based on the optimized model stored in the model library during the execution of the smallest task unit, and send the control commands that the robot can execute to the robot body driving module. The data simulation module, connected to the robot body drive module, is used to dynamically construct a simulation environment based on real-time collected robot body data, simulate the execution process of the smallest task unit, and verify the training and optimization effects of the trained and optimized models in the simulation environment.

3. The all-in-one intelligent device according to claim 1, characterized in that, The hardware timestamp alignment includes: When the data acquisition module sends an acquisition request, a start timestamp is generated using the current system time; Based on the start timestamp, the robot's sensors record hardware timestamps at the physical layer; The collected robot body data, along with a hardware timestamp, is sent to the data acquisition module. The data acquisition module synchronizes data time according to the start timestamp and the hardware timestamp.

4. The all-in-one intelligent machine according to claim 1, characterized in that, The software timestamp alignment includes: When the data acquisition module sends an acquisition request, a start timestamp is generated using the current system time; Based on the starting timestamp, the first timestamp correction is performed to obtain a dynamic estimate of the network round-trip time delay; Based on the dynamic estimate of the network round-trip time delay, the receiving time of the robot body data is corrected, and the time drift of the network round-trip time delay is corrected a second time at a specified frequency to obtain the dynamic estimate of the network round-trip time delay after the second correction. The data acquisition module synchronizes data time according to the start timestamp and the dynamic estimate of the network round-trip time delay after the second correction.

5. A holographic intelligent all-in-one machine according to claim 4, characterized in that, The method for correcting the robot's data reception time based on a dynamic estimate of the network round-trip time delay is as follows: ； in, The time of receiving the corrected robot body data. d0 represents the time it takes to receive robot body data, and d0 represents a dynamic estimate of the network round-trip time delay.

6. The all-in-one intelligent device according to claim 1, characterized in that, The timestamp correction includes: The sensors of all robot bodies are synchronized in time periodically via NTP or PTP protocol. The first timestamp correction is performed for the network round-trip time delay, and the deviation coefficient of each correction is corrected a second time.

7. A holographic intelligent all-in-one machine according to claim 6, characterized in that, The first timestamp correction is calculated as follows: ； in, Let N be the measured network round-trip time delay for the i-th time, and N be the number of time synchronization attempts. The moving average estimate of the network round-trip time delay is used as the compensation reference after the first correction. That is, the dynamic estimate of the network round-trip time delay. If the fluctuation of the dynamic estimate of the network round-trip time delay is less than the preset fluctuation threshold, the robot body data collected in this round passes the first correction. If the fluctuation of the dynamic estimate of the network round-trip time delay is greater than or equal to the preset fluctuation threshold, the robot body data collected in this round is automatically discarded and the process of aligning the robot body data with the timestamp is re-executed.

8. A holographic intelligent all-in-one machine according to claim 6, characterized in that, The second timestamp correction is calculated as follows: ； Where P is the deviation coefficient for each correction. The measured value of the network round-trip time delay for the i-th time. The compensation reference value after the first correction is used. If the deviation coefficient of each correction is within the set range, the robot body data collected this time will be corrected for the second time. If the deviation coefficient of the correction exceeds the set range, the compensation reference value after the first correction will be stopped from being updated further.

9. A holographic intelligent all-in-one machine according to claim 1, characterized in that, The delay compensation includes static time compensation and dynamic time compensation; The static time compensation refers to the static delay compensation performed on the robot's sensors before the robot's data is transmitted back, given the known network round-trip time delay. The dynamic time compensation refers to the dynamic time compensation of the robot body data by using dynamic time warping when the difference between the sampling frequency of the first sensor and the sampling frequency of the second sensor is greater than a preset frequency threshold.

10. A holographic intelligent all-in-one machine according to claim 9, characterized in that, The static time compensation is calculated as follows: ； in, This refers to the time of data acquisition for the robot body after static time compensation. This is the measured value of the network round-trip time delay for the current round. For the time it takes to receive robot body data, This refers to the time when the robot's body data is sent.

11. A body-worn intelligent all-in-one machine according to claim 9, characterized in that, The dynamic time compensation includes: Based on the order of the start timestamp, time series X and Y are constructed using robot body data collected by the first sensor and robot body data collected by the second sensor. Calculate the distance between time series X and Y at every time step, and construct a distance matrix; The FastDTW algorithm, which employs a path constraint strategy that optimizes bandwidth, searches for the minimum path in the distance matrix. Calculate the time compensation amount based on the time length corresponding to the shortest path; Add time compensation to the robot body data collected by the second sensor to obtain dynamically time-compensated robot body data.

12. A body-worn intelligent all-in-one machine according to claim 11, characterized in that, The distance matrix is ​​calculated as follows: ； in, The element in the i-th row and j-th column of the distance matrix is... and Distance between them This refers to the robot body data collected by the first sensor in the i-th iteration. This refers to the robot body data collected by the second sensor in the j-th iteration. for and Distance between them for and Distance between them for and The distance between them.

13. A body-worn intelligent all-in-one machine according to claim 11, characterized in that, The path constraint strategy for optimizing bandwidth, and the formula for calculating the optimized bandwidth are as follows: ； in, The sampling frequency of the first sensor. The sampling frequency of the second sensor. For bandwidth control parameters, This represents the optimized bandwidth.

14. A embodied intelligent system, implemented using an embodied intelligent all-in-one machine as described in any one of claims 1 to 13, characterized in that, include: It integrates intelligent all-in-one machines, factory automation systems, and various robot bodies; The factory automation system is used to receive sensor data, engine data, mechanical device data, execution equipment data and manufacturing execution system data in the factory automation, and send the received sensor data, engine data, mechanical device data, execution equipment data and manufacturing execution system data in the factory automation to the automation system drive module; The various robot bodies are used to transmit various data of the robot bodies to the robot body drive package.

15. A method for operating an embodied intelligent all-in-one machine, implemented using the embodied intelligent system described in claim 14, characterized in that, include: The automation system IDE module receives industrial tasks provided by the process package, breaks down the industrial tasks into the smallest task units, converts the smallest task units into structured scheduling instructions, and sends the structured scheduling instructions to the automation system driver module. The automation system drive module converts structured scheduling instructions into control commands that the robot can execute, receives data information generated by the factory automation system, and sends the control commands that the robot can execute and the data information generated by the factory automation system to the robot body drive module. The robot body drive module controls the corresponding robot body to perform corresponding actions according to the control commands that the robot can execute. At the same time, according to the request of the function supply module or the automation system drive module, it receives various data of the corresponding robot body in real time and sends the various data of the robot body to the function supply module or the automation system drive module. The functional supply module provides the robot body drive module with time-synchronized data acquisition, simulation task execution, model training, and model inference functions based on the control commands that the robot can execute.

16. The method for operating a holographic intelligent all-in-one machine according to claim 15, characterized in that, Also includes: The data acquisition module collects data from various robot bodies in real time and processes the collected data in real time using time synchronization methods. The time synchronization methods include: timestamp alignment, timestamp correction, time window aggregation, and delay compensation. The model training module retrieves real-time robot body data and simulation-generated data from the database to train and continuously optimize the model, and saves the optimized model in the model library. During the execution of the smallest task unit, the model reasoning module generates control commands that the robot can execute in real time based on the optimized model stored in the model library, and sends the control commands that the robot can execute to the robot body drive module. The data simulation module dynamically constructs a simulation environment based on real-time collected robot body data, simulates the execution process of the smallest task unit, and verifies the training and optimization effects of the trained and optimized models in the simulation environment.

Images