An embodied intelligent robot electric core loading system and method based on imitation learning

Abstract

The application discloses a kind of embodied intelligent robot electric core loading system and method based on imitation learning, comprising: active arm, for receiving the remote operation input of operator and generating control instruction sequence;Passive arm, for executing electric core grabbing action and feedback joint state sequence;RGB camera is installed at the end of robot, for collecting the RGB image sequence of working environment;Data acquisition module is electrically connected with active arm, passive arm and RGB camera respectively, for synchronously collecting control instruction sequence, joint state sequence and RGB image sequence, and adding time stamp for control instruction sequence, joint state sequence and RGB image sequence respectively;Data processing module is used for time stamp alignment and missing data interpolation to the collected control instruction sequence, joint state sequence and RGB image sequence, generates continuous alignment multimodal teaching data set.The system and method of the application can realize the flexible grabbing and loading of electric core, significantly improve production efficiency and product qualification rate.

Description

Technical Field

[0001] This invention relates to the fields of intelligent manufacturing and robotics, and in particular to an embodied intelligent robot battery cell loading system and method based on imitation learning. Background Technology

[0002] As a core component of new energy vehicles and energy storage devices, battery cells require precise and flexible placement of individual cells to designated workstations in their module assembly (PACK) production lines. The level of automation in this process directly impacts production efficiency and product quality. Currently, the industry primarily relies on the following two technological solutions: The first method is manual feeding; operators rely on experience to manually pick up and place the battery cells. The advantage of this method is that human hand-eye coordination can handle unstructured scenarios such as inconsistent cell placement and varying incoming specifications. However, its disadvantages are significant: firstly, it is inefficient and cannot meet the demands of continuous mass production; secondly, manual operation suffers from poor stability and consistency, easily affected by operator fatigue and skill level, leading to scratches on the casing or damage to the electrodes due to improper gripping force, making it difficult to guarantee product qualification rates; finally, it poses certain occupational safety risks.

[0003] The second approach is a wheeled robot loading solution based on traditional teach-programming. In this solution, the robot moves along a preset path and grasps the battery cells using a fixed gripper with pre-set parameters. This approach frees up manpower to some extent and improves operational stability. However, this solution has fundamental flaws: First, poor adaptability. The robot can only handle battery cells with strictly standardized positions and postures. Once the placement of the battery cells shifts, the stack tilts, or production batch changes lead to variations in cell size, the entire system malfunctions. Professional engineers must then perform tedious path planning and parameter adjustments, resulting in long debugging cycles and high switching costs. Second, lack of flexibility. Traditional grippers lack real-time force sensing and feedback adjustment capabilities; the grasping process is "blind." Excessive grasping pressure can damage the battery cells, while insufficient pressure may cause them to slip during transport, also leading to quality issues. Third, lack of learning ability. The system cannot accumulate experience from successful operations, nor can it quickly master new skills through simple demonstrations, resulting in a low level of intelligence.

[0004] Imitation learning technology has shown great potential in robot skill acquisition, enabling robots to learn complex movements by observing human demonstrations. However, applying imitation learning from laboratory research to the specific industrial scenario of battery cell loading still faces a series of severe challenges: on the one hand, it requires high-fidelity, synchronous acquisition of multi-dimensional state data (such as images, joint angles, and control commands) during the teaching process; on the other hand, industrial sites suffer from data loss and timing asynchrony, which need to be effectively addressed to construct a high-quality training dataset. Simultaneously, to meet the stringent requirements of production lines for real-time performance (typically requiring end-to-end latency of less than 100 milliseconds) and reliability, the large-scale neural network models trained must be compressed and optimized. Furthermore, force feedback needs to be effectively integrated into the actual grasping process to achieve truly flexible adaptive control. Currently, there is a lack of a complete technical solution that can systematically solve the aforementioned problems across the entire chain.

[0005] Therefore, there is an urgent need to propose an innovative battery cell loading system and method to overcome the shortcomings of traditional methods in terms of poor adaptability and insufficient flexibility, and to solve the data, model and real-time control problems faced in the industrialization of imitation learning technology. Summary of the Invention

[0006] To address the technical problems existing in the background art, this invention proposes an embodied intelligent robot battery cell feeding system and method based on imitation learning.

[0007] This invention proposes an embodied intelligent robot battery cell loading system based on imitation learning, comprising: The active arm is used to receive remote operation input from the operator and generate a sequence of control commands; The passive arm is used to perform cell grasping actions and provide feedback on the joint state sequence. An RGB camera, installed at the end effector of the robot, is used to capture RGB image sequences of the working environment; The data acquisition module is electrically connected to the active arm, passive arm, and RGB camera respectively. It is used to synchronously acquire control command sequences, joint state sequences, and RGB image sequences, and add timestamps to the control command sequences, joint state sequences, and RGB image sequences respectively. The data processing module is used to perform timestamp alignment and missing data interpolation on the acquired control command sequence, joint state sequence and RGB image sequence to generate a continuously aligned multimodal teaching dataset; The model training module is used to train a battery cell grasping action model using a multimodal teaching dataset; The model compression module is used to optimize and compress the trained cell grabbing action model into a lightweight engine model. The control module is electrically connected to the model compression module, the RGB camera, and the passive arm, respectively. It is used to call the lightweight engine model to generate the desired joint state in real-time operation and control the passive arm to complete the cell gripping and loading.

[0008] Preferably, the data processing module specifically includes: The timestamp alignment unit is used to unify the timestamps of the control command sequence, joint state sequence and RGB image sequence to the same time base and establish a one-to-one correspondence between the three sets of sequences. The interpolation unit is used to detect missing data frames in the three sets of sequences. When the time interval between adjacent data frames exceeds a preset threshold, the joint state sequence is numerically interpolated using linear interpolation, and the RGB image sequence is image interpolated using bilinear interpolation. The interpolation range does not exceed 5 consecutive missing frames. The dataset generation unit is used to integrate the three interpolated sequences into a continuously aligned multimodal teaching dataset in chronological order, and to store the multimodal teaching dataset as a structured data file in HDF5 format. The structured data file contains image data groups, joint state data groups, and control command data groups indexed by timestamps.

[0009] Preferably, the model training module specifically includes: The training unit is used to take the RGB image sequence and control command sequence in the multimodal teaching dataset as the model input features and the corresponding joint state sequence as the supervision label. By iteratively executing the forward propagation, loss calculation and back propagation process, the difference between the joint state predicted by the model and the supervision label is minimized, thereby training the initial cell grasping action model. The validation unit is used to evaluate the performance of the initial cell grasping action model during training using an independent validation dataset, and to monitor the prediction accuracy and generalization ability of the initial cell grasping action model on unseen data. The model saving unit is used to save the battery cell grasping action model whose performance meets the preset requirements as a model parameter file after training is completed, and output the model parameter file to the model compression module.

[0010] Preferably, the model compression module specifically includes: The analysis unit is used to perform structural analysis and performance evaluation on the trained battery cell grasping action model, and to identify the optimizable computational graph structure and redundant parameters in the battery cell grasping action model. The optimization unit is used to prune the identified redundant parameters and quantize the weights of the cell grasping action model using FP16 or INT8. At the same time, it performs operator fusion and inter-layer optimization to reduce the computational complexity and memory usage of the cell grasping action model. The generation unit is used to convert the optimized cell grabbing action model into a lightweight engine model in TensorRT engine format, and output the converted lightweight engine model to the control module for real-time use.

[0011] Preferably, the control module specifically includes: The sensing unit is used to receive RGB images of the current working environment from the RGB camera in real time; The inference unit is used to call the lightweight engine model, input the current working environment RGB image into the lightweight engine model, and obtain the expected joint state output by the lightweight engine model. The instruction generation unit is used to generate robot control instructions based on the desired joint state and send the robot control instructions to the passive arm to drive the passive arm to complete the battery cell gripping and loading operations.

[0012] Preferably, the data acquisition module specifically includes: The synchronization control unit is used to generate a unified acquisition trigger signal and send it to the active arm, passive arm and RGB camera respectively to start the synchronous acquisition of control commands, joint status and RGB images; The data caching unit is communicatively connected to the active arm, the passive arm, and the RGB camera, and is used to receive and temporarily store control command data from the active arm, joint state data from the passive arm, and RGB image data from the RGB camera, respectively. The timestamp addition unit, connected to the data cache unit, is used to add timestamps to each set of control command data, joint status data and RGB image data temporarily stored in the data cache unit, based on the same system clock source, thereby generating a control command sequence, joint status sequence and RGB image sequence with timestamps.

[0013] Preferably, it further includes: The visualization verification module is used to receive the multimodal teaching dataset and perform visualization processing on the multimodal teaching dataset; the visualization verification module specifically includes a video generation unit and a curve drawing unit; The video generation unit is used to synthesize RGB image sequences from the multimodal teaching dataset into video files in chronological order for visual review of motion smoothness and scene integrity. The curve plotting unit is used to extract the joint state sequence and control command sequence from the multimodal teaching dataset, and plot the curves of joint angle change over time and control command change over time, for data analysis of joint motion continuity and command consistency.

[0014] Preferably, it further includes: A force sensor, mounted on the end gripper of the passive arm, is used to monitor the gripping force in real time; the control module dynamically adjusts the gripping action based on the feedback data from the force sensor.

[0015] Preferably, it further includes: The wheeled mobile platform, in which the active arm, passive arm, and RGB camera are all mounted, enables free movement within the working area.

[0016] This invention proposes a battery cell loading method based on imitation learning using an embodied intelligent robot, applicable to the battery cell loading system based on imitation learning described in any of the above claims. The method includes the following steps: S1. Receive remote operation input from the operator through the active arm and execute teaching actions for grabbing and placing battery cells; S2. The control command sequence, RGB image sequence obtained by the RGB camera, and joint state sequence fed back by the passive arm are synchronously collected through the data acquisition module during the teaching process, and timestamps are added to the control command sequence, joint state sequence and RGB image sequence respectively. S3. The data processing module performs timestamp alignment and missing data interpolation on the control command sequence, joint state sequence, and RGB image sequence to generate a continuously aligned multimodal teaching dataset. S4. The battery cell grasping action model is trained using the multimodal teaching dataset through the model training module. S5. The cell grabbing action model is optimized and compressed into a lightweight engine model through the model compression module. S6. In real-time loading operations, the control module calls the lightweight engine model to generate the desired joint state based on the real-time images captured by the RGB camera, and controls the passive arm to complete the gripping and loading of battery cells.

[0017] The embodied intelligent robot battery cell loading system and method proposed in this invention, through the deep integration of imitation learning and multimodal perception, enables the robot to quickly master battery cell grasping skills through a small number of demonstrations. This allows for adaptive grasping and precise loading of battery cells of different specifications and placement postures, improving the flexibility of the production line. Simultaneously, the system optimizes the entire process from data acquisition, processing, model training to lightweight deployment, ensuring high reliability and real-time performance in actual industrial environments. This not only improves production efficiency and product consistency and reduces the risk of damage to battery cells during grasping, but also shortens production line changeover and debugging time, reducing reliance on professional programmers. Attached Figure Description

[0018] Figure 1This is a schematic diagram of the system architecture of an embodied intelligent robot battery cell loading system based on imitation learning proposed in this invention. Figure 2 This is a schematic diagram illustrating the workflow of an embodied intelligent robot battery cell loading method based on imitation learning proposed in this invention. Detailed Implementation

[0019] Reference Figure 1 and Figure 2 The present invention proposes an embodied intelligent robot battery cell loading system based on imitation learning, comprising: The active arm is used to receive remote operation input from the operator and generate a sequence of control commands.

[0020] The passive arm is used to perform cell grasping actions and provide feedback on the joint state sequence.

[0021] An RGB camera, installed at the end effector of the robot, is used to capture RGB image sequences of the working environment.

[0022] The data acquisition module is electrically connected to the active arm, passive arm, and RGB camera respectively. It is used to synchronously acquire control command sequences, joint state sequences, and RGB image sequences, and add timestamps to the control command sequences, joint state sequences, and RGB image sequences respectively.

[0023] In this embodiment, the data acquisition module specifically includes: The synchronization control unit is used to generate a unified acquisition trigger signal and send it to the active arm, passive arm and RGB camera respectively to start the synchronous acquisition of control commands, joint status and RGB images; The data caching unit is communicatively connected to the active arm, the passive arm, and the RGB camera, and is used to receive and temporarily store control command data from the active arm, joint state data from the passive arm, and RGB image data from the RGB camera, respectively. The timestamp addition unit, connected to the data cache unit, is used to add timestamps to each set of control command data, joint status data and RGB image data temporarily stored in the data cache unit, based on the same system clock source, thereby generating a control command sequence, joint status sequence and RGB image sequence with timestamps.

[0024] Specifically, the implementation of the data acquisition module includes: Synchronization control unit: It uses hardware triggering or high-precision software timer (such as PTP protocol) to generate a unified acquisition trigger pulse signal, which is sent to the trigger ports of active arm controller, passive arm controller and RGB camera to ensure that the start time of the acquisition action of the three is strictly synchronized, with a synchronization error of less than 1 millisecond.

[0025] Data buffer unit: Employing a circular buffer or queue structure, it receives joint target angle commands (control commands) from the active arm, real-time joint angles and angular velocities (joint states) from the passive arm encoder, and high-speed image frames (RGB images) from the camera. The buffer size is set according to the acquisition frequency (e.g., 100Hz) and teaching duration (5-10 minutes) to prevent data overflow.

[0026] Timestamp addition unit: integrates a high-precision clock chip (such as GPS / BeiDou discipline clock) to add a microsecond-level timestamp based on this clock source to each set of data entering the buffer (control command data packet, joint status data packet, image frame) at the moment of reception.

[0027] The data processing module is used to perform timestamp alignment and missing data interpolation on the acquired control command sequence, joint state sequence and RGB image sequence to generate a continuously aligned multimodal teaching dataset.

[0028] In this embodiment, the data processing module specifically includes: The timestamp alignment unit is used to unify the timestamps of the control command sequence, joint state sequence and RGB image sequence to the same time base and establish a one-to-one correspondence between the three sets of sequences. The interpolation unit is used to detect missing data frames in the three sets of sequences. When the time interval between adjacent data frames exceeds a preset threshold, the joint state sequence is numerically interpolated using linear interpolation, and the RGB image sequence is image interpolated using bilinear interpolation. The interpolation range does not exceed 5 consecutive missing frames. The dataset generation unit is used to integrate the three interpolated sequences into a continuously aligned multimodal teaching dataset in chronological order, and store the multimodal teaching dataset as a structured data file in HDF5 format. The structured data file contains image data groups, joint state data groups, and control command data groups indexed by timestamps.

[0029] Specifically, the implementation of the data processing module includes: Timestamp Alignment Unit: Reads three sets of original sequences with timestamps. An alignment time window is set (e.g., ±2 milliseconds). Using the timestamp of the image frame as a reference, the nearest neighbor control command and joint state data point are found within this window, establishing a one-to-one mapping relationship between the three. If no corresponding data is found within the window, it is marked as needing interpolation.

[0030] Imputation unit: Scans the aligned sequence to detect missing data. The preset threshold is 66 milliseconds (corresponding to approximately 6 frames of data at 100Hz). When a missing data point is detected: Linear interpolation is used for the joint state sequence. Assume the starting state of the missing interval is... The endpoint is ,interval The missing point, then the first The state of each missing point .

[0031] For RGB image sequences, bilinear interpolation is used to interpolate the missing preceding and following frames to generate visually smooth intermediate frames.

[0032] The interpolation range is strictly limited to no more than 5 consecutive missing frames. If the range is exceeded, the data is considered invalid, and it is recommended to check for problems at the data source (acquisition stage).

[0033] Dataset Generation Unit: This unit integrates the processed data in chronological order. Each data packet corresponding to a given time point contains: an RGB image, a set of 7-dimensional joint state vectors, and a set of 7-dimensional control command vectors. This series of data packets is stored as a structured file in HDF5 format. The file's internal structure is designed as three datasets, each indexed by a timestamp array, facilitating fast random access by time.

[0034] The model training module is used to train a battery cell grasping action model using a multimodal teaching dataset.

[0035] In this embodiment, the model training module specifically includes: The training unit is used to take the RGB image sequence and control command sequence in the multimodal teaching dataset as the model input features and the corresponding joint state sequence as the supervision label. By iteratively executing the forward propagation, loss calculation and back propagation process, the difference between the joint state predicted by the model and the supervision label is minimized, thereby training the initial cell grasping action model. The validation unit is used to evaluate the performance of the initial cell grasping action model during training using an independent validation dataset, and to monitor the prediction accuracy and generalization ability of the initial cell grasping action model on unseen data. The model saving unit is used to save the battery cell grasping action model that meets the preset performance requirements as a model parameter file after training is completed, and output the model parameter file to the model compression module.

[0036] Specifically, the implementation of the model training module includes: Training Unit: Model architecture: An encoder-decoder structure combining convolutional neural networks (CNN) and long short-term memory networks (LSTM) is adopted. The CNN encoder processes the RGB image sequence and extracts visual features; the control instruction sequence and visual features are fused in the LSTM layer, and the decoder outputs the predicted joint state sequence.

[0037] Input features: Input consecutive RGB image frames and corresponding control commands in the form of a time sliding window (e.g., with a length of 10).

[0038] Supervision label: The actual joint state corresponding to the last moment of the input window.

[0039] Loss function: Mean squared error loss (MSE) is used. ; in, The joint states predicted by the model. This represents the actual state of the joints. Batch size; This is the mean squared error loss. This loss is minimized iteratively using a backpropagation algorithm (such as the Adam optimizer).

[0040] Validation Unit: 20% of the total dataset is allocated as the validation set. The model accuracy is evaluated on the validation set after each training epoch. The validation set loss is monitored for a continuous decrease; if it stops decreasing or begins to increase after several consecutive epochs, overfitting may occur, triggering the Early Stopping mechanism.

[0041] Model saving unit: Saves the model parameters that perform best on the validation set, in a file named model_epoch_best.pth. It also periodically saves checkpoints from the training process, such as model_epoch_50.pth, to support resuming training from breakpoints.

[0042] The model compression module is used to optimize and compress the trained cell-grabbing action model into a lightweight engine model.

[0043] In this embodiment, the model compression module specifically includes: The analysis unit is used to perform structural analysis and performance evaluation on the trained cell grasping action model, and to identify the optimizable computational graph structure and redundant parameters in the cell grasping action model. The optimization unit is used to prune the identified redundant parameters and quantize the weights of the cell grasping action model using FP16 or INT8. At the same time, it performs operator fusion and inter-layer optimization to reduce the computational complexity and memory usage of the cell grasping action model. The generation unit is used to convert the optimized cell grabbing action model into a lightweight engine model in TensorRT engine format, and output the converted lightweight engine model to the control module for real-time use.

[0044] Specifically, the implementation of the model compression module includes: Analysis Unit: Load the trained model_epoch_best.pth, use model analysis tools (such as NVIDIA's PyTorch Profiler) to analyze the computation time and memory usage of each operator in the computation graph, identify fusionable operator pairs (such as Conv+BatchNorm+ReLU) and weight distribution, and evaluate the appropriate quantization accuracy.

[0045] Optimization Unit: Pruning: Sparsity pruning is performed on weights with small absolute values ​​in the model to remove connections with low contribution.

[0046] Quantization: Convert model weights and activation values ​​from FP32 precision to FP16 mixed precision or INT8 precision. For INT8 quantization, a calibration dataset is needed to determine the scaling factor and zero point for each layer.

[0047] Operator fusion: Combines common sequential operators (such as convolution, bias, activation functions) into a single, more efficient GPU operator.

[0048] Generation Unit: Utilizing the TensorRT framework, the optimized computation graph is compiled into a highly optimized inference engine and saved as the `model_optimized.engine` file. This engine file is strongly dependent on the hardware (specific GPU model), achieving extreme inference acceleration.

[0049] The control module is electrically connected to the model compression module, the RGB camera, and the passive arm, respectively. It is used to call the lightweight engine model to generate the desired joint state in real-time operation and control the passive arm to complete the cell gripping and loading.

[0050] In this embodiment, the control module specifically includes: The sensing unit is used to receive RGB images of the current working environment from the RGB camera in real time; The inference unit is used to call the lightweight engine model, input the current working environment RGB image into the lightweight engine model, and obtain the expected joint state output by the lightweight engine model. The instruction generation unit is used to generate robot control instructions based on the desired joint state and send the robot control instructions to the passive arm to drive the passive arm to complete the battery cell gripping and loading operations.

[0051] Specifically, the implementation of the control module includes: Sensing unit: Acquires real-time video stream from RGB camera at a fixed frame rate (e.g., 30Hz) and performs necessary preprocessing (e.g., scaling, normalization) on each frame.

[0052] Inference Unit: Loads the model_optimized.engine file and initializes the TensorRT runtime. It inputs the preprocessed image frame into the engine, performs inference on the GPU, and outputs the expected joint state (7-dimensional angle vector) for the next time step within milliseconds (usually less than 10ms).

[0053] Command generation unit: Converts the desired joint state into a command format (such as target position, velocity, and torque) understandable by the robot's underlying controller (e.g., a position-based impedance controller). The core of this unit lies in fusing feedback from force sensors. Force sensors measure in real time the normal and tangential forces between the gripper fingertips and the battery cell.

[0054] The instruction generation unit has a built-in admittance control or adaptive impedance control algorithm. When the feedback force exceeds the preset safety threshold (e.g., 25N), the algorithm dynamically adjusts the desired joint state or directly generates a compensation torque command, so that the robotic arm can "smoothly" reduce the gripping force to prevent damage to the battery cell; when the feedback force is too small and there is a risk of detachment, the gripping force is appropriately increased.

[0055] This process constitutes a high-frequency (>500Hz) force closed-loop control, ensuring the flexibility and safety of the grasping process.

[0056] In this embodiment, it also includes: The visualization verification module is used to receive the multimodal teaching dataset and perform visualization processing on the multimodal teaching dataset; the visualization verification module specifically includes a video generation unit and a curve plotting unit; The video generation unit is used to synthesize RGB image sequences from the multimodal teaching dataset into video files in chronological order for visual review of motion smoothness and scene integrity. The curve plotting unit is used to extract the joint state sequence and control command sequence from the multimodal teaching dataset, and plot the curves of joint angle change over time and control command change over time, for data analysis of joint motion continuity and command consistency.

[0057] In this embodiment, it also includes: A force sensor, mounted on the end gripper of the passive arm, is used to monitor the gripping force in real time; the control module dynamically adjusts the gripping action based on the feedback data from the force sensor.

[0058] In this embodiment, it also includes: The wheeled mobile platform, with its active arm, passive arm, and RGB camera all mounted on it, allows for free movement within the work area.

[0059] Specifically, the visualization verification module operates as a standalone tool. It reads the HDF5 format teaching dataset, and the video generation unit uses the OpenCV library to encode the image sequences into MP4 videos for manual review to ensure natural movements and comprehensive scene coverage. The curve plotting unit uses Matplotlib to plot curves of joint angles, angular velocities, and control commands over time, checking data continuity and consistency, and automatically detecting abnormal transition points. The wheeled mobile platform uses an omnidirectional Mecanum wheel chassis and integrates a laser SLAM navigation system. When the system needs to serve multiple loading stations, the control system plans the movement path, the platform autonomously navigates to the target cell storage area, and after repositioning by the vision system, it performs the gripping and loading operation, greatly expanding the workspace.

[0060] During real-time material loading, the system can use successful operation data (images, status, instructions) as new positive samples and failed operations as negative samples. Periodically or after accumulating a certain amount, the system can start a background incremental training task to fine-tune model_epoch_best.pth, so as to achieve continuous evolution of the model and adapt to the slow changes in the production line.

[0061] It should be noted that the system workflow is divided into two main stages: the offline training stage and the real-time material loading stage.

[0062] During the offline training phase, operators use the active arm to remotely teach the grabbing and placement of battery cells. The data acquisition module simultaneously collects control command sequences, RGB image sequences from an RGB camera, and joint state sequences fed back by the passive arm, adding high-precision timestamps based on the same clock source to each sequence. The data processing module aligns the timestamps and imputes missing data in these three sets of sequences, generating a continuously aligned multimodal teaching dataset. The model training module uses this dataset to train a battery cell grabbing action model using an imitation learning algorithm. The model compression module optimizes and compresses the model, converting it into a lightweight engine model suitable for real-time inference.

[0063] During the real-time loading phase, the control module is activated, invoking the lightweight engine model. An RGB camera captures real-time images of the working environment, which the control module inputs into the model to obtain the predicted desired joint states. This generates control commands to drive the passive arm to perform flexible gripping and precise loading of the battery cells. A force sensor installed at the end of the fixture provides real-time feedback on the gripping force, allowing the control module to dynamically adjust the commands and achieve closed-loop force control.

[0064] Reference Figure 1 and Figure 2The present invention proposes a battery cell loading method based on imitation learning for embodied intelligent robots, applicable to any of the above-mentioned battery cell loading systems based on imitation learning for embodied intelligent robots. The method includes the following steps: S1. Receive remote operation input from the operator through the active arm and execute teaching actions for grabbing and placing battery cells; S2. The control command sequence, RGB image sequence obtained by the RGB camera, and joint state sequence fed back by the passive arm are synchronously collected through the data acquisition module during the teaching process, and timestamps are added to the control command sequence, joint state sequence and RGB image sequence respectively. S3. The data processing module performs timestamp alignment and missing data interpolation on the control command sequence, joint state sequence, and RGB image sequence to generate a continuously aligned multimodal teaching dataset. S4. The battery cell grasping action model is trained using the multimodal teaching dataset through the model training module. S5. The cell grabbing action model is optimized and compressed into a lightweight engine model through the model compression module. S6. In real-time loading operations, the control module calls the lightweight engine model to generate the desired joint state based on the real-time images captured by the RGB camera, and controls the passive arm to complete the gripping and loading of battery cells.

[0065] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A embodied intelligent robot battery cell feeding system based on imitation learning, characterized in that, include: The active arm is used to receive remote operation input from the operator and generate a sequence of control commands; The passive arm is used to perform cell grasping actions and provide feedback on the joint state sequence. An RGB camera, installed at the end effector of the robot, is used to capture RGB image sequences of the working environment; The data acquisition module is electrically connected to the active arm, passive arm, and RGB camera respectively. It is used to synchronously acquire control command sequences, joint state sequences, and RGB image sequences, and add timestamps to the control command sequences, joint state sequences, and RGB image sequences respectively. The data processing module is used to perform timestamp alignment and missing data interpolation on the acquired control command sequence, joint state sequence and RGB image sequence to generate a continuously aligned multimodal teaching dataset; The model training module is used to train a battery cell grasping action model using a multimodal teaching dataset; The model compression module is used to optimize and compress the trained cell grabbing action model into a lightweight engine model. The control module is electrically connected to the model compression module, the RGB camera, and the passive arm, respectively. It is used to call the lightweight engine model to generate the desired joint state in real-time operation and control the passive arm to complete the cell gripping and loading.

2. The embodied intelligent robot cell loading system based on imitation learning according to claim 1, characterized in that, The data processing module specifically includes: The timestamp alignment unit is used to unify the timestamps of the control command sequence, joint state sequence and RGB image sequence to the same time base and establish a one-to-one correspondence between the three sets of sequences. The interpolation unit is used to detect missing data frames in the three sets of sequences. When the time interval between adjacent data frames exceeds a preset threshold, the joint state sequence is numerically interpolated using linear interpolation, and the RGB image sequence is image interpolated using bilinear interpolation. The interpolation range does not exceed 5 consecutive missing frames. The dataset generation unit is used to integrate the three interpolated sequences into a continuously aligned multimodal teaching dataset in chronological order, and to store the multimodal teaching dataset as a structured data file in HDF5 format. The structured data file contains image data groups, joint state data groups, and control command data groups indexed by timestamps.

3. The embodied intelligent robot cell loading system based on imitation learning according to claim 1, characterized in that, The model training module specifically includes: The training unit is used to take the RGB image sequence and control command sequence in the multimodal teaching dataset as the model input features and the corresponding joint state sequence as the supervision label. By iteratively executing the forward propagation, loss calculation and back propagation process, the difference between the joint state predicted by the model and the supervision label is minimized, thereby training the initial cell grasping action model. The validation unit is used to evaluate the performance of the initial cell grasping action model during training using an independent validation dataset, and to monitor the prediction accuracy and generalization ability of the initial cell grasping action model on unseen data. The model saving unit is used to save the battery cell grasping action model whose performance meets the preset requirements as a model parameter file after training is completed, and output the model parameter file to the model compression module.

4. The embodied intelligent robot cell loading system based on imitation learning according to claim 1, characterized in that, The model compression module specifically includes: The analysis unit is used to perform structural analysis and performance evaluation on the trained battery cell grasping action model, and to identify the optimizable computational graph structure and redundant parameters in the battery cell grasping action model. The optimization unit is used to prune the identified redundant parameters and quantize the weights of the cell grasping action model using FP16 or INT8. At the same time, it performs operator fusion and inter-layer optimization to reduce the computational complexity and memory usage of the cell grasping action model. The generation unit is used to convert the optimized cell grabbing action model into a lightweight engine model in TensorRT engine format, and output the converted lightweight engine model to the control module for real-time use.

5. The embodied intelligent robot cell loading system based on imitation learning according to claim 1, characterized in that, The control module specifically includes: The sensing unit is used to receive RGB images of the current working environment from the RGB camera in real time; The inference unit is used to call the lightweight engine model, input the current working environment RGB image into the lightweight engine model, and obtain the expected joint state output by the lightweight engine model. The instruction generation unit is used to generate robot control instructions based on the desired joint state and send the robot control instructions to the passive arm to drive the passive arm to complete the battery cell gripping and loading operations.

6. The embodied intelligent robot cell loading system based on imitation learning according to claim 1, characterized in that, The data acquisition module specifically includes: The synchronization control unit is used to generate a unified acquisition trigger signal and send it to the active arm, passive arm and RGB camera respectively to start the synchronous acquisition of control commands, joint status and RGB images; The data caching unit is communicatively connected to the active arm, the passive arm, and the RGB camera, and is used to receive and temporarily store control command data from the active arm, joint state data from the passive arm, and RGB image data from the RGB camera, respectively. The timestamp addition unit, connected to the data cache unit, is used to add timestamps to each set of control command data, joint status data and RGB image data temporarily stored in the data cache unit, based on the same system clock source, thereby generating a control command sequence, joint status sequence and RGB image sequence with timestamps.

7. The embodied intelligent robot cell loading system based on imitation learning according to claim 1, characterized in that, Also includes: The visualization verification module is used to receive the multimodal teaching dataset and perform visualization processing on the multimodal teaching dataset; The visualization verification module specifically includes a video generation unit and a curve drawing unit; The video generation unit is used to synthesize RGB image sequences from the multimodal teaching dataset into video files in chronological order for visual review of motion smoothness and scene integrity. The curve plotting unit is used to extract the joint state sequence and control command sequence from the multimodal teaching dataset, and plot the curves of joint angle change over time and control command change over time, for data analysis of joint motion continuity and command consistency.

8. The embodied intelligent robot cell loading system based on imitation learning according to claim 1, characterized in that, Also includes: Force sensors are mounted on the end gripper of the passive arm to monitor the gripping force in real time. The control module dynamically adjusts the gripping action based on feedback data from the force sensor.

9. The embodied intelligent robot cell loading system based on imitation learning according to claim 1, characterized in that, Also includes: The wheeled mobile platform, in which the active arm, passive arm, and RGB camera are all mounted, enables free movement within the working area.

10. A method for feeding battery cells using an embodied intelligent robot based on imitation learning, characterized in that, The method, applied to the embodied intelligent robot cell loading system based on imitation learning as described in any one of claims 1-9, comprises the following steps: S1. Receive remote operation input from the operator through the active arm and execute teaching actions for grabbing and placing battery cells; S2. The control command sequence, RGB image sequence obtained by the RGB camera, and joint state sequence fed back by the passive arm are synchronously collected through the data acquisition module during the teaching process, and timestamps are added to the control command sequence, joint state sequence and RGB image sequence respectively. S3. The data processing module performs timestamp alignment and missing data interpolation on the control command sequence, joint state sequence, and RGB image sequence to generate a continuously aligned multimodal teaching dataset. S4. The battery cell grasping action model is trained using the multimodal teaching dataset through the model training module. S5. The cell grabbing action model is optimized and compressed into a lightweight engine model through the model compression module. S6. In real-time loading operations, the control module calls the lightweight engine model to generate the desired joint state based on the real-time images captured by the RGB camera, and controls the passive arm to complete the gripping and loading of battery cells.

Images