Clothes folding method of clothes folding robot

Through robot recognition, modeling and deep learning training, the automated folding of clothing is achieved, which solves the problems of low efficiency and difficulty in consistency of manual folding, and improves the automation and intelligence level of clothing folding.

CN119077733APending Publication Date: 2024-12-06GREATER BAY AREA INST FOR INNOVATION HUNAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411274934.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-12
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Artificial folding clothing has problems such as low operation efficiency, high cost, difficulty in maintaining consistency and adapting to complex designs. Especially when facing special materials, it is difficult for ordinary people to master the required folding skills.

Method used

A method of folding robots is adopted, including clothing recognition, environmental modeling, path planning, robot hand-eye coordination training and deep learning training, to imitate human movements and reinforcement learning, generate accurate folding trajectories and strategies, and achieve automation and intelligence. Clothing folded.

Benefits of technology

It realizes accurate identification and automated folding of clothing, improves efficiency and consistency, adapts to complex environments and special materials, and reduces labor costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119077733A_ABST
    Figure CN119077733A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of robots, and particularly relates to a clothes folding method of a clothes folding robot, which has remarkable beneficial effects in the aspects of environment modeling, path planning, motion trail generation, real-time adjustment and the like. According to the method, the clothing is accurately identified, detailed steps of folding operation are recorded and generated, path planning and accurate control of the robot are executed, and real-time adjustment is carried out when an obstacle or a target point moves, so that automation and intelligentization of the clothing folding process are ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of robots, and in particular relates to a clothes folding method of a clothes folding robot. Background Art

[0002] In contemporary industrial production and daily life, automation and intelligence are increasingly playing a dual role in improving efficiency and living standards. Clothing folding, as a microcosm of automation tasks, is not only part of daily household chores, but also an indispensable part of clothing manufacturing and retail. This work seems simple on the surface, but in fact it involves precise control of spatial positioning, meticulous operation procedures, and flexible decision-making. With the continuous evolution of robotics technology, especially the significant progress in machine vision, machine learning, and manipulator control, the application of robots in automatic folding of clothing has become a new focus of research. With the help of advanced technologies such as imitation learning and reinforcement learning, these robots can observe and simulate human folding movements to automate the clothing folding process. Clothing folding.

[0003] Limitations and challenges of manual folding

[0004] Although manual folding has obvious advantages in terms of operational flexibility and adaptability to different situations, its inherent limitations cannot be ignored. First, long-term manual operation can easily lead to operator fatigue, which not only reduces work efficiency but also may increase the chance of errors. Secondly, in the current context of rising labor costs, relying on the large amount of manpower required for manual folding of clothing is becoming increasingly uneconomical. In addition, manual operation has difficulties in maintaining consistency and standardization of operations, which is particularly critical for business operations because consumers always expect the appearance of the clothes they buy to be neat and uniform. Finally, when faced with clothing made of special materials or with complex designs, the specific folding skills required are often beyond the capabilities of ordinary people without professional training. Summary of the invention

[0005] The purpose of the present invention is to provide a clothing folding method for a clothing folding robot

[0006] In order to achieve the above technical objectives, the technical solution adopted by the present invention is as follows:

[0007] A clothes folding method using a clothes folding robot comprises the following steps:

[0008] Step 1: Identify the clothes and record the folding process;

[0009] Step 2: Model the environment and plan the path;

[0010] Step 3: Train and calibrate the robot’s hand-eye coordination;

[0011] Step 4: Perform inversion training on the robot.

[0012] The step 4 and later also include robot deep learning training, and the specific steps are as follows:

[0013] Step 1: The robot imitates human actions to learn;

[0014] Step 2: Robot imitation reinforcement;

[0015] Step 3: Imitation and implementation of the robot's special function of folding clothes.

[0016] The specific method for the robot to imitate human actions to learn is:

[0017] (1) Calculate the similarity index;

[0018] (2) Optimize the total loss function;

[0019] (3) Perform GAN inversion processing;

[0020] (4) Optimize and adjust the model;

[0021] (5) Evaluate the success confidence of the model;

[0022] (6) Perform T cross entropy method on the model.

[0023] The specific method of the robot imitation reinforcement is:

[0024] (1) Collection and operation demonstration;

[0025] (2) Decompose the tasks;

[0026] (3) Enhanced demonstration in a simulated environment;

[0027] (4) Conduct hierarchical strategy learning;

[0028] (5) Enhance and learn the data;

[0029] (6) Using humans to perform residual learning in the loop.

[0030] The specific method of imitating and realizing the robot's special function of folding clothes is as follows:

[0031] (1) Perform prior extraction and conversion;

[0032] (2) Time alignment of learning content;

[0033] (3) Generate trajectories;

[0034] (4) Perform GAN inversion initialization training;

[0035] (5) Perform interactive optimization;

[0036] (6) Further strategy learning;

[0037] (7) conduct hindsight interactive contrast loss;

[0038] (8) Perform temporal monotonic loss learning.

[0039] Compared with the prior art, the present invention has at least the following advantages:

[0040] The robot clothing folding system of the present invention has significant beneficial effects in terms of environmental modeling, path planning, motion trajectory generation and real-time adjustment. It can accurately identify clothing, record and generate detailed steps of folding operations, perform path planning and precise control of the robot, and make instant adjustments when encountering obstacles or the target point moves, thereby ensuring the automation and intelligence of the clothing folding process. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The present invention is further illustrated by means of the non-limiting examples given in the accompanying drawings.

[0042] Figure 1 It is a schematic diagram of the system flow of the present invention.

[0043] Figure 2 It is a detailed framework diagram of the YOLOv4 model architecture of the present invention, showing the internal structure and components of the model.

[0044] Figure 3 It is a schematic diagram of the user interface (UI) of the present invention, showing the operation interface of the robot interacting with the user when generating a clothing folding sequence. DETAILED DESCRIPTION

[0045] In order to enable those skilled in the art to better understand the present invention, the technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0046] like Figure 1-3 As shown, a clothes folding method using a clothes folding robot comprises the following steps:

[0047] Step 1: Identify the clothes and record the folding process;

[0048] Step 2: Model the environment and plan the path;

[0049] Step 3: Train and calibrate the robot’s hand-eye coordination;

[0050] Step 4: Perform inversion training on the robot.

[0051] 2. The specific process of step 1 is as follows:

[0052] 1.1. Collect and process clothing image information:

[0053] (1) Locating clothing marks: Before performing the folding operation, the key folding points of the clothing are first located using marks. This method uses green stickers to clearly mark the key folding areas on the surface of the clothing to ensure that these marks are clearly visible when the image is captured;

[0054] (2) Capturing clothing images: Using the robot’s binocular camera system to take a comprehensive picture of the clothing, ensuring that key markings are clearly visible in the image;

[0055] (3) Image processing process: Given the obvious color contrast between the artificial marks on the clothes and the background, the positions of these marks can be directly identified through image processing technology. The detailed steps include:

[0056] (3.1) Color space conversion: convert the image from RGB color mode to HSV mode to facilitate color recognition and segmentation;

[0057] (3.2) Color threshold segmentation: On the basis of ensuring the green component in the H channel, high thresholds of the S and V channels are set at the same time to accurately extract the green mark;

[0058] (3.3) Morphological operation: Perform morphological operations on the binarized image to remove noise and connect the broken green areas;

[0059] (3.4) Contour detection: Use the contour detection algorithm to identify the contours of all green marks in the image;

[0060] (3.5) Marking drawing: Clearly mark the locations of all detected green marks on the original clothing image.

[0061] 1.2. Record the robot's folding action:

[0062] Records of key folding points:

[0063] (1) Folding recognition: Using the recognition algorithm in step 1, the clothes are monitored in real time during the folding process to identify the key folding points of the clothes.

[0064] (2) Coordinate capture: Once the key folding point is identified, its exact position in the coordinate system is immediately recorded.

[0065] (3) Point marking: Each key folding point is clearly marked on the image for subsequent review and analysis.

[0066] 1.3. Recording of folding action time: Record the start and end time of each folding action, and determine the duration of the action by calculating the difference between the two time points. The present invention requires detailed recording of the execution time of each folding step.

[0067] 1.4. Recording the distance between folding points: Record the distance between all key folding points. The specific methods include:

[0068] (1) Obtain the coordinates of the folding point: use the coordinates recorded in step 3.3.1;

[0069] (2) Calculate the distance: Use the Euclidean distance formula to calculate the distance between two points;

[0070] (3) Record distance: record the distance between each pair of folding points;

[0071] 1.5. Generate a sequence of folding steps:

[0072] A list of folding steps is compiled based on the recorded information, including but not limited to the key folding point positions, operation duration, and robot arm motion trajectory, and a folding process sequence arranged in order is generated based on the compiled data;

[0073] 1.6. Submit for review and execute operations:

[0074] (1) Visual display: The generated folding step sequence is displayed to the user in a list format through the user interaction interface. The list includes key information such as the specific location of each folding point, the time required, and the order of operations. By integrating distance information, the folding path is marked and connected to assist the user in verifying the entire folding process, thereby improving the accuracy and efficiency of the operation.

[0075] (2) Execution authorization: After verifying that the list is correct, the user can authorize the robot to start the folding task;

[0076] (3) Record keeping: After each folding operation is completed, the operation sequence is saved and archived for future reference and reuse.

[0077] The specific process of step 2 is as follows:

[0078] 2.1 Environment modeling: Divide the three-dimensional space into grids, each grid point represents a node, the connection between nodes is represented by edges, and the weight of the edge (w) is based on the distance between nodes;

[0079] 2.2 Target point sequence: According to the target point position sequence generated in step 2, these points are regarded as special nodes in the graph;

[0080] 2.3 Dynamic obstacle detection: Detect obstacles in the environment in real time, and add obstacle nodes at the corresponding positions in the graph to increase their costs;

[0081] 2.4 Cost update: When the target point or obstacle position changes, update the cost of the relevant nodes;

[0082] 2.5 Heuristic function: Since the present invention involves image recognition and robot control in three-dimensional space, the Euclidean distance is used as the heuristic function, and its expression is as follows:

[0083]

[0084] Among them, (x n ,y n ,z n ) is the coordinate of node (n), (x g ,y g ,z g ) are the coordinates of the target point.

[0085] The specific process of step three is as follows:

[0086] (1) Environmental preparation: Ensure that the workbench provides sufficient operating space for the robot arm and that there are no obstacles within the range of motion of the robot arm so that the robot can operate flexibly in a variety of postures;

[0087] (2) Image acquisition: Operate the camera on the robotic arm to capture images of the clothing on the workbench from multiple angles to ensure the clarity of image details;

[0088] (3) Edge detection: Perform edge detection on the clothing image to obtain the pixel-level coordinates of the clothing edge;

[0089] (4) Feature matrix construction: Combine the edge coordinates of the clothing with the known clothing feature data to calculate the clothing feature matrix to provide data support for subsequent folding work;

[0090] (5) Robot arm calibration: Compare the feature matrix of the clothing with the motion trajectory of the robot arm, and accurately calibrate the motion path of the robot arm to ensure that it can accurately grasp and manipulate clothing;

[0091] (6) Posture adjustment: For each step of the operation, the motion trajectory and posture of the robot arm are adjusted to meet the predetermined folding plan;

[0092] (7) Hand-eye coordination: Align the end effector of the robot arm to the position of the clothing, use the camera to capture the image, and apply image processing technology to determine the position information of the clothing in the camera coordinate system. This information will be input into the robot arm control system to achieve precise control based on real-time image data;

[0093] (8) Parameter optimization: All relevant operating parameters are carefully optimized, including the camera calibration parameters, the robot arm's motion trajectory parameters, and the hand-eye coordination transformation matrix, in order to improve the accuracy and operational efficiency of clothing folding.

[0094] The specific process of step 4 is as follows:

[0095] (1) Prior extraction and conversion: Use 3D human key point recognition technology to capture action key point data, use object bounding box detection technology to obtain the spatial position and size attributes of the object, convert video and image prior information into feature vectors, locate the markers on the clothing, capture images, process the images to identify the markers, and use the YOLOv4 algorithm for target detection;

[0096] (2) Temporal alignment: Self-attention and cross-attention mechanisms are used to ensure the temporal synchronization between actions and objects in the video sequence, maintain the temporal order of task execution, and ensure that the model gradually absorbs each stage of the task.

[0097] (3) Trajectory generation: The TrajGAN generator creates action trajectories, and the discriminator evaluates the similarity with real actions. Real-world interaction data is collected, and the TS-CEM algorithm is used to adjust the trajectory. The behavior cloning technology is used to cultivate high-level strategies to select subtask sequences, and low-level strategies execute subtasks. Key folding points, action time, and distance between points are monitored and recorded in real time.

[0098] (4) GAN inversion initialization: fix the generator parameters, find the latent vector that matches the environment variables through optimization, and design the loss function so that the model learns from failures and enhances the understanding of the task logic.

[0099] The step 4 and later also include robot deep learning training, and the specific steps are as follows:

[0100] Step 1: The robot imitates human actions to learn;

[0101] Step 2: Robot imitation reinforcement;

[0102] Step 3: Imitation and implementation of the robot's special function of folding clothes.

[0103] The specific method for the robot to imitate human actions to learn is:

[0104] (1) Calculate the similarity index;

[0105] In actual application scenarios, the calculation of the similarity index helps guide the robot to identify key objects and ensure the accuracy and efficiency of task execution. When learning task priors, the following formula is used to calculate the similarity index between vocabulary and item categories:

[0106] Similarity Index=∣W∩I∣∣W∪I∣

[0107] Where: (W) represents the vocabulary set in the verbal instruction (I) represents the set of object categories in the scene. W∩I represents the intersection of the vocabulary and the set of object categories, that is, the number of elements contained in both. W∪I represents the union of the vocabulary and the set of object categories, that is, the total number of all unique elements in both.

[0108] The intersection W∩I represents the number of direct matches between the vocabulary and the item category, while the union W∪I represents the number of all possible unique items. By calculating the ratio of the intersection to the union, the degree of overlap between the two is obtained. Scale: This ratio ranges between [0,1]. A similarity index of 1 indicates exact match (all instruction vocabulary matches the item category), while a similarity index of 0 means that there are no common elements between the two sets, that is, there is no match.

[0109] Using the similarity index, the robot can more accurately identify and select objects that are closely related to the task at hand. In practical operations such as robot grasping and placing, this significantly improves the accuracy and efficiency of task completion. When faced with changing task environments, the similarity index also serves as a key tool for the robot to continuously self-adjust and learn. It enables the robot to quickly adapt to different environmental conditions and optimize its behavioral strategies to ensure that each execution can efficiently achieve the task goals.

[0110] (2) Optimize the total loss function;

[0111] Constructing the overall loss function. In the training phase of the model, the overall loss function plays a vital role. It combines the temporal cycle consistency loss, the post-hoc interaction contrast loss, and the temporal monotonic loss to comprehensively improve the performance of the model.

[0112] The role of the temporal cycle consistency loss is to ensure that the alignment between video frames remains stable and reliable, thus providing a solid prediction foundation for the model.

[0113] The post-hoc interactive contrast loss enhances the model’s ability to identify failures, allowing the robot to learn and improve itself when it encounters errors.

[0114] The temporal monotonic loss ensures the temporal sequence of the action execution process and guarantees the coherent execution of the action.

[0115] The combined application of these three loss functions gives the robot the ability to dynamically adjust and optimize strategies during the learning process, enabling it to flexibly respond to diverse task requirements.

[0116] In the model training phase, the weighted overall loss function is introduced to fine-tune the model:

[0117] Consideration of temporal cycle consistency loss. The purpose of this loss mechanism is to ensure that the model's prediction results are consistent on the time axis when processing time series related tasks. Taking motion prediction or trajectory generation as an example, the model's predictions need to maintain logical consistency at consecutive time points. If the prediction at any time point deviates from the predictions at the previous and next time points, it may cause instability or distortion of the model output. Temporal cycle consistency loss ensures that the model's predicted trajectory or data on the time series maintains a high degree of consistency by measuring and reducing this prediction deviation.

[0118] Application of post-interaction contrast loss, which is a loss function that evaluates the consequences of events or interactive behaviors. Its core function is to quantify the differences in the model's processing of the same task after different interactions. This mechanism significantly improves the model's ability to identify multiple interaction consequences and enhances the model's performance in complex interactive tasks. Taking the robot's task execution as an example, different operation steps may lead to different execution results. The post-interaction contrast loss enables the model to identify and distinguish the pros and cons of these results, and then optimize the robot's behavior strategy.

[0119] The introduction of time monotonicity loss, the introduction of time monotonicity loss function is to ensure that the model output shows monotonic characteristics on the time axis, that is, the prediction results should change in an orderly manner over time. This avoids reversals or confusion in time series predictions. By imposing monotonicity constraints on the model's time series predictions, the time monotonicity loss function strengthens the logic and coherence of the prediction, which is particularly critical for the control decision-making link in task execution.

[0120] Weighted total loss function formula:

[0121] The weighted total loss function (L total )

[0122] = weighting factor (γ tcc )*Loss function (L tcc )+weighting factor (γ nic )

[0123] *Loss function (L nic )+weighting factor (γ mon )*Loss function (L mon )

[0124] By optimizing the model using a weighted total loss function, we can comprehensively consider key factors such as temporal coherence, interaction effectiveness, and the orderliness of the time series. By adjusting the weight coefficients, the model can flexibly meet the specific needs of different tasks and ensure that it can meet the preset goals in all performance indicators. This strategy not only deepens the model's understanding of the task process, but also significantly improves the model's performance in handling complex tasks and adapting to changing application environments.

[0125] (3) Perform GAN inversion processing;

[0126] GAN inversion technology plays a core role in the robot learning process. By optimizing the objective function, this technology can quickly capture the initial trajectory in the environment, providing the robot with a near-optimal starting point. This not only greatly improves the learning efficiency, but also ensures the accuracy and practical feasibility of the trajectory by reducing the difference between the environmental variables and the contact points of the generated trajectory. In the process of adapting to the new environment, GAN inversion technology gives the robot the ability to quickly adjust its behavior, enabling it to agilely adapt to changes in the environment. When adapting to new scenarios, use GAN inversion technology to quickly initialize the trajectory:

[0127] Linv=∥V E -C(τ)∥2

[0128] This optimization objective is used to minimize the distance between the environment variables and the contact points of the generated trajectory.

[0129] Where: V E represents the representation of the environment variables (usually a feature vector extracted from the environment). C(τ) represents the representation of the trajectory (τ) generated by the generative model C (usually also a feature vector). The goal is to minimize the environment variable V E The distance between the generated trajectory contact point C(τ). This ensures that the generated trajectory is not only highly consistent with the environmental characteristics, but also can be efficiently adapted to the current environment.

[0130] Implementation process of GAN inversion technology

[0131] First, extract the necessary features from the environment to form the environment variable V E These features can include geometric information of the environment, location of objects, dynamic changes, etc.

[0132] ②Use the generator (G) in the generative adversarial network (GAN) to generate candidate trajectories (τ) based on the initial random noise vector or initial trajectory. Then, the generated trajectory (τ) is input into the model C, mapped to the feature space, and the feature representation C(τ) is obtained.

[0133] ③ By optimizing the target Linv, adjusting the generator parameters or initial vector, VE Minimize the distance between and C(τ). This process often involves back propagation and gradient descent algorithms.

[0134] (4) Optimize and adjust the model;

[0135] GAN inversion technology can quickly produce initial trajectories that match the new environment, allowing the robot to quickly adapt to changes and shorten the time to plan trajectories from the basics, greatly improving the efficiency of the learning process.

[0136] Step implementation: By minimizing the gap between environmental variables and trajectory contact points, it ensures that the generated trajectory is both logical and feasible, effectively reducing invalid attempts during the learning process and reducing the risk of collision or failure.

[0137] (5) Evaluate the success confidence of the model;

[0138] Success confidence is a key indicator for evaluating the model's belief in the possibility of task completion, and its calculation is based on the application results of GAN inversion technology. This indicator maps the robot's intrinsic evaluation of the success of strategy execution and provides a basis for deciding whether to perform a specific task. High confidence means that the robot can confidently carry out the task, while low confidence indicates that the robot needs further learning or strategy adjustment. Considering success confidence improves the robot's adaptability and robustness when dealing with uncertain situations. Evaluating the possibility of task success:

[0139] θ success =21+eLinvθ success =1+eLinv2

[0140] This confidence is used to decide whether to execute the task or reject it in advance.

[0141] Among them, Linvθ is the optimization loss value obtained from the GAN inversion process, which reflects the environmental variables (V E ) and the contact point of the generated trajectory (C(τ)). The smaller this distance is, the closer the generated trajectory is to the optimal solution, and therefore the higher the corresponding confidence.

[0142] The exponential function eLinvθ is used to convert the optimization loss Linvθ into a format suitable for confidence calculation. Since the exponential function grows very fast, the loss far from zero will be quickly amplified, which will significantly affect the confidence. The denominator 1+eLinv2 in the calculation formula has a significant impact on the confidence. The denominator term in the calculation process helps to control the result within a smaller range.

[0143] By formula 21+eLinvθ success , success confidence θ successThe range of is limited to [0,2], ensuring that the confidence is closely related to the actual loss, providing strong support for the decision-making process.

[0144] Success confidence θ success Plays a vital role in the decision-making process:

[0145] Success confidence is a key indicator for measuring the likelihood of a robot completing a task. High confidence indicates that the strategy and trajectory are close to optimal, allowing the robot to confidently perform the task; low confidence indicates that the strategy needs to be adjusted to reduce the risk of failure. As a real-time evaluation tool, success confidence helps the robot quickly adapt to environmental changes, make reasonable decisions, and improve its adaptability and flexibility in uncertain environments. Overall, success confidence provides strong decision-making support for the robot, enhancing its stability and task execution efficiency.

[0146] (6) Perform T cross entropy method on the model.

[0147] The T-Cross Entropy Method (TS-CEM) is a key technology in the policy fine-tuning process. It uses a reward function to evaluate the effectiveness of policy execution and provide guidance for the model's learning process. The reward function measures the difference between the robot's actions and the human demonstration actions, providing immediate feedback to the robot. Based on this feedback, the robot can continuously learn and self-adjust during task execution to gradually reduce the difference with human demonstrations. By adopting TS-CEM, the robot can not only obtain preliminary strategies based on imitation learning, but also further optimize these strategies to improve the completion rate of tasks.

[0148] In the process of strategy learning, we use an innovative reward mechanism to evaluate and optimize the execution effect of the strategy. This reward function not only takes into account the quality and efficiency of task completion, but also integrates the stability and adaptability of the strategy execution process, providing comprehensive feedback for the model and guiding its evolution towards a more efficient strategy.

[0149] Rew=∑t=0T∥zrt-z ht ∥2

[0150] This reward function is based on the alignment distance calculated by IAAformer and is used to guide the optimization of the policy.

[0151] In general, the execution process of using TS-CEM can be divided into the following steps:

[0152] 1. In the initial stage, the robot learns the initial strategy by imitating human example actions.

[0153] 2. During the policy execution process, the reward function is used to evaluate the effectiveness of the current policy. The reward function measures the difference between the robot's actions and the human example actions, and the function value is usually inversely proportional to the difference between the two.

[0154] 3. Feedback the difference between the robot’s actions and human example actions (represented by the value of the reward function) for policy correction.

[0155] For example, during training, parameters might be adjusted to minimize negative rewards.

[0156] The function given here is a measure of the actual action zrt and the theoretically optimal action z ht The sum of squared Euclidean distances between the differences between , which is used as the reward function. In this case, working to minimize this function may lead to better policy performance.

[0157] The specific method of the robot imitation reinforcement is:

[0158] (1) Collection and operation demonstration;

[0159] Human movements captured by an external RGB-D camera are converted into corresponding movements of the robot. This conversion process relies on computer vision and deep learning technology to ensure accurate replication of human movements. In actual operation, human movements captured by the RGB-D camera, including the coordinates of key parts such as wrists, elbows, and knees, are converted into digital signals, which are essential for capturing the core elements of the movement. Given the essential differences between the movement mechanisms of robots and humans, complex algorithms must be used to map the key points of the human body into the robot's motion framework to ensure the consistency of robot movements with human movements. Different systems may use different conversion algorithms, but the core goal is to use deep learning and artificial intelligence technology to accurately convert human movements into a set of instructions that can be executed by the robot. The application of this technology enables robots to accurately imitate human movements and perform delicate tasks in many fields such as medicine, manufacturing, and services, significantly improving work efficiency and execution results.

[0160] (2) Decompose the tasks;

[0161] This strategy involves breaking down large and complex tasks into multiple smaller, more manageable subtasks. Each subtask has a clear start and end gesture. This logical division not only simplifies the understanding and execution of the task, but also significantly reduces the complexity of learning and processing the task.

[0162] In the field of robot learning, the practice of task decomposition begins with a thorough analysis of the overall goal. In essence, task decomposition is to split a complex task into several smaller, independently executable subtasks, each of which corresponds to a series of independent action steps. Drawing on the dynamic programming concept proposed by Bellman, the optimal completion of the entire task is achieved by optimizing each subtask one by one. In this process, each subtask is regarded as an independent problem unit, focusing on finding the local optimal solution, so as to gradually achieve the global optimal solution. This method enables the robot to learn and improve the execution strategy of each subtask in stages, which not only improves the learning efficiency, but also enhances the robot's adaptability to changes and unexpected situations during operation.

[0163] (3) Enhanced demonstration in a simulated environment;

[0164] When enhancing a trajectory, Gaussian noise is added to the key points of the trajectory using the following formula:

[0165] n x ~N(μ,σ 2 ), n y ~N(μ,σ 2 )

[0166] In order to achieve translation and flipping of the trajectory, the key points can be transformed accordingly in the coordinate space:

[0167] By adding a translation n in the x and y directions respectively x and n y to achieve.

[0168] [(x′, y′) = (x + t x , y+t y )]

[0169] For example, a horizontal flip can be achieved by inverting the values ​​on the x-axis.

[0170] [(x′, y′) = (-x, y)]

[0171] By analyzing the position and orientation of the robot end-effector in the simulation environment, we are able to capture the key points of the trajectory. These key points are then used to construct a polynomial trajectory model to ensure that the generated trajectory is smooth and evenly distributed.

[0172] Here are the specific steps:

[0173] The position and orientation information of the end effector is extracted from the data collected when the robot performs the task. These data points are used as key points, and a continuous and smooth trajectory is generated by fitting a polynomial (such as a quadratic or cubic polynomial). In addition, to ensure the smoothness and uniformity of the trajectory, a uniform sampling technique is used to extract a series of equally spaced points from the polynomial trajectory.

[0174] The implementation process of data augmentation involves extracting key points of the trajectory from the recorded data and using these key points to fit a polynomial trajectory, such as using cubic spline interpolation. Next, uniform sampling is performed on the fitted trajectory to produce equally spaced trajectory points. Finally, Gaussian noise is introduced to these trajectory points, and necessary translation and flipping operations are performed to enhance the generalization ability of the model.

[0175] 1. Import necessary libraries: numpy, scipy.interpolate (for interpolation calculations), and matplotlib.pyplot (for data visualization) are used here.

[0176] 2. Define keypoints: Extract some sample keypoints from the robot recordings and save them in a numpy array.

[0177] 3. Polynomial trajectory fitting: Use spline interpolation (CubicSpline) to fit a smooth trajectory for key points.

[0178] 4. Uniform sampling: uniformly use “num samples " samples, num samples Here it is set to 100.

[0179] 5. Add noise: Add Gaussian noise to the trajectory to simulate the uncertainty and noise in the real world. The standard deviation (sigma) is set to 0.1.

[0180] 6. Translation and flipping: The trajectory after adding noise is translated (by adding a constant vector) and flipped (by multiplying by [-1, 1]).

[0181] 7. Visualization results: Use the matplotlib library to plot and display the original trajectory, noisy trajectory, translated trajectory, and flipped trajectory.

[0182] In general, this program generates a polynomial trajectory, then performs a series of modifications to that trajectory (adding noise, translating, and flipping), and visualizes the results.

[0183] The translation and flipping of the trajectory is achieved by transforming the key points in the coordinate space.

[0184] In a simulated environment, we enhance the collected demonstrations and improve the robustness of the learned policy by performing data augmentation at the trajectory level. We first extract keypoints by reading the position and orientation of the robot end-effector and fit a polynomial trajectory using these keypoints. To ensure smoothness and uniform distribution of the trajectory, we apply a uniform sampling method. Keypoints are extracted from the recorded data, a polynomial trajectory is fit, and uniform sampling is applied to generate smooth and uniformly distributed trajectories.

[0185] (4) Conduct hierarchical strategy learning;

[0186] The VITAL system adopts a hierarchical policy learning approach that uses behavior cloning (BC) technology to cultivate high-level policies to select appropriate subtask sequences and low-level policies to perform the execution of each subtask. This hierarchical policy learning approach not only enhances the efficiency and adaptability of task processing, but also gives the robot the ability to perform multi-step tasks under complex environmental conditions. In this hierarchical architecture, high-level policies are responsible for global planning and task allocation. High-level policies can be implemented through state machines or dynamic learning models based on deep learning or reinforcement learning. As a deterministic model based on predefined rules, state machines perform well in highly structured and predictable environments. The learning model can train dynamic policies based on historical data and current status, enabling high-level policies to make reasonable decisions when facing more complex or unknown environments. Low-level policies uniformly manage the execution of multiple subtasks, each of which is trained through behavior cloning. This approach enables low-level policies to learn specific operation steps based on demonstration data, such as accurately grasping and placing objects, or moving and avoiding obstacles in a changing environment. Behavioral cloning trains the network model through demonstration data, ensuring the reliability and consistency of low-level strategies when executing each subtask, reducing the trial-and-error cost during the learning process. In addition, low-level strategies trained with demonstration data can share knowledge and skills between different tasks, improving overall execution efficiency. Overall, the hierarchical strategy learning method of the VITAL system combined with behavioral cloning provides an efficient and flexible solution for the decomposition and execution of complex tasks. High-level strategies are responsible for making global plans and making task selections, while low-level strategies focus on specific operation execution, enabling robots to achieve efficient and accurate task completion in diverse and complex environments.

[0187] (5) Enhance and learn the data;

[0188] Gaussian noise, as a common statistical noise, has a numerical distribution that conforms to the normal distribution and is usually defined as a normal distribution with zero mean and a certain standard deviation.

[0189] Introducing Gaussian noise into trajectory data can reproduce the uncertainty and random errors in the real environment, such as noise interference from sensors or random changes in the environment. This helps the model to be more adaptable to random fluctuations in the data during training, enhances the robustness of the model to noise and small disturbances, and thus enhances the generalization and stability of the model. Data translation, as another means of data enhancement, generates a dataset that is slightly different from the original data by moving the trajectory data in space, such as horizontally or vertically, thereby increasing the diversity of the data.

[0190] This technology helps the model learn trajectory features at different spatial locations, improving the model's adaptability and robustness to different positions and coordinate transformations.

[0191] Flipping technology is particularly suitable for trajectory data processing, including horizontal and vertical flipping. By changing the perspective of the data, new trajectory data is generated, enabling the model to learn and adapt to trajectory features in different directions, thereby enhancing the robustness of the model and its wide application adaptability.

[0192] The comprehensive use of these data enhancement techniques can effectively enrich the diversity of the data set, enabling the model to learn and adapt to changing environments and conditions more effectively during training, significantly improving the model's generalization ability and overall performance.

[0193] Based on the above theory, we enhance the trajectory data by introducing Gaussian noise, performing translation and flipping operations, and resampling new starting points.

[0194] The cumulative arc length of the trajectory is calculated using the following formula to ensure uniform distribution of sampling points:

[0195]

[0196] The feasibility of the enhanced trajectory is verified in the digital twin environment by the following formula:

[0197] Linv=∥V E -C(τ)∥2

[0198] Among them, V E are environment variables and C(τ) are the contact points for generating trajectories.

[0199] (6) Using humans to perform residual learning in the loop.

[0200] When the robot task execution fails to achieve the expected goal, the human operator will intervene to provide the necessary corrections. The measurement and application of the corrections can be scaled and adjusted by the following formula:

[0201] Delta Position

[0202] =empirically determined factor×3D delta position changeDeltaPosition

[0203] =empirically determined factor×3D delta position change

[0204] The empirically determined factor is a scaling factor less than 0.1 determined experimentally, the purpose of which is to reduce the movement speed of the robot to avoid possible risks.

[0205] 1. We need a 3D trajectory of an object, which could be a numpy array containing x, y, z coordinates, where each row represents the position of the object at a certain moment.

[0206] 2. Use the formula ddelta positionchange = np.diff(trajectory,axis=0) uses the numpy diff function, which calculates the difference between every two adjacent elements in an array. This calculation is performed on the 0th axis of the trajectory array, that is, in the top-to-bottom direction. The result is a new array in which each element is the change in position of the object between a pair of consecutive moments. This is actually calculating the speed of the object, assuming that the time interval is 1 each time.

[0207] 3.empirically determinedfactor = 0.05 is a scaling factor we set. The factor (here 0.05) is determined empirically to reduce the average speed of the object in each time interval. In fact, this is to slow down the speed of the object at each time step.

[0208] 4. Use the formula delta position =empirically determinedfactor *delta positionchange By multiplying the scale factor, we are actually recalculating the object's velocity. This new velocity will be used to update the object's position in the next step.

[0209] 5. Use the formula adjusted trajectory =trajectory[:-1]+delta position Add the newly calculated velocity to the original position to get the new position of the object. trajectory[:-1] gets all rows except the last row

[0210] Correction data from humans in the loop is recorded and integrated using the following formula:

[0211] Rew=∑t=0T∥zrt-z ht ∥2

[0212] Among them, zrt and z ht are the predicted and actual trajectory points respectively.

[0213] Applying an empirical scaling factor to control the robot's movement speed helps reduce the risks that may be caused by rapid movement. At the same time, by optimizing the difference between the predicted trajectory and the actual corrected trajectory, the robot's adaptability and robustness in performing actual tasks have been significantly improved. In addition, combining data enhancement techniques such as Gaussian noise, translation, flipping, and resampling can greatly enhance the generalization ability of the model and improve its stability and performance in a variety of application scenarios.

[0214] The specific method of imitating and realizing the robot's special function of folding clothes is as follows:

[0215] (1) Perform a priori extraction and conversion;

[0216] Before the learning process starts, the first task is to extract key information from human demonstration videos. This involves using 3D human key point recognition and object bounding box detection technology to capture "human behavior priors" and "object attribute priors". The specific steps include the following:

[0217] ①Pre-knowledge extraction:

[0218] Before a machine learning model performs a task, prior knowledge extraction is an indispensable step. The so-called "human behavior prior" refers to the knowledge and information accumulated through observation and learning of human behavior habits. This type of information is crucial for robots to understand and imitate human movements. For example, with the help of three-dimensional human key point recognition technology, key point data of movements can be captured, such as the positioning and movement path of hands, feet and head. The "object attribute prior" focuses on information such as the shape, size, position and dynamics of the object, which is usually obtained through object bounding box detection technology.

[0219] ②Pre-knowledge transformation:

[0220] After information extraction, the next task is to convert these human-featured prior information (such as videos, images, etc.) into a format that machines can understand and learn, namely "learnable action priors". A common method is to convert these prior information into feature vectors or feature descriptors, covering the spatial position, size, color, texture and other attributes of objects, as well as the posture, speed, strength and other parameters of human movements. In this way, robots can use these feature data for learning and decision-making, and then achieve accurate imitation of human behavior and adaptation to environmental changes in actual operations.

[0221] In the process of extracting and converting prior information, the following processing steps are performed:

[0222] 1. Image processing and computer vision technology;

[0223] 2. Applications of deep learning and neural networks, including the application of convolutional neural networks (CNN) in image recognition, the application of recurrent neural networks (RNN) in time series data processing, and the latest self-attention mechanism and transformer networks in various tasks.

[0224] 3. Feature representation and data conversion technology, such as using encoders to convert multimedia data such as images, text or sound into dense vectors, or converting high-dimensional data (such as vocabulary) into low-dimensional continuous vector space through embedding learning.

[0225] (2) Time alignment of learning content;

[0226] Temporal synchronization is critical for deep understanding of video content. Using the Interaction-Aware Alignment Transformer (IAT), the model can accurately match corresponding frames in two video sequences that perform the same task, such as a human demonstration video and a robot execution video. This process combines self-attention and cross-attention mechanisms, allowing the model to gain insight into the interactive relationship between actions and objects in the video. Self-attention mechanism, the self-attention mechanism gives the model the ability to process sequence data, enabling it to focus on different positions within the sequence and generate an expression for each element based on this. This expression is calculated based on the interaction between the element and other elements in the sequence. The formula is expressed as:

[0227] Self-attention mechanism formula:

[0228] Self Attention(Q,K,V)=Softmax(QKTd)V

[0229] Where Q represents the "query" vector, K represents the "key" vector, V represents the "value" vector, and d is the corresponding dimension. The dot product of attention calculates the correlation between each element in the sequence, and then converts it into a probability distribution through the Softmax function, which defines how much attention to give to position j when calculating the output representation of position i.

[0230] Cross-attention mechanism formula:

[0231] CrossAttention(Qobj,Kaction,Vaction)

[0232] =Softmax(QobjKactionTd)VactionCross

[0233] -Attention(Qobj,Kaction,Vaction)=Softmax(dQobjKactionT)Vaction

[0234] Here, Qobj is the query vector of the object, Kaction and Vaction are the key vector and value vector of the action. In this way, the model can learn the interaction between actions and objects in the video.

[0235] Self-attention and cross-attention mechanisms play a key role in video content parsing and temporal synchronization. Using the self-attention mechanism, the model can identify the relationship between elements in the input sequence and then construct a semantically rich expression. In the field of video content analysis, this mechanism is particularly effective, enabling the model to capture and maintain the temporal dependencies in the video sequence, thereby extracting the key information of the video. In addition, the introduction of the cross-attention mechanism further enhances the model's ability to understand the connection between objects and actions. For example, the model is now able to associate the action of "picking up clothes" with the object entity "clothing". The Interaction-Aware Alignment Transformer (IAT), which combines self-attention and cross-attention, performs well in temporal synchronization and can effectively match the key frames of the same action in different video sequences, significantly improving the efficiency of the robot's imitation of human demonstration learning tasks.

[0236] In this embodiment, the time alignment method is specifically as follows:

[0237] 1. First, perform image preprocessing: preprocess the captured images, such as color space conversion, denoising, etc.

[0238] 2. Feature extraction: In addition to the image itself, the shape and position information of the area detected by the contour are also extracted as one of the features of target detection. The specific method is as follows:

[0239] (1) Contour feature extraction: converting contour information into feature vectors. The present invention uses descriptors such as shape context to represent contour features. The specific steps are:

[0240] (1.1) Obtaining contour information: The contour can be represented as a set of two-dimensional coordinate points (x i ,y i ), where (i) is the index of the point on the contour.

[0241] (1.2) Normalized contour: The contour is normalized by translating it to the center of the image by calculating its centroid so that it has the same starting point and scale. Specifically, the centroid (C x ,C y );

[0242] Then each point (x i ,y i ) pans to the center of the image

[0243]

[0244]

[0245] (1.3) Applying shape context descriptor: The shape context descriptor is used to represent the shape information of the contour. The specific steps are:

[0246] (1.3.1) Point set matching: The normalized contour is represented as a set of points, each point (x i ,y i ) represents a sampling point on the contour.

[0247] (1.3.2) Construct descriptor: For each contour point (x i ,y i ), calculate its difference with other points (x j ,y j )'s relative polar coordinates ((r {ij} ,θ {ij} )), where (r {ij} ) is the point (x i ,y i ) to point (x j ,y j ) distance, (θ {ij} ) is the angle between two points. A distance-angle histogram is calculated.

[0248] (1.3.3) Descriptor representation: Connect the values ​​of all bins of the histogram in sequence into a vector as the feature vector of the contour.

[0249] (2) Image feature extraction: Extract feature vectors from images.

[0250] (3) Feature vector merging: Merge the contour feature vector and the image feature vector into a new feature vector.

[0251] 3. Perform object detection: Using the results of feature extraction, use the YOLOv4 object detection algorithm to locate and identify key folding positions in the image.

[0252] 4. Verify the results: Verify the results of target detection to ensure the accuracy and stability of the results. Use the results of contour detection as an auxiliary reference indicator to assist in verifying the results of target detection.

[0253] 5. Map the detection results: Map the key points obtained by target detection into the depth map. The depth map contains the depth information of each pixel in three-dimensional space. Mapping the results of contour detection and target detection into the depth map can obtain the accurate position information of the clothes, and assist the robot in determining the depth required to fold the clothes and other important information. Specifically, for each pixel within the key point, the corresponding depth value is obtained from the depth map according to its position in the original image; for the depth value that does not exist in the depth map, it is estimated by interpolation processing.

[0254] (3) Generate trajectories;

[0255] TrajGAN is a trajectory generation model that generates trajectories that imitate actions in videos. These trajectories serve as guidance for the robot's actions and are a key output in the learning process. Generating trajectories is an important part of enabling robots to take effective actions in real environments. In learning tasks from videos, this usually involves parsing and imitating actions in the video and converting these actions into actual trajectories that the robot can perform.

[0256] TrajGAN (Trajectory Generation Model) is a tool to achieve this task. GAN (Generative Adversarial Network) is an important model in the field of deep learning. It consists of two parts, the generator and the discriminator, and learns by the confrontation between the two.

[0257] In the TrajGAN model:

[0258] TrajGAN, an advanced trajectory generation model, is dedicated to creating paths that simulate video actions. These paths not only guide the robot's actions, but also serve as the core output of the entire learning process.

[0259] It is critical for robots to perform effective actions in real-world environments. When learning a task, the robot needs to interpret the actions in the video, imitate them, and convert them into actual executable paths.

[0260] TrajGAN is a tool that uses the principles of generative adversarial networks (GANs), which consist of generators and discriminators that promote learning through competition. The generator, a deep neural network, accepts random noise vectors and creates new paths. The discriminator evaluates the similarity of these paths to the original human actions and encourages the generator to optimize the path through feedback.

[0261] This adversarial training enables the generator to continuously improve and generate continuous paths that are closer and closer to real actions. Specifically, the model is able to generate joint position, velocity, and acceleration data based on human demonstration videos, which are concatenated into a complete robot action path.

[0262] In addition, the model can learn from limited samples and create diverse paths, thanks to the diversity of results brought by the noise vector input to the generator. As the training progresses, the generated paths gradually approach the real action, showing the powerful ability of GAN in trajectory generation. TrajGAN can also learn from a small number of samples and generate diverse trajectories, because the noise vector input to its generator can increase the diversity of the generated results. And, due to the characteristics of GAN, the generated trajectory can get closer and closer to the real action trajectory.

[0263] 1. Define the generator and discriminator: Both the generator and the discriminator use a fully connected network and introduce nonlinearity through the ReLU activation function. The generator receives a 100-dimensional random vector as input and outputs a fake sample with a dimension of 1. The discriminator receives a sample with a dimension of 1 as input and outputs a probability value indicating the authenticity of the sample.

[0264] 2. Instantiate the network and set training parameters: Instantiate the generator and discriminator, set the learning rate to 0.001, train for 200 cycles, and use the binary cross entropy loss function (BCELoss).

[0265] 3. Select the optimizer: Adam optimizer is used here to optimize the generator and discriminator.

[0266] 4. Train the generator and discriminator: For each batch of data, first train the discriminator. The optimizer first clears the gradient, then uses the real data and the fake data generated by the generator to calculate the real loss and fake sample loss respectively, performs back propagation, and finally executes the optimizer's step function to update the parameters. Secondly, train the generator. The optimizer first clears the gradient, the generator generates fake data, obtains the result through the discriminator, calculates the loss for back propagation, and finally updates the generator's parameters.

[0267] In this way, the Generative Adversarial Network (GAN) can be trained. Through this iterative process, the generator will learn to generate fake samples that are closer and closer to the real data, and the discriminator will improve its ability to distinguish between real and fake samples.

[0268] Among them, the generator and the discriminator are both simple fully connected networks. The generator receives a 100-dimensional random vector as input and outputs a 1D vector; the discriminator receives a 1D vector as input and outputs a probability value between 0 and 1, indicating the probability that the input vector is real data. Finally, by optimizing the loss function of the generator and the discriminator, we have realized the training process of GAN.

[0269] (4) Perform GAN inversion initialization training;

[0270] In order to adapt to new scenes, the rough trajectory is initialized from TrajGAN using GAN inversion technology. This process minimizes the distance between the environmental variables and the generated trajectory contact points through optimization. Since the actual environment is complex and changeable, for each new environment, the robot needs to generate an action trajectory that adapts to the environment. How to generate an initial action trajectory that is suitable for the new environment is a complex problem. To this end, we can use GAN inversion technology to get a new initialization trajectory from TrajGAN.

[0271] Specifically, the main goal of the GAN inversion technique is to find a latent vector under the premise of fixing the parameters of the generator (TrajGAN) so that the trajectory generated by the generator using this vector coincides with the environmental variables in some sense. This coincidence is usually measured by some form of distance or error, for example, in this formula, it is the Euclidean distance (L2 norm) between the generated trajectory and the environmental variables.

[0272] Here, our optimization goal is to minimize the distance in the formula:

[0273] Linv=∥V E -C(τ)∥2

[0274] Among them, Linv is the inversion loss, VE is the environmental variable (such as the position and size of the object), and C(τ) is the actual position of the generated trajectory in the environment, which is usually obtained through some sensor information. Simply put, we hope that the trajectory generated by the generator (where the object should be) is as close as possible to the actual position of the object in the environment. This process is an optimization problem, and various optimization algorithms, such as gradient descent, can be used to find the optimal input potential vector so that the generated trajectory best meets the environmental requirements. Using inversion technology, we can customize an initial trajectory for the robot that is suitable for the new environment, which not only lays the foundation for the robot to learn new tasks, but also steadily improves its operating performance and stability with continuous learning and optimization.

[0275] (5) Perform interactive optimization;

[0276] Interaction optimization is an iterative process of adapting to new scenarios. By collecting real-world interaction data, the model can learn how to adjust the trajectory to better adapt to the environment. The Trajectory Semantic Cross-Entropy Method (TS-CEM) is an optimization algorithm used to find trajectories that minimize the reward function. The reward function (Rew) is defined here as the generated trajectory z h,t and the true trajectory z r,t The sum of the Euclidean distances between them, that is:

[0277]

[0278] Among them, z r,t is the true trajectory at time (t), z h,t is the generated trajectory at time (t), (T) is the total number of time steps of the data, It is the square of the L2 norm, that is, the square of the Euclidean distance.

[0279] The construction of this reward function is based on a simple and intuitive concept: the closeness between the generated trajectory and the actual trajectory is the key to evaluating its quality. In the subsequent iterations, by continuously reducing the value of the reward function, the optimization algorithm can gradually adjust and improve the generated trajectory so that it can more accurately adapt to the characteristics of the new environment.

[0280] Trajectory semantics-based cross entropy method (TS-CEM) formula:

[0281] 1. Initialization: Initially, a set of trajectory samples of size N is randomly generated. Each sample trajectory z consists of continuous values ​​of d dimensions, forming an N*d matrix.

[0282] 2. Evaluation: For each trajectory sample z, input it into the reward function to calculate the corresponding Rew. In this way, N reward values ​​corresponding to N sample trajectories can be obtained.

[0283] 3. Sorting: Sort the N sample trajectories according to the reward value, take out the best sample trajectory corresponding to the probability p, and form a new set z best p is usually set to 0.1, which means that in each iteration, only the 10% trajectory samples with the highest reward are retained.

[0284] 4. Update: Get z by Gaussian fitting best The mean μ and variance σ of 2 , then with μ as the center, σ 2 is the variance, and is the new set of N sample trajectories.

[0285] 5. Iteration: Repeat the above 2-4 steps until the stopping condition is met (for example, the number of iterations reaches the limit, or the change in the reward function is less than a certain threshold, etc.).

[0286] Through the above iterative process, the TS-CEM algorithm can gradually optimize the trajectory, making it closer and closer to the true trajectory, thereby minimizing the value of the reward function. In practice, this method has been proven to achieve excellent performance in several complex experimental scenarios.

[0287] (6) Further strategy learning;

[0288] Strategy learning constitutes the ultimate link of the CIA framework. It combines reinforcement learning and imitation learning techniques to optimize the robot's action strategy and ensure that it can successfully complete tasks in unknown scenarios.

[0289] In the strategy acquisition phase, the robot first acquires a basic strategy through imitation learning, and then uses reinforcement learning to further adjust and improve this strategy to adapt to specific tasks and environmental requirements. In general, the strategy acquisition phase is a key step in converting the learning and adaptation results of the previous phase into specific actions.

[0290] (7) conduct hindsight interactive contrast loss;

[0291] In order to enhance the model's ability to identify substandard situations, a reflective interactive contrast loss mechanism is specially designed. This mechanism encourages the model to learn from failed experiences while analyzing successful cases, thereby deepening its insight into the task process. Reflective interactive contrast loss plays a vital role in model training, allowing the model to extract lessons from past successes, remain sensitive to failed cases, and deepen its understanding of task logic.

[0292] In many machine learning applications, models tend to focus on successful cases and ignore the information that failures can provide, which limits the model's ability to handle atypical situations. The introduction of reflective interactive contrast loss is to make up for this deficiency, so that the model can learn and grow even when facing failures. The calculation method of this loss function is as follows:

[0293]

[0294] In this formula, e s im(zi, m, zj, n) is used to express the interaction similarity λ between the model's i instance at time m and the j instance at time n t emp is a regularization parameter, sum n For example, and are the sum of the negative example set, the enhanced action set, and the enhanced observation set. t emp is the temperature parameter used to adjust the sharpness of softmax.

[0295] The introduction of reflective interactive contrast loss promotes the model's in-depth understanding of substandard situations, and its benefits are mainly reflected in two key areas: First, it significantly enhances the robustness of the model, allowing the model to respond appropriately when encountering atypical or failure scenarios. Second, it improves the expressiveness of the model and gives the model the ability to grasp the task logic at a deeper level. These improvements are crucial to improving the accuracy and effectiveness of the model in handling diverse and complex tasks and adapting to changing environments.

[0296] By carefully designing and incorporating a reflective interactive contrastive loss, the model is able to adopt a more comprehensive and in-depth learning approach to master and understand the task.

[0297] (8) Perform temporal monotonic loss learning.

[0298] The temporal coherence loss aims to maintain the coherence of the time series, ensuring that the model can gradually absorb each stage of the task in a logical order.

[0299] In many real-world application scenarios, the various stages of a task must be executed in a specific order. Any disorder in the order may lead to a complete failure of the task. Temporal Coherence Loss is a loss function designed to ensure temporal alignment consistency. It guides the model to gradually learn each stage of the task in the appropriate order by enforcing the consistency of the operation order. Specifically, the formula for temporal monotonic loss is:

[0300]

[0301] Among them, σ() represents the activation function, such as Sigmoid or ReLU, which is used to map the function value to the range of (0,1) so that they have good expressiveness; M is the total number of stages, indicating that the task can be decomposed into M stages; MI() represents the mutual information, which is used to measure the correlation between the action in a specific stage and the action in all stages. This loss function is to minimize the sum of the differences in the mutual information between the actions in each stage and the actions in the current stage.

[0302] This loss function helps ensure that the model performs operations in the proper order. First, it ensures that the model follows the correct order of tasks, thereby preventing failures due to incorrect order. Second, by using mutual information as an evaluation criterion, it enhances the model's insight into the relationship between different stages, thereby improving the model's expression accuracy. Finally, since the loss function is based on minimizing the sum of mutual information differences, it helps prevent the model from being overly dependent on operations in a specific stage, thereby enhancing the model's generalization performance.

[0303] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A clothing folding method using a clothing folding robot, characterized in that: The following steps are involved: Step 1: Identify the clothes and record the folding process; Step 2: Model the environment and plan the path; Step 3: Train and calibrate the robot’s hand-eye coordination; Step 4: Perform inversion training on the robot.

2. A clothes folding method using a clothes folding robot according to claim 1, characterized in that: The specific process of step one is as follows: 1.

1. Collect and process clothing image information: (1) Locating clothing marks: Before performing the folding operation, the key folding points of the clothing are first located using marks. This method uses green stickers to clearly mark the key folding areas on the surface of the clothing to ensure that these marks are clearly visible when the image is captured; (2) Capturing clothing images: Using the robot’s binocular camera system to take a comprehensive picture of the clothing, ensuring that key markings are clearly visible in the image; (3) Image processing process: Given the obvious color contrast between the artificial marks on the clothes and the background, the positions of these marks can be directly identified through image processing technology. The detailed steps include: (3.1) Color space conversion: convert the image from RGB color mode to HSV mode to facilitate color recognition and segmentation; (3.2) Color threshold segmentation: On the basis of ensuring the green component in the H channel, high thresholds of the S and V channels are set at the same time to accurately extract the green mark; (3.3) Morphological operation: Perform morphological operations on the binarized image to remove noise and connect the broken green areas; (3.4) Contour detection: Use the contour detection algorithm to identify the contours of all green marks in the image; (3.5) Marking drawing: Clearly mark the locations of all detected green marks on the original clothing image. 1.

2. Record the robot's folding action: Records of key folding points: (1) Folding recognition: Using the recognition algorithm in step 1, the clothes are monitored in real time during the folding process to identify the key folding points of the clothes. (2) Coordinate capture: Once the key folding point is identified, its exact position in the coordinate system is immediately recorded. (3) Point marking: Each key folding point is clearly marked on the image for subsequent review and analysis. 1.

3. Recording of folding action time: Record the start and end time of each folding action, and determine the duration of the action by calculating the difference between the two time points. The present invention requires detailed recording of the execution time of each folding step. 1.

4. Recording the distance between folding points: Record the distance between all key folding points. The specific methods include: (1) Obtain the coordinates of the folding point: use the coordinates recorded in step 3.3.1; (2) Calculate the distance: Use the Euclidean distance formula to calculate the distance between two points: Among them, (x1, y1) and (x2, y2) are the position coordinates of the two folding points respectively; (3) Record distance: record the distance between each pair of folding points; 1.

5. Generate a sequence of folding steps: A list of folding steps is compiled based on the recorded information, including but not limited to the key folding point positions, operation duration, and robot arm motion trajectory, and a folding process sequence arranged in order is generated based on the compiled data; 1.

6. Submit for review and execute operations: (1) Visual display: The generated folding step sequence is displayed to the user in a list format through the user interaction interface. The list includes key information such as the specific location of each folding point, the time required, and the order of operations. By integrating distance information, the folding path is marked and connected to assist the user in verifying the entire folding process, thereby improving the accuracy and efficiency of the operation. (2) Execution authorization: After verifying that the list is correct, the user can authorize the robot to start the folding task; (3) Record keeping: After each folding operation is completed, the operation sequence is saved and archived for future reference and reuse.

3. A clothes folding method using a clothes folding robot according to claim 2, characterized in that: the specific process of step 2 is: 2.1 Environment modeling: Divide the three-dimensional space into grids, each grid point represents a node, the connection between nodes is represented by edges, and the weight of the edge (w) is based on the distance between nodes; 2.2 Target point sequence: According to the target point position sequence generated in step 2, these points are regarded as special nodes in the graph; 2.3 Dynamic obstacle detection: Detect obstacles in the environment in real time, and add obstacle nodes at the corresponding positions in the graph to increase their costs; 2.4 Cost update: When the target point or obstacle position changes, update the cost of the relevant nodes; 2.5 Heuristic function: Since the present invention involves image recognition and robot control in three-dimensional space, the Euclidean distance is used as the heuristic function, and its expression is as follows: Among them, (x n ,y n , z n ) is the coordinate of node (n), (x g ,y g , z g ) are the coordinates of the target point.

4. A clothes folding method using a clothes folding robot according to claim 2, characterized in that: The specific process of step three is as follows: (1) Environmental preparation: Ensure that the workbench provides sufficient operating space for the robot arm and that there are no obstacles within the range of motion of the robot arm so that the robot can operate flexibly in a variety of postures; (2) Image acquisition: Operate the camera on the robotic arm to capture images of the clothing on the workbench from multiple angles to ensure the clarity of image details; (3) Edge detection: Perform edge detection on the clothing image to obtain the pixel-level coordinates of the clothing edge; (4) Feature matrix construction: Combine the edge coordinates of the clothing with the known clothing feature data to calculate the clothing feature matrix to provide data support for subsequent folding work; (5) Robot arm calibration: Compare the feature matrix of the clothing with the motion trajectory of the robot arm, and accurately calibrate the motion path of the robot arm to ensure that it can accurately grasp and manipulate clothing; (6) Posture adjustment: For each step of the operation, the motion trajectory and posture of the robot arm are adjusted to meet the predetermined folding plan; (7) Hand-eye coordination: Align the end effector of the robot arm to the position of the clothing, use the camera to capture the image, and apply image processing technology to determine the position information of the clothing in the camera coordinate system. This information will be input into the robot arm control system to achieve precise control based on real-time image data; (8) Parameter optimization: All relevant operating parameters are carefully optimized, including the camera calibration parameters, the robot arm's motion trajectory parameters, and the hand-eye coordination transformation matrix, in order to improve the accuracy and operational efficiency of clothing folding.

5. The clothing folding method of a clothing folding robot according to claim 4, characterized in that: The specific process of step 4 is as follows: (1) Prior extraction and conversion: Use 3D human key point recognition technology to capture action key point data, use object bounding box detection technology to obtain the spatial position and size attributes of the object, convert video and image prior information into feature vectors, locate the markers on the clothing, capture images, process the images to identify the markers, and use the YOLOv4 algorithm for target detection; (2) Temporal alignment: Self-attention and cross-attention mechanisms are used to ensure the temporal synchronization between actions and objects in the video sequence, maintain the temporal order of task execution, and ensure that the model gradually absorbs each stage of the task. (3) Trajectory generation: The TrajGAN generator creates action trajectories, and the discriminator evaluates the similarity with real actions. Real-world interaction data is collected, and the TS-CEM algorithm is used to adjust the trajectory. The behavior cloning technology is used to cultivate high-level strategies to select subtask sequences, and low-level strategies execute subtasks. Key folding points, action time, and distance between points are monitored and recorded in real time. (4) GAN inversion initialization: fix the generator parameters, find the latent vector that matches the environment variables through optimization, and design the loss function so that the model learns from failures and enhances the understanding of the task logic.

6. A clothes folding method using a clothes folding robot according to claim 5, characterized in that: The step 4 and later also include robot deep learning training, and the specific steps are as follows: Step 1: The robot imitates human actions to learn; Step 2: Robot imitation reinforcement; Step 3: Imitation and implementation of the robot's special function of folding clothes.

7. A clothes folding method using a clothes folding robot according to claim 6, characterized in that: The specific method for the robot to imitate human actions to learn is: (1) Calculate the similarity index; (2) Optimize the total loss function; (3) Perform GAN inversion processing; (4) Optimize and adjust the model; (5) Evaluate the success confidence of the model; (6) Perform T cross entropy method on the model.

8. The clothing folding method of a clothing folding robot according to claim 7, characterized in that: The specific method of the robot imitation reinforcement is: (1) Collection and operation demonstration; (2) Decompose the tasks; (3) Enhanced demonstration in a simulated environment; (4) Conduct hierarchical strategy learning; (5) Enhance and learn the data; (6) Using humans to perform residual learning in the loop.

9. A clothes folding method using a clothes folding robot according to claim 8, characterized in that: The specific method of imitating and realizing the robot's special function of folding clothes is as follows: (1) Perform prior extraction and conversion; (2) Time alignment of learning content; (3) Generate trajectories; (4) Perform GAN inversion initialization training; (5) Perform interactive optimization; (6) Further strategy learning; (7) conduct hindsight interactive contrast loss; (8) Perform temporal monotonic loss learning.