Fresh food cutting robot motion trail optimization method based on reinforcement learning driving

Abstract

The invention relates to the field of process control, in particular to a fresh food cutting robot motion trail optimization method based on reinforcement learning driving. Comprising the following steps: acquiring geometric information of fresh food by using a three-dimensional sensor, extracting three-dimensional space coordinate points, and constructing a fresh food morphological function through an implicit neural representation network; extracting morphological parameters through an auto-encoder; generating a candidate cutting sequence through an adaptive strategy network; simulating the candidate cutting sequence through a forward dynamic model to obtain a trajectory evaluation value; optimizing the adaptive strategy network through a soft actor-commentator algorithm to obtain a strategy network; obtaining a motion numerical control instruction through a strategy network; a tracking error signal is obtained through a servo control system, a joint driving signal is generated through a PID control algorithm, and an end effector is driven to complete cutting motion. According to the method, the forward dynamic model simulation and the soft actor-commentator algorithm are utilized, so that automatic cutting with higher efficiency can be realized in the face of non-standard fresh products.

Description

Technical Field

[0001] This invention relates to the field of process control, specifically to a method for optimizing the motion trajectory of a fresh food cutting robot based on reinforcement learning. Background Technology

[0002] In automated control systems, trajectory control and optimization of the position, attitude, and velocity of robot end effectors are core technologies for achieving high-precision and high-efficiency operations. Existing control methods generally employ pre-programmed position control modes, such as teach-and-playback or offline programming based on CAD / CAM models. These methods pre-define a static geometric path for the control system, devoid of temporal information or dynamic constraints. The servo system then performs tracking control solely based on the position commands along this path. This is the mainstream position control and trajectory generation scheme in the current industrial robot field.

[0003] However, the aforementioned position control scheme has significant technical shortcomings when applied to complex control tasks involving cutting flexible objects. Its control strategy is generated solely based on kinematic models, neglecting the dynamic characteristics of the controlled system itself and the contact mechanics issues during interaction with the external environment. Due to the nonlinear physical characteristics of fresh produce, such as flexibility and variability, this open-loop, static path-based control system lacks the ability to adaptively adjust to contact forces and deformations. This makes it difficult for the control system to achieve a balance between ensuring operational quality and increasing operating speed; it cannot plan an optimal motion trajectory for the fresh produce cutting robot that adapts to the physical characteristics of flexible, non-standard cutting objects, thus failing to guarantee cutting quality and achieve the optimal balance between cutting efficiency, motion stability, and raw material yield.

[0004] To address this, a reinforcement learning-driven method for optimizing the motion trajectory of a fresh food cutting robot is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a method for optimizing the motion trajectory of a fresh food cutting robot based on reinforcement learning, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for optimizing the motion trajectory of a fresh food cutting robot based on reinforcement learning, comprising: Geometric information of fresh produce is collected from the coordinate system of the end effector of the fresh produce cutting robot using a 3D sensor; 3D spatial coordinate points are extracted based on the geometric information of the fresh produce, and a fresh produce morphology function is constructed through an implicit neural representation network; the 3D spatial coordinate points are reduced in dimension and compressed through an autoencoder to extract morphological parameters. Based on morphological parameters, candidate cutting sequences are generated through an adaptive policy network; based on the fresh produce morphology function, the candidate cutting sequences are simulated through a forward dynamic model to obtain the expected cutting results; based on the expected cutting results, trajectory evaluation values ​​are output through preset performance indicators; the morphological parameters, candidate cutting sequences, and trajectory evaluation values ​​are stored in an experience replay pool; experience samples are sampled from the experience replay pool using a soft actor-critic algorithm to iteratively optimize the adaptive policy network to obtain the policy network; motion numerical control commands are obtained through the policy network. The servo control system compares the motion numerical control commands with the actual measured values ​​to obtain the tracking error signal. The PID control algorithm then generates the joint drive signal to drive the end effector to complete the cutting motion.

[0007] Preferably, the specific generation process of the fresh produce morphology function includes: In the end effector coordinate system, a three-dimensional sensor scans the fresh produce to acquire its geometric information. Digital filtering is then performed on this geometric information to remove noise and background points, extracting three-dimensional spatial coordinates. These coordinates are input to the encoder of the implicit neural representation network, where feature extraction is used to generate morphological feature vectors. The three-dimensional coordinates are then concatenated with these morphological feature vectors and input to the decoder of the implicit neural representation network, outputting a scalar symbolic distance value to form a fresh produce morphology function. The zero-level set of this morphology function constitutes the continuous three-dimensional surface of the fresh produce.

[0008] Preferably, the specific architecture of the implicit neural representation network includes: The system comprises an encoder and a decoder. The encoder employs a multilayer perceptron structure, using fully connected layers and nonlinear activation functions to extract features from the input three-dimensional spatial coordinates point by point, mapping them into a high-dimensional morphological feature vector. The high-dimensional morphological feature vector is then concatenated with the three-dimensional spatial coordinates. This is then input into the decoder. The decoder performs information fusion and spatial mapping using fully connected layers and nonlinear activation functions, and constructs the fresh produce morphology function by regressing and predicting scalar symbolic distance values ​​through a linear output layer.

[0009] Preferably, the specific process for generating the morphological parameters includes: The three-dimensional spatial coordinate points are input into the autoencoder. Based on the point cloud processing architecture, the three-dimensional spatial coordinate points are extracted point by point through a shared multilayer perceptron, and global morphological features are generated by converging them using a symmetric aggregation function. Based on the global morphological features, the mean vector and standard deviation vector are output. Based on the Gaussian distribution parameterized by the mean vector and standard deviation vector, a reparameterization method is used to sample and generate a low-dimensional latent vector as the morphological parameter.

[0010] Preferably, the specific process for generating the candidate cutting sequence includes: The morphological parameters are input into an adaptive policy network to generate a mean vector and a log-standard deviation vector. Based on the mean vector, the log-standard deviation vector, and a noise vector sampled from a standard normal distribution, an initial action vector is generated through reparameterization calculation. Based on the initial program action vector, the program action is compressed using a hyperbolic tangent function to generate program actions representing cutting trajectory segments. The program actions are iteratively aggregated to form the candidate cutting sequence.

[0011] Preferably, the specific process for generating the trajectory evaluation value includes: The candidate cutting sequence and the fresh morphology function are input into the forward dynamic model for fast forward prediction, and the three-dimensional morphology is output as the expected cutting result. The expected cutting result is input into a preset performance index function, and the chamfer distance between the expected cutting result and the target template morphology is calculated and combined with the collision penalty generated by the collision between the candidate cutting sequence and the preset obstacle to output the trajectory evaluation value.

[0012] Preferably, the specific process of the forward dynamic model includes: Based on the finite element method and damage mechanics model, and using the element deletion technique, offline simulations are performed on the candidate cutting sequence and the fresh morphology function to generate a training dataset containing simulation results. Based on the training dataset, a deep neural network is trained to quickly predict the corresponding new three-dimensional morphology. The surrogate model that has been trained is the forward dynamic model.

[0013] Preferably, the specific generation process of the motion numerical control command includes: Using the morphological parameters and the candidate cutting sequence as input, a soft Q-value is generated through forward propagation calculation via the critic network in the soft actor-critic algorithm. The morphological parameters are then used as input, and a policy entropy is obtained through forward propagation calculation via the adaptive policy network. A parameter update gradient is generated by combining the soft Q-value and the policy entropy. Based on the parameter update gradient, the network parameters of the adaptive policy network are updated. This update process is iteratively executed until the performance index of the adaptive policy network converges, resulting in the policy network. The morphological parameters are then input into the policy network, and program action vectors are iteratively generated using an autoregressive approach, culminating in a complete desired trajectory sequence. Based on the desired trajectory sequence, the motion control commands are generated through inverse kinematics calculation and interpolation.

[0014] Preferably, the specific process of generating joint drive signals through the PID control algorithm includes: The motion numerical control command is used as the desired trajectory, and the actual measurement value of the end effector is obtained. By comparing the desired trajectory with the actual measurement value, the tracking error signal is generated. The tracking error signal is proportionally amplified by a proportional element to generate a proportional control term. The tracking error signal is integrally processed by an integral element to eliminate steady-state error and generate an integral control term. The tracking error signal is differentiated by a differential element to predict the error change trend and generate a differential control term. The proportional control term, integral control term, and differential control term are linearly superimposed to generate a servo drive control quantity as the joint drive signal. The joint drive signal is used to drive the end effector to complete the cutting motion.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. A fresh produce morphology function is constructed using an implicit neural representation network; morphological parameters are extracted using an autoencoder; discrete and irregular geometric information of fresh produce is translated into a continuous and differentiable morphological function, enabling the modeling of the complex and variable shapes of fresh produce. Furthermore, by processing the three-dimensional spatial coordinates using the autoencoder, low-dimensional morphological parameters are obtained, forming an idealized state space for the adaptive strategy network. This not only reduces the learning difficulty and convergence time of the subsequent strategy network but also provides high information density, enabling the model to identify the commonalities and individual characteristics of different fresh produce items. This lays the data foundation for adapting to new samples never seen before and generating refined cutting strategies.

[0016] 2. Candidate cutting sequences are generated through an adaptive policy network, and efficient simulation is performed using a forward dynamic model, followed by iterative optimization using a soft actor-critic algorithm. This closed-loop "simulation-evaluation-optimization" approach can efficiently and stably explore a vast trajectory space and converge to the policy network in a virtual environment, avoiding wasted time, resource depletion, and potential safety risks, thus improving the overall efficiency and robustness of trajectory optimization.

[0017] 3. The servo control system compares the motion CNC commands generated by the strategy network with the actual measurements from the end effector in real time. A PID control algorithm is then used to dynamically generate joint drive signals based on the tracking error signal between the two. By leveraging the synergistic effect of proportional, integral, and derivative components, the end effector can track and reproduce the optimized complex cutting trajectory with high precision and stability, effectively suppressing the impact of external disturbances and model uncertainties. This ensures the accuracy of the optimal planning commands at the physical level.

[0018] 4. Based on the morphological function constructed by the implicit neural representation network and the low-dimensional morphological parameters extracted by the autoencoder, an idealized and easily processed state input is provided for reinforcement learning. Utilizing forward dynamic model simulation and the soft actor-critic algorithm, the optimal cutting sequence for a specific shape is efficiently generated iteratively in a virtual environment. A stable and reliable PID servo control system ensures that this optimal instruction is executed accurately by the end effector in the real world. This collaborative design of "precise perception - efficient optimization - accurate execution" ensures high-precision and high-efficiency automated cutting when dealing with non-standard fresh produce of various shapes. Attached Figure Description

[0019] Figure 1 This is a flowchart of the motion trajectory optimization method for a fresh food cutting robot based on reinforcement learning, as proposed in an embodiment of this invention. Figure 2 This is a flowchart of the morphology perception and strategy optimization proposed in an embodiment of this invention application; Figure 3 This is a flowchart of motion command generation and servo control proposed in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figures 1-3 The present invention provides a method for optimizing the motion trajectory of a fresh food cutting robot based on reinforcement learning, the specific steps of which are as follows: Geometric information of fresh produce is collected from the coordinate system of the end effector of the fresh produce cutting robot using a 3D sensor; 3D spatial coordinate points are extracted based on the geometric information of the fresh produce, and a fresh produce morphology function is constructed through an implicit neural representation network; the 3D spatial coordinate points are reduced in dimension and compressed through an autoencoder to extract morphological parameters. Based on morphological parameters, candidate cutting sequences are generated through an adaptive policy network; based on the fresh produce morphology function, the candidate cutting sequences are simulated through a forward dynamic model to obtain the expected cutting results; based on the expected cutting results, trajectory evaluation values ​​are output through preset performance indicators; the morphological parameters, candidate cutting sequences, and trajectory evaluation values ​​are stored in an experience replay pool; experience samples are sampled from the experience replay pool using a soft actor-critic algorithm to iteratively optimize the adaptive policy network to obtain the policy network; motion numerical control commands are obtained through the policy network. The servo control system compares the motion numerical control commands with the actual measured values ​​to obtain the tracking error signal. The PID control algorithm then generates the joint drive signal to drive the end effector to complete the cutting motion.

[0022] The technical solution of the present invention will be further described in detail below with reference to specific embodiments.

[0023] Example 1 This application discloses a reinforcement learning-driven method for optimizing the motion trajectory of a fresh food cutting robot. See also... Figure 1 The specific steps proposed in this invention include: S1. Collecting fresh geometric information from the coordinate system of the end effector of the fresh food cutting robot using a three-dimensional sensor; extracting three-dimensional spatial coordinate points based on the fresh geometric information, and constructing a fresh morphology function through an implicit neural representation network; reducing and compressing the three-dimensional spatial coordinate points using an autoencoder to extract morphological parameters; S2. Generating candidate cutting sequences based on morphological parameters through an adaptive strategy network; simulating the candidate cutting sequences using a forward dynamic model based on the fresh morphology function to obtain the expected cutting results; outputting trajectory evaluation values ​​based on the expected cutting results through preset performance indicators; storing the morphological parameters, candidate cutting sequences, and trajectory evaluation values ​​in an experience playback pool; S3. Sampling experience samples from the experience playback pool using a soft actor-critic algorithm, iteratively optimizing the adaptive strategy network to obtain a strategy network; obtaining motion numerical control commands through the strategy network; S4. Comparing the motion numerical control commands with actual measured values ​​using a servo control system to obtain a tracking error signal, generating joint drive signals through a PID control algorithm to drive the end effector to complete the cutting motion.

[0024] Furthermore, geometric information of the fresh produce is collected using a 3D sensor from the coordinate system of the end effector of the fresh produce cutting robot; 3D spatial coordinate points are extracted based on the geometric information, and a fresh produce morphology function is constructed using an implicit neural representation network; the 3D spatial coordinate points are then compressed by an autoencoder to extract morphological parameters; this corresponds to step S1 above; the specific implementation process includes: Starting from the coordinate system of the end effector of the fresh food cutting robot, a high-precision 3D sensor, such as a structured light scanner capable of providing sub-millimeter accuracy, is used to scan the target fresh product (e.g., a piece of salmon fillet) placed on the workbench to collect its raw geometric information. This produces an unstructured raw point cloud dataset; for example, a single scan may generate a massive dataset containing approximately 500,000 3D coordinate points, inevitably mixed with sensor noise, optical reflection anomalies caused by surface wetness, and background environmental points such as those on the workbench surface. To ensure the accuracy of subsequent modeling, rigorous preprocessing operations must be performed based on this geometric information. First, a statistical outlier removal algorithm is applied, calculating the average distance from each point to its k nearest neighbors and removing points whose average distance exceeds a multiple of the global mean plus standard deviation, effectively eliminating sparse noise points. Subsequently, the Random Sample Consensus (RANSAC) algorithm is used to iteratively extract small subsets from the point cloud to fit a planar model, ultimately identifying and removing the largest set of interior points representing the workbench surface. After this series of filtering and segmentation processes, the point cloud size is reduced to approximately 250,000 points, forming a clean set of three-dimensional spatial coordinate points that accurately describe the surface morphology of the fish fillet.

[0025] To transform these discrete point clouds into a continuous, differentiable mathematical representation, thereby overcoming the limitations of traditional mesh or voxel representations in terms of memory efficiency and topological flexibility, this clean point cloud is input into an implicit neural representation network. In this embodiment, the DeepSDF architecture is preferably adopted. The encoder part of the network first extracts features from the 250,000 input points, aggregating their complex geometric information into a fixed-dimensional, for example, 256-dimensional global morphological feature vector. Subsequently, the decoder part of the network learns a mapping function that can accept the coordinates of any spatial query point and the aforementioned 256-dimensional morphological feature vector as joint input, and outputs a scalar signed distance value. The absolute magnitude of this SDF value represents the shortest distance from the query point to the surface of the fish fillet, and its positive or negative sign indicates whether the point is located outside or inside the fish fillet. Thus, a fresh morphology function that can continuously describe the three-dimensional morphology of this specific fish fillet is constructed.

[0026] The extracted clean set of 3D spatial coordinates (e.g., approximately 250,000 points) is fed into an autoencoder model specifically designed for processing unstructured point clouds, such as the variational autoencoder (Point-VAE) based on the PointNet architecture. The encoder part of this autoencoder first maps each 3D spatial coordinate to a high-dimensional feature space (e.g., 1024-dimensional) independently through a multilayer perceptron (MLP) network with shared weights. Then, a global morphological feature vector is extracted from the features of all points using a symmetric aggregation function (e.g., max pooling). This global feature vector is then fed into two parallel fully connected layers, which output the mean vector and logarithmic standard deviation vector, respectively, to parameterize a Gaussian distribution. Finally, a reparameterization technique is applied to sample from this Gaussian distribution, generating a low-dimensional latent vector, such as a 64-dimensional vector. This 64-dimensional vector is the final extracted morphological parameter, capturing the core geometric features of the salmon fillet (such as length, thickness distribution, curvature, etc.) in a highly condensed form with good topological structure.

[0027] Implicit neural representation networks are used to learn discrete, noisy raw point cloud data as a continuous signed distance function. This approach is not only memory efficient but also allows for querying the distance from any point in space to the surface of an object, capturing the geometric details of fresh produce surfaces and providing a high-quality geometric model foundation for subsequent accurate trajectory planning. Variational autoencoders are used to nonlinearly reduce the dimensionality of the 3D spatial coordinate point set, encoding it into a low-dimensional, structured latent space and generating morphological parameters. This not only reduces the complexity of the state space but also accelerates the learning process of the policy network. Furthermore, due to the learning characteristics of variational autoencoders, fresh produce with similar shapes will be mapped to similar locations in the latent space, giving the trained policy network generalization capabilities. This allows the segmentation knowledge learned from a subset of samples to be effectively applied to new samples with similar shapes that have not been seen before.

[0028] Further, based on morphological parameters, candidate cutting sequences are generated through an adaptive policy network; based on the fresh produce morphology function, the candidate cutting sequences are simulated using a forward dynamic model to obtain the expected cutting results; based on the expected cutting results, trajectory evaluation values ​​are output through preset performance indicators; the morphological parameters, candidate cutting sequences, and trajectory evaluation values ​​are stored in an experience replay pool; this corresponds to step S2 above; see reference. Figure 2 The specific implementation process includes: The aforementioned 64-dimensional morphological parameters are used as state inputs and fed into a policy network. This network employs a recurrent neural network (RNN) or gated recurrent unit (GRU) architecture to handle time-dependent program actions. The morphological parameters can serve as the initial hidden state of the RNN / GRU, or be concatenated with the program action generated at each time step and used as the input for the current time step. After one forward propagation calculation, the network's output is not a definite action, but rather two vectors parameterized by a Gaussian distribution: a mean vector and a log-standard deviation vector. A noise vector is sampled from a standard normal distribution and combined with the mean and log-standard deviation vectors using a reparameterization technique to generate an initial action vector. This vector represents a small six-degree-of-freedom (6-DoF) pose increment of the cutting tool in the end effector coordinate system. To ensure that this action is physically feasible and bounded, the initial action vector is compressed using a hyperbolic tangent function (tanh), smoothly mapping its components to a standardized interval of [-1, 1]. This compressed vector constitutes a program action. Subsequently, this program action is used as the input of the network at the next time step to generate a second program action in an autoregressive manner; this process is executed iteratively until a predetermined sequence length (e.g., 50 action units) is reached, which is then aggregated to form the candidate cutting sequence.

[0029] Before physical execution, this sequence undergoes virtual evaluation, using the finite element method to simulate the complex deformation behavior of fresh produce. The contact between the cutting tool and the fresh produce is analyzed using a model based on a continuous symbolic distance field (SDF), thus avoiding discrete collision detection. The core process of cutting, namely crack formation and propagation, is simulated using a damage mechanics model based on a finite element mesh. In this model, as the cutting tool moves along a candidate trajectory, the mesh elements in its contact area with the fresh produce are subjected to stress. The strain energy or equivalent stress of these elements is calculated. When the damage variable of a certain element (calculated based on the cumulative stress / strain history) exceeds a predetermined material failure threshold, the element is removed from the simulation calculation or its stiffness matrix is ​​set to zero (i.e., the "element deletion" technique), thereby macroscopically simulating the physical effects of material separation and crack propagation. Specifically, this offline simulation process includes: first, voxelization based on the zero-level set of the fresh produce morphology function or generation of an initial finite element analysis mesh using a traveling cube algorithm, with the average side length of the mesh elements set to 1.5 mm. Secondly, the mesh is assigned a constitutive model of a hyperelastic material representing fresh produce (e.g., the Mooney-Rivlin model) and material damage parameters are set. For example, the hyperelastic material constitutive model of the fresh produce adopts the Mooney-Rivlin model, with material constants C10 set to 0.18 MPa and C01 set to 0.04 MPa. The material damage model adopts a failure criterion based on equivalent plastic strain, and the material failure threshold is set to 0.8. A finite element solver based on explicit dynamics (e.g., Abaqus / Explicit or LS-DYNA) is used. The candidate cutting sequence (i.e., a series of tool poses) is transformed into a time-varying forced displacement boundary condition, which is applied to the contact area between the tool and the mesh. Finally, the FEM solver is run, and the equivalent plastic strain of the contact elements is calculated in each incremental step. When the cumulative value of the equivalent plastic strain of any element exceeds 0.8, the element is immediately removed from the mesh topology (i.e., element deletion), and the mesh deformation result of this step is recorded. Repeating this process generated a training dataset containing approximately 500,000 sample pairs.

[0030] Using the dataset generated above, the surrogate model (GNN architecture) is trained through supervised learning. Specifically, based on the zero-level set of the fresh morphology function, a lightweight voxelization method is used to quickly generate a coarse-grained mesh with a unified topology, and the node coordinates and connectivity of this mesh are used as the initial graph input to the GNN. The GNN surrogate model can adopt a GNS-based architecture. In the encoding phase, the candidate cutting sequence (i.e., a series of tool pose increments) is first converted into the absolute pose path of the tool in the simulation time step. At each simulation time step, the set of mesh nodes currently in contact is determined by calculating the SDF collision detection between the tool geometry model and the fresh morphology function (or its meshed representation). The "node type" features of these nodes are labeled as "boundary conditions," and their expected displacement is set as the motion displacement of the tool in that time step. The GNN updates the node state through multi-round message passing, and its message passing function learns to simulate the force transmission between nodes, for example, by using a multilayer perceptron (MLP) to aggregate the relative position and velocity information of neighboring nodes. In the decoder stage, another MLP predicts the acceleration of a node in the next time step based on the updated node state, and updates the position and velocity of the node through an integrator (such as Euler integral) to achieve progressive prediction of the new 3D shape after cutting, thereby outputting a point cloud representing the new 3D shape.

[0031] To quantify the quality of the expected performance, a comparison is made with a pre-defined target template shape (e.g., an ideal geometric model of a standard 200-gram fish fillet). This comparison is performed using a pre-defined performance metric function, preferably employing chamfer distance as the core metric. The chamfer distance is calculated as follows: First, the squared Euclidean distance from each point in the expected cutting result point cloud to the nearest point in the target template point cloud is calculated, and the average of these distances is obtained. Then, the squared Euclidean distance from each point in the target template point cloud to the nearest point in the expected cutting result point cloud is calculated in reverse, and the average is obtained again. Finally, the two averages are added together to obtain a comprehensive, symmetrical distance metric. To ensure that this evaluation value aligns with the goal of maximizing rewards in reinforcement learning, the trajectory evaluation value is defined as a negative value of the chamfer distance. Furthermore, if simulations show that the candidate cutting sequence comes into contact with pre-defined obstacles such as a workbench or fixture, the trajectory evaluation value is directly set to a very large negative penalty (e.g., -500) to guide the policy network to avoid such invalid trajectories. Therefore, the trajectory evaluation value is a reward signal aimed at maximizing the target shape. The larger the value, the better the expected cutting result matches the target shape and the less collision occurs.

[0032] After obtaining the trajectory evaluation value, the complete experience of this simulation—namely, the morphological parameters (state), candidate cutting sequences (actions), and trajectory evaluation value—is packaged into a data sample and stored in the experience replay pool. This embodiment can enhance the experience replay pool. When the trajectory evaluation value of a candidate cutting sequence is low, the expected cutting result is treated as a new virtual target, the trajectory evaluation value of the candidate cutting sequence relative to the new virtual target is recalculated, and the successful experience sample of morphological parameters, candidate sequences, virtual targets, and trajectory evaluation values ​​is stored in the experience replay pool. In this way, failed explorations are transformed into valuable learning signals, improving sample efficiency when learning how to accurately achieve a specific cutting shape.

[0033] By employing an adaptive policy network and combining reparameterization techniques with noise sampling, a randomized candidate cutting sequence generation mechanism is implemented. This mechanism breaks away from traditional fixed paths, exploring non-intuitive but potentially superior cutting schemes. Compression processing using the hyperbolic tangent function ensures that the generated motion vectors are physically feasible and bounded, avoiding invalid or dangerous trajectory fragments. Predicting the cutting effect in a simulation environment avoids time-consuming physical trial and error on a physical robot. Using the chamfer distance between the expected cutting result and the target template shape as the trajectory evaluation value provides an optimization objective, enabling a quantitative assessment of the merits of each candidate sequence. This closed-loop "generation-simulation-evaluation" mechanism allows for rapid screening and approximation of the optimal cutting strategy through virtual iteration without contact with actual materials, improving the efficiency and safety of trajectory optimization.

[0034] Furthermore, experience samples are sampled from the experience replay pool using the soft actor-critic algorithm, and the adaptive policy network is iteratively optimized to obtain the policy network; through the policy network, motion numerical control commands are obtained; corresponding to step S3 above; the specific implementation process includes: The entire process from sequence generation to gradient update is iteratively optimized within a more robust and powerful soft actor-critic algorithm framework. SAC is an advanced off-policy, maximum entropy reinforcement learning algorithm, particularly suitable for complex robot manipulation tasks due to its high sample efficiency and stability. In this framework, in addition to the "actors" (policy network), there are one or more "critic" networks (Q networks) responsible for evaluating the long-term value (soft Q-value) of performing a specific action in a given state. The architecture of the critic network includes: a multilayer perceptron (MLP) for processing the morphological parameters of the input (state), a recurrent neural network (RNN) or gated recurrent unit (GRU) for processing candidate cut sequences of the input (actions), concatenating the output feature vectors of both, and then regressing and predicting the soft Q-value through another MLP network.

[0035] In each iteration, a small batch of data (e.g., batch size 512) is randomly sampled from a large experience replay pool (e.g., 50,000 experiences). This batch contains {morphological parameters, candidate cut sequences, trajectory evaluation values, and next morphological parameters}, where the next morphological parameters correspond to the expected cut results. A target Q-value is calculated, which integrates the trajectory evaluation value (reward) of the next state, the target Q-value of the next state (calculated via the target critic network and the target policy network), and the policy entropy of the next state. Subsequently, the critic network parameters are updated by minimizing the mean squared error (e.g., soft Bellman residual) between the Q-value predicted by the current critic network and the target Q-value. The morphological parameters from the batch data are input into the current adaptive policy network to generate a probability distribution of actions, from which actions are sampled. The policy entropy is calculated based on this probability distribution. Simultaneously, the morphological parameters and the sampled actions are input into the updated critic network to obtain the corresponding soft Q-values. The update objective of the actor network is to maximize the sum of the soft Q-values ​​and the weighted policy entropy, thereby calculating the policy gradient and updating the parameters of the adaptive policy network. Through multiple iterations, the actor network gradually learns to generate cutting sequences that consistently achieve high trajectory evaluation values ​​(i.e., high rewards) for any input fresh morphology parameters. Once the performance metrics of the policy network converge, training is complete, and the adaptive policy network becomes a generalizable policy network. This embodiment can introduce a policy ensemble method to generate the final policy network. By periodically saving snapshots of multiple adaptive policy networks and simultaneously inputting morphology parameters into multiple policy networks, different candidate expected trajectory sequences are obtained. A lightweight adjudication model, based on preset rules, selects the preferred sequence from the sequences, i.e., the motion control command. This ensemble decision-making mechanism enhances the robustness and stability of the decision-making process, effectively avoiding the accidental decision-making errors that may exist with a single network.

[0036] The 64-dimensional morphological parameters of the fresh product to be processed are used as the initial state input, and a forward propagation calculation is performed through the internal neural network layer of the policy network. The mean vector is taken, and its components are smoothly mapped to the standardized interval [-1, 1]. This vector is the first deterministic program action of the sequence. Subsequently, this generated deterministic program action is concatenated with the 64-dimensional morphological parameters as the joint input for the next time step, and fed back into the policy network to generate the second deterministic program action. By iterating this process, a sequence of 50 deterministic actions is generated, which is the final desired trajectory sequence. Then, based on the desired trajectory sequence, inverse kinematics calculation and interpolation are used to generate motion numerical control commands executable by the robot controller.

[0037] Iterative optimization of the adaptive policy network using the soft actor-critic algorithm efficiently utilizes historical data from the experience replay pool. By repeatedly sampling experience samples to update network parameters, data utilization and learning speed are improved, shortening the training time required for the model to converge to the optimal policy. Furthermore, the introduction of policy entropy maximization effectively prevents the policy from prematurely converging to local optima, resulting in a policy network that not only boasts superior performance but also stronger generalization ability and robustness. By autoregressively generating a complete desired trajectory sequence through the policy network and combining inverse kinematics calculation and interpolation, the conversion from abstract optimization policies to concrete, smooth, and executable motion control commands is ensured, providing reliable assurance for robot execution.

[0038] Furthermore, the servo control system compares the motion CNC commands with the actual measured values ​​to obtain a tracking error signal. A PID control algorithm then generates a joint drive signal to drive the end effector to complete the cutting motion; this corresponds to step S4 above; see [link to relevant documentation]. Figure 3 The specific implementation process includes: The motion control commands generated by the strategy network are used as the desired trajectory, and the actual measurement values ​​of the end effector are obtained in real time through the servo control system (e.g., through robot joint encoders and forward kinematics). The tracking error signal is generated by comparing the desired trajectory with the actual measurement values. An online anomaly handling mechanism can be introduced for the tracking error signal. The tracking error signal is integrated in parallel using a sliding window. When the integral value exceeds a preset threshold, a high-resistance event is triggered. The servo control system pauses the current motion control commands and performs a safety backtracking action to release stress. After the safety backtracking action, the trajectory segment that triggered the anomaly is re-executed at a lower feed rate. When the tracking error signal returns to normal, the remaining motion control commands are executed at the original speed. If the high-resistance event is triggered again at low feed rate, the motion is stopped, and a complete rescan and replanning process is awaited. In this way, the underlying tracking error signal is utilized as an environmental perception tool, enabling online and real-time adaptive adjustment to uncertainties in the physical world, improving the robustness and safety of the cutting process.

[0039] To achieve accurate trajectory tracking, the tracking error signal is proportionally amplified by a proportional (P) stage to generate a proportional control term for rapid response to the current error. Secondly, the tracking error signal is integrally processed by an integral (I) stage to accumulate historical errors and eliminate steady-state errors, generating an integral control term. Thirdly, the tracking error signal is differentially processed by a derivative (D) stage to predict error change trends and provide forward-looking suppression, generating a derivative control term.

[0040] The proportional control term, integral control term, and derivative control term are linearly superimposed to generate a total control quantity as the joint drive signal. The joint drive signal is used to drive the servo motors of each joint of the end effector to complete a high-precision cutting motion.

[0041] By using joint drive signals, optimized instructions are safely and accurately translated into physical reality, improving the robot's safety and robustness in contact tasks. This protects the cutting tools, products, and the robot itself, and improves the final cutting quality, avoiding squeezing or tearing of soft fresh products caused by rigid execution, ensuring smooth cuts and neat finished products.

[0042] This invention provides a reinforcement learning-driven method for optimizing the motion trajectory of a fresh produce cutting robot. By acquiring and parametrically representing the three-dimensional geometric information of each fresh product in the robot's end effector coordinate system in real time, it can generate corresponding cutting schemes for each fresh product with different shapes and sizes, improving the adaptability to automated processing of non-standard natural products. Furthermore, by using a forward dynamic model for virtual simulation and evaluation of the cutting process, it can rapidly iteratively optimize candidate cutting trajectories without consuming physical raw materials. By quantitatively comparing the expected cutting results with the target template and updating the policy network using a soft actor-critic algorithm, it can converge to the optimal cutting path, reducing the waste of high-value raw materials. The high-level motion numerical control commands generated by the policy network are connected to the low-level closed-loop servo system based on a PID control algorithm. This hierarchical control architecture ensures that the trajectory planned by artificial intelligence can be physically reproduced by the robot's end effector with high fidelity and robustness.

[0043] Example 2 This second embodiment demonstrates the specific application of the method of the present invention in fresh food cutting; the specific implementation method is as follows: Taking a fresh Atlantic salmon fillet weighing 1.62 kg and measuring 52.3 cm (length) × 18.9 cm (width) × 4.1 cm (height) as an example, its density is approximately 1050 kg / m³. The fillet is placed in the robot's work area. A six-axis FANUCRMate200iD / 7L robot, equipped with a Keyence LJ-X8080 laser profilometer at its end effector, scans along the long axis of the fillet at a reference height of 73 mm above the surface at a constant speed of 120 mm / s. The sensor operates at a sampling frequency of 1 kHz, capturing 3200 data points per profile. The entire scanning process lasts approximately 4.36 s, generating a raw point cloud containing approximately 130,000 three-dimensional coordinate points. To ensure data quality, the raw point cloud is then processed by a statistical outlier removal filter based on a point cloud library (PCL). The filter parameters are set as follows: 50 nearest neighbors for mean distance estimation and a standard deviation threshold of 1.2. This process removes approximately 1.5% of noise and background points caused by water vapor or conveyor belt reflections, ultimately generating a filtered point cloud containing approximately 130,000 coordinate points that accurately and cleanly describes the surface morphology of the fish fillets.

[0044] This process transforms discrete point cloud data into a continuous and compact mathematical representation, laying the foundation for subsequent intelligent decision-making. First, the filtered point cloud is input into a DeepSDF-based implicit neural representation network. This network consists of an 8-layer encoder and decoder, each with 512 neurons, using the ReLU activation function. Trained on a dataset containing thousands of different fish fillet samples, the network learns a continuous symbolic distance function (SDF) that represents the fish fillet morphology. During training, the network is optimized by minimizing the L1 loss between its predicted and true SDF values. After training, the network can output the directed distance from any query point in space to the fish fillet surface, where the zero-level set precisely defines the continuous three-dimensional surface of the fish fillet. Subsequently, to further compress information and extract key morphological features, the filtered point cloud (approximately 130,000 coordinate points) is directly input into a PointNet-based variational autoencoder (VAE). The encoder of this VAE extracts pointwise features through a shared multilayer perceptron and uses a symmetric aggregation function to converge and generate global features. Finally, it outputs the mean vector and standard deviation vector of a parameterized Gaussian distribution. A 128-dimensional latent vector is generated through reparameterized sampling as the morphological parameter.

[0045] The soft actor-critic reinforcement learning algorithm is trained in a simulation environment, with its state input being 128-dimensional morphological parameters. The training process takes place in a replay pool with a capacity of 1,000,000 experiences, a batch size of 256, and a learning rate of 3*10^6. -4In each training step, the agent's policy network generates a candidate cutting sequence based on the current morphological parameters. A differentiable forward dynamic model (agent model) receives this sequence, simulates the cutting process, and predicts the shape of the cut fillet. A pre-defined performance metric function evaluates the expected cutting results. This function primarily calculates the reward based on the chamfer distance between the expected cut block and a 200g standard fillet target template; an ideal cut might have a chamfer distance as low as 0.005, while a poor cut could have a chamfer distance as high as 0.15. Furthermore, any trajectory that collides with the conveyor belt or clamps is penalized with a penalty of -500. When the policy network converges to a single policy network, it can generate the optimal cutting trajectory sequence for the input fillet morphological parameters.

[0046] After a new fish fillet is scanned and its morphological parameters are extracted, its 128-dimensional morphological parameters are input into a well-trained policy network. The network deterministically outputs the optimal desired trajectory sequence. Each pose point in this sequence is converted into specific angle values ​​required by the robot's six joints through inverse kinematics calculation. The trajectory planner performs smooth interpolation between these joint angles to generate a continuous motion path, which is ultimately compiled into KAREL language motion numerical control instructions executable by the FANUC controller. These instructions are sent as desired setpoints to the robot's underlying servo control system. During execution, the PID controller of each joint runs at a frequency of 2kHz, comparing the desired position of the instructions with the actual position fed back by the motor encoder in real time, generating a tracking error signal. Taking the J2 joint, which bears a large dynamic load, as an example, its finely tuned PID parameters might be: proportional gain (Kp) = 50.0, integral gain (Ki) = 1.8, and derivative gain (Kd) = 9.5. Based on this error, the PID algorithm, combined with these gain parameters, generates a precise joint drive signal (i.e., motor torque), thereby driving the pure water jet nozzle (working pressure 379.2MPa, nozzle diameter 0.12mm) mounted at the end, reproducing the optimal cutting path planned by the reinforcement learning agent, and completing the precise, efficient, and personalized processing of fresh products.

[0047] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for optimizing the motion trajectory of a fresh food cutting robot based on reinforcement learning, characterized in that, include: Geometric information of fresh produce is collected using three-dimensional sensors from the coordinate system of the end effector of the fresh produce cutting robot. Based on the geometric information of fresh produce, three-dimensional spatial coordinate points are extracted, and a fresh produce morphology function is constructed through an implicit neural representation network. The morphological parameters are extracted by dimensionality reduction and compression of the three-dimensional spatial coordinate points through an autoencoder. Candidate cutting sequences are generated based on morphological parameters using an adaptive policy network. Based on the fresh produce morphology function, the candidate cutting sequence is simulated using a forward dynamic model to obtain the expected cutting results; Based on the expected cutting results, the trajectory evaluation value is output through preset performance indicators; The morphological parameters, candidate cutting sequences, and trajectory evaluation values ​​are stored in the experience replay pool. Experience samples are sampled from the experience replay pool using the soft actor-critic algorithm, and the adaptive policy network is iteratively optimized to obtain the policy network; the motion numerical control commands are obtained through the policy network. The servo control system compares the motion CNC commands with the actual measured values ​​to obtain the tracking error signal. The PID control algorithm then generates the joint drive signal to drive the end effector to complete the cutting motion.

2. The method for optimizing the motion trajectory of a fresh food cutting robot based on reinforcement learning as described in claim 1, characterized in that, The specific generation process of the fresh produce morphology function includes: scanning the fresh produce using a three-dimensional sensor in the end effector coordinate system to obtain its geometric information; performing digital filtering on the geometric information to remove noise and background points and extracting three-dimensional spatial coordinate points; inputting the three-dimensional spatial coordinate points into the encoder part of the implicit neural representation network to generate a morphological feature vector through feature extraction; concatenating the three-dimensional spatial coordinate points with the morphological feature vector and inputting it into the decoder part of the implicit neural representation network to output a scalar symbolic distance value, thus forming the fresh produce morphology function. The zero level set of the fresh produce morphology function constitutes the continuous three-dimensional surface of the fresh produce.

3. The method for optimizing the motion trajectory of a fresh food cutting robot based on reinforcement learning as described in claim 1, characterized in that, The specific architecture of the implicit neural representation network includes an encoder part and a decoder part. The encoder part adopts a multilayer perceptron structure, and extracts features from three-dimensional spatial coordinates point by point through fully connected layers and nonlinear activation functions, mapping them into high-dimensional morphological feature vectors. The high-dimensional morphological feature vectors are concatenated with the three-dimensional spatial coordinates and input into the decoder part. The decoder part performs information fusion and spatial mapping through fully connected layers and nonlinear activation functions, and constructs the fresh produce morphology function by regressing and predicting scalar symbolic distance values ​​through linear output layers.

4. The method for optimizing the motion trajectory of a fresh food cutting robot based on reinforcement learning as described in claim 1, characterized in that, The specific generation process of the morphological parameters includes: inputting the three-dimensional spatial coordinate points into the autoencoder; extracting point-by-point features of the three-dimensional spatial coordinate points based on a point cloud processing architecture using a shared multilayer perceptron; and using a symmetric aggregation function to aggregate and generate global morphological features; outputting a mean vector and a standard deviation vector based on the global morphological features; and sampling a low-dimensional latent vector as the morphological parameters based on a Gaussian distribution parameterized by the mean vector and the standard deviation vector using a reparameterization method.

5. The method for optimizing the motion trajectory of a fresh food cutting robot based on reinforcement learning as described in claim 1, characterized in that, The specific generation process of the candidate cutting sequence includes: inputting the morphological parameters into an adaptive policy network to generate a mean vector and a log-standard deviation vector; generating an initial program action vector by reparameterization based on the mean vector, the log-standard deviation vector, and a noise vector sampled from a standard normal distribution; generating program actions by compression processing using a hyperbolic tangent function based on the initial program action vector; and iteratively accumulating the program actions to form the candidate cutting sequence.

6. The method for optimizing the motion trajectory of a fresh food cutting robot based on reinforcement learning as described in claim 1, characterized in that, The specific process of generating the trajectory evaluation value includes: inputting the candidate cutting sequence and the fresh morphology function into the forward dynamic model, performing forward prediction, and outputting the three-dimensional morphology as the expected cutting result; inputting the expected cutting result into a preset performance index function, calculating the chamfer distance between the expected cutting result and the target template morphology, and combining it with the collision penalty generated by the collision between the candidate cutting sequence and the preset obstacle, and outputting the trajectory evaluation value.

7. The method for optimizing the motion trajectory of a fresh food cutting robot based on reinforcement learning as described in claim 1, characterized in that, The specific process of the forward dynamic model includes: performing offline simulation of the candidate cutting sequence and the fresh morphology function based on the finite element method and damage mechanics model, and using element deletion technology to generate a training dataset containing simulation results; and training a deep neural network based on the training dataset to quickly predict the corresponding new three-dimensional morphology, wherein the deep neural network is the forward dynamic model.

8. The method for optimizing the motion trajectory of a fresh food cutting robot based on reinforcement learning as described in claim 1, characterized in that, The specific generation process of the motion numerical control command includes: taking the morphological parameters and the candidate cutting sequence as input, performing forward propagation calculation through the critic network in the soft actor-critic algorithm to generate a soft Q value; taking the morphological parameters as input, performing forward propagation calculation through the adaptive policy network to obtain the policy entropy; combining the soft Q value and the policy entropy to generate a parameter update gradient; updating the network parameters of the adaptive policy network based on the parameter update gradient; iteratively executing the update process until the performance index of the adaptive policy network converges to obtain the policy network; inputting the morphological parameters into the policy network, iteratively generating program action vectors through autoregression, and collecting them to form a desired trajectory sequence; and generating the motion numerical control command based on the desired trajectory sequence through inverse kinematics solution and interpolation operation.

9. The method for optimizing the motion trajectory of a fresh food cutting robot based on reinforcement learning as described in claim 1, characterized in that, The specific process of generating the joint drive signal using the PID control algorithm includes: taking the motion numerical control command as the desired trajectory and obtaining the actual measurement value of the end effector; generating the tracking error signal by comparing the desired trajectory with the actual measurement value; proportionally amplifying the tracking error signal through a proportional element to generate a proportional control term; performing integral operation on the tracking error signal through an integral element to eliminate steady-state error and generate an integral control term; performing differential operation on the tracking error signal through a derivative element to predict the error change trend and generate a derivative control term; linearly superimposing the proportional control term, integral control term, and derivative control term to generate a servo drive control quantity as the joint drive signal, and using the joint drive signal to drive the end effector to complete the cutting motion.

Images