A three-stage cooperative production line body intelligence control method and system
By employing a three-level collaborative intelligent control method for production lines, the problems of computational delay and insufficient adaptive capability of traditional control models are solved, enabling rapid response to the production environment and high-precision operation, which is suitable for clustered applications of flexible production lines.
Patent Information
- Application Number
- CN202610630591.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-09
- Publication Date
- 2026-07-24
Smart Images
Figure CN122442640A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of intelligent manufacturing technology, specifically relating to a three-level collaborative intelligent control method and system for production lines. Background Technology
[0002] With the evolution of smart manufacturing technologies, embodied intelligence has become a core driving force for industrial automation in flexible production lines. Embodied intelligent robots, by deeply integrating artificial intelligence algorithms with physical actuators, enable autonomous perception, decision-making, and operation in complex industrial environments, significantly improving the flexibility and efficiency of production lines. These technologies typically involve cross-domain knowledge integration, including semantic understanding, spatial modeling, and multi-degree-of-freedom motion control, and are key infrastructure for realizing unmanned factories and highly customized production.
[0003] Among these, the multi-level collaborative control architecture is the technological cornerstone for ensuring that embodied intelligent robots can complete complex interactive tasks. This architecture aims to transform macroscopic production instructions into precise robotic arm end-effector movements through information processing across different dimensions, in order to cope with real-time changes in spatial constraints and physical feedback during the production process. In an ideal collaborative state, the control system needs to possess efficient task decomposition capabilities and be able to perform real-time parameter correction and motion compensation for uncertainties in dynamic environments, thereby ensuring a high degree of consistency between logical decision-making and physical execution.
[0004] However, traditional end-to-end deep learning-based control models suffer from high system response latency due to massive computational resource consumption when handling long-range complex tasks, making it difficult to meet the real-time cycle time requirements of production lines. Simultaneously, existing centralized control architectures often lack the ability to adapt to subtle physical changes in the production environment. When faced with sudden disturbances such as workpiece pose shifts or conveyor belt vibrations, they cannot provide effective dynamic compensation at the execution layer in real time, leading to compromised operational accuracy or even mechanical collisions. Furthermore, the lack of an effective high-frequency linkage mechanism between high-level logical decisions and low-level physical execution makes the system sluggish when handling the flexible feedback required for precision assembly, making it difficult to balance operational flexibility and safety. Therefore, a three-level collaborative intelligent control solution for production lines is desired. Summary of the Invention
[0005] The purpose of this invention is to provide a three-level collaborative intelligent control method and system for production lines, which can effectively solve the problems mentioned in the background art.
[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A three-level collaborative intelligent control method for production lines includes the following specific steps: Establish a global task planning layer running on a cloud server or edge server, process semantic parsing and logical decomposition within a first preset frequency range, and generate atomic action sequences; A local spatial perception layer is established running on the edge computing unit on the production line side. It receives atomic motion sequences at a second preset frequency range, and uses real-time data from edge sensors to construct a dynamic local map. The workpiece pose is determined by point cloud registration, and the atomic motion sequences are mapped into a continuous motion trajectory flow. A real-time motion execution layer is established running on the robot controller terminal. It receives the instantaneous target pose in the motion trajectory stream at a third preset frequency range, reads the terminal feedback data, calculates the theoretical driving torque by combining the Lagrange dynamics equation, and switches to impedance control mode at the moment of contact to adjust compliance and drive the actuator to complete the task. An asynchronous collaboration layer is established to enable asynchronous communication between the global task planning layer, the local spatial perception layer, and the real-time action execution layer through the industrial network, and to control the first preset frequency range, the second preset frequency range, and the third preset frequency range to be distributed in a stepped manner.
[0007] Furthermore, the steps to establish a global task planning layer running on a cloud server specifically include: Acquire and parse long-range operation instructions, use deep neural networks based on attention mechanisms for intent recognition, and extract operation verbs, target objects, and process constraint parameters; The identified intent is decomposed into actions using a long short-term memory network, generating an ordered sequence of multiple atomic actions. Each atomic action is encapsulated in the data structure as a feature vector containing target object attributes, expected pose, and operation torque limit. An anomaly diagnosis module based on a support vector machine classifier is constructed to identify operational anomalies by monitoring the real-time data stream of the production line and to issue a safe shutdown order or a replanning request.
[0008] Furthermore, the steps for establishing a local spatial awareness layer running on the edge computing unit of the production line specifically include: Configure and collect sensor data from the red, green and blue depth cameras integrated at the end of the robot and the lidar distributed around the production line. The sampling frequency of the sensors shall not be lower than the preset sampling frequency. Statistical filtering is performed on the raw point cloud data acquired by the red-green-blue depth camera to remove outliers, and voxel downsampling is used to simplify the data. Point cloud data registration is performed, and an improved iterative nearest point algorithm is used to determine the precise pose of the workpiece. The registration process first performs coarse registration by extracting fast point feature histogram features, and then performs fine registration by using the point-to-plane error metric function. Based on the workpiece pose information obtained from registration, a local dynamic spatial model reflecting the workpiece pose and obstacle distribution is constructed. The elastic band algorithm is then used to apply an attractive force pointing towards the target pose and a repulsive force avoiding obstacles on the basis of the original reference trajectory, thereby generating a corrected local path.
[0009] Furthermore, the step of establishing a local spatial awareness layer running on the edge computing unit on the production line side also includes: The pose sequence fed back by the sensor is predicted and smoothed by the Kalman filter algorithm to suppress the vibration interference of the conveyor belt and output a stable target pose sequence. The state vector of the Kalman filter is defined as the two-dimensional position and two-dimensional velocity of the workpiece on the conveyor belt plane, and the observation vector is provided by the position component in the pose obtained by registration.
[0010] Furthermore, the steps for establishing a real-time motion execution layer running on the robot controller terminal specifically include: The feedback values of the joint encoder and the multi-dimensional torque sensor are read at a high frequency, which is the same as the third preset frequency range. The theoretical driving torque required by the robotic arm in the current motion state is calculated using the Lagrange dynamics equation. The equation is: the joint driving torque vector is composed of the product of the inertia matrix and the joint acceleration vector, the Coriolis force and centrifugal force terms, the gravity term and the external contact torque. Based on the deviation between the actual external contact torque sensed by the torque sensor and the theoretically calculated value, the output current of the motor is corrected in real time using a proportional-integral-derivative algorithm to compensate for positioning errors caused by mechanical clearance and nonlinear friction.
[0011] Furthermore, the step of switching to impedance control mode at the moment of contact to adjust compliance specifically includes: At the moment when the actuator completes the assembly task and comes into contact with the workpiece, the control mode is switched to the impedance control mode. The control law of this mode expresses the external contact force vector between the end effector and the environment as the sum of the products of the virtual inertia matrix, the virtual damping matrix and the virtual stiffness matrix with the desired acceleration deviation, the desired velocity deviation and the desired pose deviation, respectively. By adjusting virtual stiffness, virtual damping, and virtual mass parameters, the robotic arm can exhibit preset compliance in the contact direction, allowing it to be guided to the target assembly position by physical contact force even with minor pose errors.
[0012] Furthermore, the asynchronous communication steps between the global task planning layer, the local spatial perception layer, and the real-time action execution layer via the industrial network specifically include: The decision frequency of the global task planning layer is set to 0.5 Hz to 2 Hz, forming the first preset frequency range; The processing frequency of the local spatial perception layer is set to 30 Hz to 100 Hz, forming a second preset frequency range; The control frequency of the real-time action execution layer is set to 1000Hz to 4000Hz, forming the third preset frequency range; Each level operates independently and does not require clock synchronization. When the local spatial perception layer detects that the amount of environmental change exceeds the preset safety boundary value, the current trajectory execution is immediately interrupted and a replanning request is sent to the global task planning layer, triggering a secondary decision by the logic layer.
[0013] Furthermore, the step of detecting that the amount of environmental change exceeds a preset safety boundary value specifically includes: In the dynamic local map, a three-dimensional safety boundary box is set for the critical operation area. The size of the boundary box is defined as a cubic area formed by expanding outward from the target workpiece with a preset safety distance threshold as the center. The system calculates the shortest distance between the end effector of the robotic arm or the moving workpiece and the bounding box in real time. When the shortest distance is less than 0 mm, an intrusion occurs. Or when the velocity vector of a dynamic obstacle in the environment points to the bounding box and is expected to intrude in the next control cycle, it is determined that the environmental change exceeds the preset safety boundary value. Interrupting the execution of the current trajectory specifically involves sending an emergency stop command with a higher priority than the command for regular trajectory points to the real-time motion execution layer via a high-speed industrial fieldbus. Sending a replanning request specifically involves sending a replanning request message containing a timestamp, anomaly type code, and an affected action sequence identifier to the global task planning layer of the cloud server via Message Queue Telemetry Transport Protocol or Hypertext Transfer Protocol.
[0014] Furthermore, the steps for driving the actuator to complete the task also include: When the drive actuator completes the gripping task, it automatically adjusts the drive torque to achieve flexible gripping by sensing the force tactile feedback inside the gripper. The target value of the gripping force is determined according to the material properties of the workpiece and dynamically switches within the preset torque range. The synchronous clock signal of the high-speed industrial fieldbus ensures that the start-up time error of each joint motor is less than the preset synchronous error threshold, so that the pace consistency of each joint during the movement is maintained, and the motion error of the end effector in spatial pose is less than the preset deviation threshold.
[0015] A three-level collaborative production line integrated intelligent control system includes: The global task planning module, which runs on a cloud server, is used to process semantic parsing and logical decomposition within a first preset frequency range to generate atomic action sequences. The local spatial perception module runs on the edge computing unit on the production line side. It is used to receive atomic motion sequences at a second preset frequency range, and to construct a dynamic local map using real-time data from edge sensors. It determines the workpiece pose through point cloud registration and maps the atomic motion sequences into a continuous motion trajectory flow. The real-time motion execution module runs on the robot controller terminal. It is used to receive the instantaneous target pose in the motion trajectory stream at a third preset frequency range, read the terminal feedback data, calculate the theoretical driving torque by combining the Lagrange dynamics equation, and switch to the impedance control mode at the moment of contact to adjust the compliance and drive the actuator to complete the task. The asynchronous coordination module is used to realize asynchronous communication between the global task planning module, the local spatial perception module and the real-time action execution module through the industrial network, and to control the first preset frequency range, the second preset frequency range and the third preset frequency range to be distributed in a stepped manner.
[0016] In summary, this application includes at least one of the following beneficial technical effects: 1. This invention achieves spatiotemporal decoupling. The global task planning layer is responsible for semantic logic processing, the local spatial perception layer is responsible for environment modeling, and the real-time action execution layer focuses on high-frequency physical control. This architecture avoids the computational latency caused by traditional end-to-end models when handling complex long-range tasks, enabling the underlying execution mechanism to respond to changes in the physical environment within a preset period, thus optimizing the production cycle.
[0017] 2. This invention achieves rapid adaptation to minute changes in the production environment through a real-time feedback mechanism in the local spatial perception layer. When environmental vibrations occur or the workpiece position deviates within a preset range, the perception layer can directly correct the motion trajectory without requesting replanning from higher-level logic. This local closed-loop control mode significantly reduces the impact of environmental uncertainties on operational accuracy and avoids mechanical collisions or gripping failures caused by workpiece displacement.
[0018] 3. This invention deploys complex logic reasoning in the cloud or at the edge, allowing the robot controller terminal to handle only high-frequency dynamic compensation and motor drive tasks. This task allocation method reduces the hardware requirements of the robot controller terminal, effectively reducing the deployment cost of a single robot while ensuring control accuracy, which is beneficial for clustered applications in large-scale flexible production lines.
[0019] 4. This invention can effectively compensate for positioning errors caused by mechanical backlash and nonlinear friction. In precision assembly scenarios, the system achieves compliant gripping and adaptive assembly by sensing contact force, significantly improving operational flexibility and safety. It solves the problem of slow response in traditional centralized control when handling fine feedback, ensuring a high yield rate in complex processes. Attached Figure Description
[0020] Figure 1 A schematic diagram of the overall scheme for a three-level collaborative production line with an intelligent control method; Figure 2 This is a schematic diagram illustrating the core principles of a three-level asynchronous collaborative mechanism. Figure 3 A logical flowchart for global task planning and atomic action decomposition; Figure 4 A flowchart illustrating the logic of local spatial perception and dynamic trajectory planning; Figure 5 This diagram illustrates the multi-level interaction and data flow between the cloud, edge, and robot terminals. Detailed Implementation
[0021] Example 1 To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1-5 The present invention will be further described in detail below with reference to specific embodiments.
[0022] The first aspect disclosed in this application is a three-level collaborative embodied intelligent control method for production lines. This method constructs a highly adaptive embodied intelligent control system through deep decoupling and asynchronous collaboration between a global task planning layer, a local spatial perception layer, and a real-time action execution layer. In the entire system's operation, the global task planning layer is responsible for the macroscopic arrangement of long-range logic, the local spatial perception layer is responsible for the real-time mapping of dynamic environmental changes, and the real-time action execution layer is responsible for the precise compensation of high-frequency physical characteristics.
[0023] Specifically, the intelligent control method for the three-level collaborative production line of this application is implemented according to the following steps: First, for step S1, a global task planning layer is established. The global task planning layer is the top-level logic center of the entire control system. It is responsible for receiving and understanding high-level production instructions and decomposing them into a sequence of atomic actions that are executable at the bottom level and have clear temporal relationships.
[0024] This layer runs on cloud or edge servers and primarily handles tasks such as semantic parsing, logic decomposition, and global anomaly monitoring. Through this layer's processing, the originally abstract and unstructured natural language instructions are transformed into structured, standardized machine-executable logic, providing clear input for subsequent spatial perception and action execution. Specifically, this step includes the following sub-steps: Step S101: Acquire and parse long-range operation instructions; the global task planning layer receives long-range operation instructions through the industrial field bus interface or wireless communication protocol. These instructions are typically represented as highly abstract unstructured text, such as "grab type A workpiece at workstation 3 and assemble it onto base B." The natural language processing module, as the front end for instruction reception, is responsible for acquiring this text.
[0025] Step S102: Intent recognition is performed using an attention mechanism. To extract key information from unstructured text, the system employs a deep neural network based on an attention mechanism. The network structure includes an embedding layer, a multi-head self-attention layer, and a multilayer perceptron.
[0026] The training process of the network is as follows: First, historical production instruction texts are collected, such as grabbing workpieces, moving to workstation 2, starting assembly, etc., and key information in these texts is annotated by humans.
[0027] The labels include operation verbs such as grasp or assemble, target object names such as workpiece type A or base type B, and process constraint parameters such as torque limit of 5 Nm or speed of 0.1 m / s. Each labeled instruction text and its corresponding label pair constitute a training sample. The network's output layer uses the softmax activation function to output the probability distribution for each label category.
[0028] The cross-entropy loss function is used to calculate the difference between the label distribution predicted by the network and the actual label distribution annotated by humans. The training objective is to minimize this loss function, enabling the network to accurately identify operational verbs, target objects, and process constraint parameters in new instructions, thereby achieving intent recognition.
[0029] Step S103: Action decomposition using a Long Short-Term Memory (LSTM) network; after identifying the instruction intent, the system needs to decompose it into a series of ordered, indivisible atomic actions. To this end, the global task planning layer uses an LSM network to perform time-series modeling of long-term instructions in order to capture the logical causal relationships within the instructions.
[0030] The training process of the network is as follows: First, an industrial semantic library is built, which is a structured database containing a large number of standard atomic action terms. Each term corresponds to a logical unit, such as move, recognize, open gripper, close gripper, reset.
[0031] Then, the historical long-term instruction intentions, such as gripping and assembling a workpiece, are paired with their corresponding standard atomic action sequences, such as moving, recognizing, opening the gripper, closing the gripper, and resetting, to form training samples. The Long Short-Term Memory network takes the feature vector of the instruction intention as input and the atomic action sequence as output.
[0032] The loss function employed is a connectionist temporal classification loss function, which can handle the problem of misaligned input and output sequence lengths. The training objective is to minimize this loss function, enabling the network to decompose a complex intent into the correct sequence of atomic actions.
[0033] For example, the instruction to grab and assemble a workpiece can be correctly broken down into an ordered sequence of five atomic actions: moving, identifying, opening the gripper, closing the gripper, and resetting.
[0034] Step S104: Encapsulate atomic action data; each atomic action is encapsulated as a feature vector in the data structure. This vector contains fields such as target object attributes, expected pose, operating torque limit, and resource constraints.
[0035] Step S105: Set action connection conditions; actions are connected through logical trigger conditions. Logical trigger conditions include the completion feedback signal of the preceding action, timestamp constraints, and sensor trigger thresholds.
[0036] The global task planning layer will only generate and issue the logical description of subsequent actions when the preset triggering conditions are met, such as receiving a completion feedback signal of the preceding action and its value reaching a preset threshold.
[0037] Step S106: Construct an anomaly diagnosis module; the global task planning layer also includes an anomaly diagnosis module. This module uses a support vector machine classifier to monitor the real-time data stream during production line execution. The data stream includes status words, alarm codes, and performance metrics returned by each level.
[0038] The training process for this classifier is as follows: Historical operating data from the production line is collected, and each data sample is a feature vector composed of a status word, alarm code, and performance indicators. Simultaneously, each sample is manually labeled with an anomaly type, such as normal, mechanical jam, visual loss, or communication packet loss.
[0039] The goal of a Support Vector Machine (SVM) classifier is to find an optimal hyperplane that separates data samples of different classes to the greatest extent possible. Its training process essentially involves solving a convex quadratic programming problem to maximize the classification margin. After training, the classifier can identify various types of production line operation anomalies in real time, including mechanical jams, visual loss, and communication packet loss.
[0040] Once an anomaly is detected, the anomaly diagnosis module will issue a safety shutdown command or replanning request within a preset response time to ensure system security.
[0041] Step S1 constitutes the complete workflow of the global task planning layer. This layer decomposes high-level production tasks into atomic action sequences with temporal relationships through semantic parsing, and ensures the safety and reliability of the operation process through an anomaly diagnosis module.
[0042] After this processing layer, the originally abstract production instructions are transformed into a series of clearly defined and logically clear atomic actions. Each atomic action carries the pose, torque, and resource constraint information required for subsequent execution. The actuator includes the gripper installed at the robot's end effector.
[0043] The next step is step S2, which establishes a local spatial perception layer. This layer runs on an edge computing unit deployed on the production line side and is responsible for mapping the atomic action sequence generated in step S1 into a spatially continuous motion trajectory flow.
[0044] This layer utilizes real-time data from edge-side sensors to construct a dynamic local map and achieves rapid response to minor environmental changes through local path planning, thereby avoiding frequent requests for logical replanning from the top layer. Specifically, this step includes the following sub-steps.
[0045] Step S201: Configure and collect edge-side sensor data; the edge-side sensors on which the local spatial perception layer depends include red, green and blue depth cameras integrated at the end of the robot and lidar distributed around the production line.
[0046] The sampling frequency of these sensors is set to be no less than a preset sampling frequency, such as 30 Hz, to ensure real-time capture of environmental changes. The system synchronously reads the raw data from each sensor via industrial Ethernet or a dedicated data acquisition interface.
[0047] Step S202, point cloud data preprocessing; the local spatial perception layer acquires raw point cloud data of the environment through red-green-blue depth cameras. The first step in constructing a dynamic local map is to preprocess the raw point cloud.
[0048] First, a statistical filtering algorithm is used to remove outliers from the original point cloud. In this algorithm, for each point, the distance distribution of all its neighboring points is calculated, and the number of neighboring points is set to a first preset value, such as 50. The average and standard deviation of the distances are calculated, and the standard deviation factor is set to a second preset value, such as 1.0.
[0049] When the average distance of a point's neighborhood exceeds a threshold equal to the average plus a second preset value times the standard deviation, the point is identified as an outlier and removed.
[0050] Subsequently, in order to reduce the computing load on the edge computing units, the system uses voxel downsampling technology to refine the point cloud density to a preset level of detail.
[0051] The voxelization process involves constructing a 3D cubic mesh with fixed side lengths, for example, 5 millimeters, and then averaging the coordinates of multiple point clouds within each mesh to replace all the original points. This method significantly reduces the amount of data while preserving the environmental topology.
[0052] Step S203: Point cloud registration and workpiece pose determination; after point cloud preprocessing, the local spatial perception layer performs point cloud data registration to determine the precise pose of the workpiece. The registration process employs an improved iterative nearest-point algorithm.
[0053] The algorithm first performs coarse registration by extracting a fast point feature histogram from the point cloud. A fast point feature histogram is a 32-dimensional vector that describes the local geometric features of the point cloud.
[0054] The system calculates fast point feature histogram features for the collected source point cloud and the pre-stored workpiece template point cloud respectively. Then, it quickly reduces the rotation and displacement deviation between the source point cloud and the template point cloud through feature matching to obtain an initial pose transformation matrix.
[0055] Based on coarse registration, the system uses a point-to-plane error metric function for fine registration, the error function of which is:
[0056] in, Represents the first point in the source point cloud. The coordinate vector of a point, Indicates the target point cloud and The corresponding coordinate vectors of the points, The rotation matrix is a 3x3 orthogonal matrix. It is a translation vector, and is a 3-dimensional vector. For target point cloud in The unit normal vector at that location, This represents the dot product operation of vectors.
[0057] Minimize the error function using a nonlinear least squares method. The system can obtain a high-precision pose transformation matrix, enabling the registration accuracy to meet the preset accuracy requirements, such as 0.1 mm, and the registration time is less than the preset time threshold, such as 100 milliseconds.
[0058] Step S204: Construct a local dynamic space model and local path planning; based on the workpiece pose information obtained from registration, the local space perception layer constructs a local dynamic space model reflecting the workpiece pose and obstacle distribution. This model uses the robot base coordinate system as a reference, representing the workpiece, conveyor belt surface, surrounding fixed obstacles, and dynamic obstacles as bounded-box spatial geometries.
[0059] When the local spatial perception layer detects a slight offset of the workpiece on the conveyor belt, such as an offset of ±5 millimeters, its built-in local path planner will automatically adjust the trajectory output without requesting the global task planning layer to make secondary decisions at the logical level.
[0060] The local path planner uses the elastic band algorithm, which applies an attractive force to the target pose on the basis of the original reference trajectory, while applying a repulsive force to avoid obstacles, thereby quickly generating a corrected local path.
[0061] Step S205: Mapping atomic action sequences to motion trajectory flows and obstacle avoidance; when mapping atomic action sequences to motion trajectory flows, the system uses a cubic spline interpolation algorithm.
[0062] Specifically, for the starting point, intermediate path point and ending point of each atomic action, the system constructs cubic spline functions in the position space, velocity space and acceleration space respectively to ensure that the entire motion trajectory is a continuous and smooth curve in the position, velocity and acceleration levels.
[0063] Simultaneously, the system incorporates a dynamic window method for local obstacle avoidance. The working principle of the dynamic window method is as follows: it discretizes combinations of linear velocity and angular velocity in the velocity space, simulates the motion trajectory over a future period for each velocity group, then selects executable velocity combinations based on robot dynamics constraints, and finally selects the optimal velocity combination according to an evaluation function.
[0064] The evaluation function comprehensively considers the progress toward the target, the distance to the nearest obstacle, and the current speed. This method ensures that when the robotic arm performs a movement, the minimum distance between its motion envelope and dynamic obstacles in the environment is always greater than or equal to a preset safe distance threshold, such as 50 millimeters.
[0065] Steps S204 and S205 realize the transformation from discrete atomic actions to continuous smooth motion trajectories, and can autonomously perform local path correction when environmental changes are detected, without relying on real-time intervention of top-level logic.
[0066] Step S206: Kalman filtering suppresses conveyor belt vibration interference. To address common conveyor belt vibration interference in production lines, the local spatial perception layer uses a Kalman filtering algorithm to predict and smooth the pose sequence fed back by the sensors. The state vector of the Kalman filter is defined as the two-dimensional position and two-dimensional velocity of the workpiece on the conveyor belt plane.
[0067] The observation vector is provided by the position component in the pose obtained through registration in step S203. The filter alternately performs prediction and update steps: the prediction step calculates the state at the next moment based on the constant velocity motion model; the update step then weights and corrects the prediction result based on the current observations. In this way, the Kalman filter can effectively suppress the influence of physical vibrations within a specific frequency range on trajectory planning and output a stable target pose sequence.
[0068] In summary, step S2 together constitutes the complete workflow of the local spatial perception layer. This layer utilizes sensor data from the edge side to construct a dynamic local map, determines the workpiece pose through point cloud preprocessing and registration, and maps the atomic motion sequence into a smooth motion trajectory flow. Simultaneously, through local path planning and Kalman filtering, this layer can autonomously cope with environmental disturbances such as workpiece displacement and conveyor belt vibration without top-level intervention.
[0069] Finally, in step S3, a real-time motion execution layer is established. This layer runs on the robot controller terminal and is responsible for receiving the instantaneous target pose from the trajectory stream generated in step S2. It then combines this with terminal feedback data to perform dynamic compensation, ultimately driving the actuators to complete the task. The control frequency of this layer is set to a preset frequency, such as 2000 Hz, and it communicates in real-time with each joint actuator via a high-speed industrial fieldbus, ensuring that the transmission delay of control commands is less than a preset delay threshold, such as 1 millisecond. Specifically, this step includes the following sub-steps.
[0070] Step S301: High-frequency reading of terminal feedback data; During the execution loop, the real-time motion execution layer reads the feedback values from the joint encoder and the multi-dimensional torque sensor at the same frequency as the control frequency. The joint encoder provides the actual position and velocity information of each joint, and the multi-dimensional torque sensor is installed at the wrist of the robotic arm or at each joint, providing force and torque data in three directions.
[0071] Step S302 involves establishing a dynamic model and calculating the theoretical driving torque. This process, which converts the target trajectory points received from the local spatial perception layer into motor control commands, involves deep dynamic compensation. The dynamic compensation process utilizes the Lagrange equations to establish a physical model of the robotic arm and calculates the theoretical driving torque required for each joint in its current motion state in real time. The equations are:
[0072] in, This represents the joint driving torque vector, whose dimension is equal to the robot's degrees of freedom. Represents the joint position vector; Represents the joint velocity vector; Represents the joint acceleration vector; The inertia matrix is a symmetric positive definite matrix that is related to the joint position. This represents the Coriolis force and centrifugal force, which are related to joint position and velocity. This is the gravity term, representing the torque required for each joint to overcome gravity; The external contact torque is generated by the contact between the end effector and the environment and mapped to each joint.
[0073] Step S303: Torque compensation is implemented using a PID algorithm. The real-time action execution layer compares the actual external contact torque sensed by the torque sensor with the theoretically calculated value to obtain the torque deviation. Then, a proportional-integral-derivative (PID) algorithm is used to correct the motor's output current in real time. The input of the PID controller is the torque deviation, and the output is the current correction. The proportional term is used to quickly respond to the current deviation, the integral term is used to eliminate steady-state errors, and the derivative term is used to suppress overshoot. In this way, the system can compensate for positioning errors caused by mechanical backlash, nonlinear friction, and external disturbances.
[0074] Step S304, Flexible Grip and Force-Tactile Feedback Adjustment: When the drive actuator completes the gripping task, the real-time motion execution layer automatically adjusts the drive torque to achieve flexible gripping by sensing the force-tactile feedback inside the gripper. A thin-film pressure sensor is integrated inside the gripper to detect the gripping force in real time. The target value of the gripping force is determined based on the material properties of the workpiece. For example, for brittle workpieces with low hardness, the target gripping force is set to 5 Newtons; for metal workpieces with a high surface friction coefficient, the target gripping force is set to 15 Newtons. The system dynamically switches the target value within the preset torque range, effectively preventing damage to brittle workpieces.
[0075] Step S305: Switch to impedance control mode at the moment of contact; When handling assembly tasks and other processes involving contact interaction, the real-time motion execution layer switches to impedance control mode at the moment of contact. The impedance control model equates the robotic arm's end effector to a mass-spring-damped system, and its control law is:
[0076] in, This represents the external contact force vector between the end effector and the environment; Represents the desired end-effector pose vector; Represents the actual end-effector pose vector; This represents the desired terminal velocity vector, i.e. The first derivative with respect to time; This represents the actual end velocity vector, i.e. The first derivative with respect to time; This represents the desired terminal acceleration vector, i.e. The second derivative with respect to time; This represents the actual terminal acceleration vector, i.e. The second derivative with respect to time; This is a virtual inertia matrix used to adjust the inertial characteristics of the system.
[0077] This is a virtual damping matrix used to adjust the energy dissipation characteristics of the system; This is a virtual stiffness matrix used to adjust the elastic recovery characteristics of the system. By adjusting the virtual stiffness, virtual damping, and virtual mass parameters, the robotic arm can exhibit a preset compliance in the contact direction. This method allows the robotic arm to automatically guide to the target assembly position through physical contact force even with minor pose errors, increasing the assembly success rate to above a preset target value, such as 99.5%.
[0078] Step S306, Multi-axis Synchronous Control; The drive of the actuator also involves multi-axis synchronous control. Through the synchronous clock signal of the high-speed industrial fieldbus, the system ensures that the start-up time error of each joint motor is less than a preset synchronization error threshold, such as 10 microseconds. Specifically, the bus master station periodically broadcasts a synchronization signal, and each slave driver synchronously samples and updates control commands according to this signal. The consistency of the pace of each joint during movement ensures that the motion error of the end effector in spatial pose is less than a preset deviation threshold, such as 0.05 mm.
[0079] The steps S3 described above together constitute the complete workflow of the real-time motion execution layer. This layer runs on the robot controller terminal, reads sensor feedback at a high-frequency control cycle, calculates theoretical torque using the Lagrange dynamics model, and then achieves closed-loop compensation through a PID algorithm and torque sensor. Simultaneously, for grasping and assembly tasks, it implements flexible grasping based on force tactile feedback and compliant assembly based on impedance control, respectively, and ensures end-effector pose accuracy through multi-axis synchronous control.
[0080] After this layer of processing, the kinematic expectation from the upper trajectory flow is converted into the actual motor drive current, enabling the robotic arm to complete the task accurately and smoothly even in the presence of dynamic disturbances and environmental contact.
[0081] Based on steps S1 to S3, this method also includes an asynchronous collaboration mechanism between layers. This mechanism coordinates the operation and interaction of the global task planning layer, the local spatial perception layer, and the real-time action execution layer at different time scales. This mechanism will be described in detail below.
[0082] In this architecture, the global task planning layer runs on a cloud server and handles high-latency, high-computing-power semantic logic reasoning. Its decision frequency is set to 0.5 Hz to 2 Hz, that is, a complete logical decision and atomic action sequence update is performed every 0.5 seconds to 2 seconds. This frequency range is defined as the first preset frequency range.
[0083] The local spatial perception layer runs on the edge computing unit on the production line side and processes medium-frequency environmental modeling tasks. Its processing frequency is set to 30 Hz to 100 Hz, that is, it completes point cloud acquisition, registration and local map update every 10 milliseconds to 33 milliseconds. This frequency range is defined as the second preset frequency range.
[0084] The real-time motion execution layer runs on the robot controller terminal and performs extremely high-frequency closed-loop control. Its control frequency is set to 1000 Hz to 4000 Hz, that is, it completes torque calculation and current output every 0.25 milliseconds to 1 millisecond. This frequency range is defined as the third preset frequency range.
[0085] The three layers communicate asynchronously via industrial wireless networks or industrial Ethernet, and each layer operates independently without requiring clock synchronization.
[0086] When the local spatial perception layer detects that the environmental changes exceed the preset safety boundary value, this layer will immediately interrupt the execution of the current trajectory and send a replanning request to the global task planning layer, triggering a secondary decision by the logic layer. Specifically: The definition and detection method of the safety boundary value are as follows: The local spatial perception layer sets a three-dimensional safety boundary box for the critical operation area in the dynamic local map. The size of this boundary box is defined in the robot coordinate system as a cubic area centered on the target workpiece and extending outwards by a preset safety distance threshold, such as 50 mm. The system calculates the shortest distance between the robot arm end effector or the moving workpiece and this boundary box in real time. When this shortest distance is less than 0 mm, an intrusion occurs; or when the velocity vector of a dynamic obstacle in the environment points to the boundary box and is expected to intrude in the next control cycle, it is determined that the environmental change exceeds the preset safety boundary value.
[0087] The specific implementation method for interrupting the current trajectory execution is as follows: the local spatial perception layer sends an emergency stop command to the real-time motion execution layer via a high-speed industrial fieldbus. This command has a higher priority than the regular trajectory point command. Upon receiving the command, the real-time motion execution layer immediately stops the current motion and switches the robotic arm to zero-torque hold mode or moves it to the standby position according to a preset safe return trajectory.
[0088] The mechanism for sending the replanning request is as follows: Simultaneously with issuing the emergency stop command, the local spatial awareness layer sends a replanning request message to the global task planning layer on the cloud server via a message queue telemetry transport protocol or HTTP protocol. This message includes the current timestamp, anomaly type code (e.g., obstacle intrusion code 0x31), and the identifier of the affected action sequence. Upon receiving the request, the global task planning layer re-executes steps S102 and S103, generates a new atomic action sequence, and sends it to the local spatial awareness layer, thereby achieving secondary decision-making.
[0089] In flexible electronic product assembly lines, the robot's cycle time is significantly reduced using the aforementioned method. Because local deviations are absorbed at the perception layer and the underlying actuators possess strong dynamic compensation capabilities, the robot's repeatability accuracy remains consistently within the preset accuracy range.
[0090] On the other hand, the intelligent control system for the three-level collaborative production line disclosed in this application includes: The global task planning module, which runs on a cloud server, is used to process semantic parsing and logical decomposition within a first preset frequency range to generate atomic action sequences. The local spatial perception module runs on the edge computing unit on the production line side. It is used to receive atomic motion sequences at a second preset frequency range, and to construct a dynamic local map using real-time data from edge sensors. It determines the workpiece pose through point cloud registration and maps the atomic motion sequences into a continuous motion trajectory flow. The real-time motion execution module runs on the robot controller terminal. It is used to receive the instantaneous target pose in the motion trajectory stream at a third preset frequency range, read the terminal feedback data, calculate the theoretical driving torque by combining the Lagrange dynamics equation, and switch to the impedance control mode at the moment of contact to adjust the compliance and drive the actuator to complete the task. The asynchronous coordination module is used to realize asynchronous communication between the global task planning module, the local spatial perception module and the real-time action execution module through the industrial network, and to control the first preset frequency range, the second preset frequency range and the third preset frequency range to be distributed in a stepped manner.
[0091] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.
[0092] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A three-level collaborative intelligent control method for a production line, characterized in that, Includes the following steps: Establish a global task planning layer running on a cloud server or edge server, process semantic parsing and logical decomposition within a first preset frequency range, and generate atomic action sequences; A local spatial perception layer is established running on the edge computing unit on the production line side. It receives atomic motion sequences at a second preset frequency range, and uses real-time data from edge sensors to construct a dynamic local map. The workpiece pose is determined by point cloud registration, and the atomic motion sequences are mapped into a continuous motion trajectory flow. A real-time motion execution layer is established running on the robot controller terminal. It receives the instantaneous target pose in the motion trajectory stream at a third preset frequency range, reads the terminal feedback data, calculates the theoretical driving torque by combining the Lagrange dynamics equation, and switches to impedance control mode at the moment of contact to adjust compliance and drive the actuator to complete the task. An asynchronous collaboration layer is established to enable asynchronous communication between the global task planning layer, the local spatial perception layer, and the real-time action execution layer through the industrial network, and to control the first preset frequency range, the second preset frequency range, and the third preset frequency range to be distributed in a stepped manner.
2. The method according to claim 1, characterized in that, The steps to establish a global task planning layer running on a cloud server include: Acquire and parse long-range operation instructions, use deep neural networks based on attention mechanisms for intent recognition, and extract operation verbs, target objects, and process constraint parameters; The identified intent is decomposed into actions using a long short-term memory network, generating an ordered sequence of multiple atomic actions. Each atomic action is encapsulated in the data structure as a feature vector containing target object attributes, expected pose, and operation torque limit. An anomaly diagnosis module based on a support vector machine classifier is constructed to identify operational anomalies by monitoring the real-time data stream of the production line and to issue a safe shutdown order or a replanning request.
3. The method according to claim 1, characterized in that, The steps for establishing a local spatial awareness layer running on the edge computing unit of the production line include: Configure and collect sensor data from the red, green and blue depth cameras integrated at the end of the robot and the lidar distributed around the production line. The sampling frequency of the sensors shall not be lower than the preset sampling frequency. Statistical filtering is performed on the raw point cloud data acquired by the red-green-blue depth camera to remove outliers, and voxel downsampling is used to simplify the data. Point cloud data registration is performed, and an improved iterative nearest point algorithm is used to determine the precise pose of the workpiece. The registration process first performs coarse registration by extracting fast point feature histogram features, and then performs fine registration by using the point-to-plane error metric function. Based on the workpiece pose information obtained from registration, a local dynamic spatial model reflecting the workpiece pose and obstacle distribution is constructed. The elastic band algorithm is then used to apply an attractive force pointing towards the target pose and a repulsive force avoiding obstacles on the basis of the original reference trajectory, thereby generating a corrected local path.
4. The method according to claim 3, characterized in that, The steps for establishing a local spatial awareness layer running on the edge computing unit on the production line side also include: The pose sequence fed back by the sensor is predicted and smoothed by the Kalman filter algorithm to suppress the vibration interference of the conveyor belt and output a stable target pose sequence. The state vector of the Kalman filter is defined as the two-dimensional position and two-dimensional velocity of the workpiece on the conveyor belt plane, and the observation vector is provided by the position component in the pose obtained by registration.
5. The method according to claim 1, characterized in that, The steps for establishing a real-time motion execution layer running on the robot controller terminal specifically include: The feedback values of the joint encoder and the multi-dimensional torque sensor are read at a high frequency, which is the same as the third preset frequency range. The theoretical driving torque required by the robotic arm in the current motion state is calculated using the Lagrange dynamics equation. The equation is: the joint driving torque vector is composed of the product of the inertia matrix and the joint acceleration vector, the Coriolis force and centrifugal force terms, the gravity term and the external contact torque. Based on the deviation between the actual external contact torque sensed by the torque sensor and the theoretically calculated value, the output current of the motor is corrected in real time using a proportional-integral-derivative algorithm to compensate for positioning errors caused by mechanical clearance and nonlinear friction.
6. The method according to claim 5, characterized in that, The steps for switching to impedance control mode at the moment of contact to adjust compliance include: At the moment when the actuator completes the assembly task and comes into contact with the workpiece, the control mode is switched to the impedance control mode. The control law of this mode expresses the external contact force vector between the end effector and the environment as the sum of the products of the virtual inertia matrix, the virtual damping matrix and the virtual stiffness matrix with the desired acceleration deviation, the desired velocity deviation and the desired pose deviation, respectively. By adjusting virtual stiffness, virtual damping, and virtual mass parameters, the robotic arm can exhibit preset compliance in the contact direction, allowing it to be guided to the target assembly position by physical contact force even with minor pose errors.
7. The method according to claim 1, characterized in that, The steps for asynchronous communication between the global task planning layer, the local spatial awareness layer, and the real-time action execution layer via an industrial network specifically include: The decision frequency of the global task planning layer is set to 0.5 Hz to 2 Hz, forming the first preset frequency range; The processing frequency of the local spatial perception layer is set to 30 Hz to 100 Hz, forming a second preset frequency range; The control frequency of the real-time action execution layer is set to 1000Hz to 4000Hz, forming the third preset frequency range; Each level operates independently and does not require clock synchronization. When the local spatial perception layer detects that the amount of environmental change exceeds the preset safety boundary value, the current trajectory execution is immediately interrupted and a replanning request is sent to the global task planning layer, triggering a secondary decision by the logic layer.
8. The method according to claim 7, characterized in that, The steps for detecting environmental changes exceeding preset safety boundary values specifically include: In the dynamic local map, a three-dimensional safety boundary box is set for the critical operation area. The size of the boundary box is defined as a cubic area formed by expanding outward from the target workpiece with a preset safety distance threshold as the center. The system calculates the shortest distance between the end effector of the robotic arm or the moving workpiece and the bounding box in real time. When the shortest distance is less than 0 mm, an intrusion occurs. Or when the velocity vector of a dynamic obstacle in the environment points to the bounding box and is expected to intrude in the next control cycle, it is determined that the environmental change exceeds the preset safety boundary value. Interrupting the execution of the current trajectory specifically involves sending an emergency stop command with a higher priority than the regular trajectory point command to the real-time action execution layer via a high-speed industrial fieldbus. Sending a replanning request specifically involves sending a replanning request message containing a timestamp, anomaly type code, and an affected action sequence identifier to the global task planning layer of the cloud server via Message Queue Telemetry Transport Protocol or Hypertext Transfer Protocol.
9. The method according to claim 5, characterized in that, The steps for driving the actuator to complete the task also include: When the drive actuator completes the gripping task, it automatically adjusts the drive torque to achieve flexible gripping by sensing the force tactile feedback inside the gripper. The target value of the gripping force is determined according to the material properties of the workpiece and dynamically switches within the preset torque range. The synchronous clock signal of the high-speed industrial fieldbus ensures that the start-up time error of each joint motor is less than the preset synchronous error threshold, so that the pace consistency of each joint during the movement is maintained, and the motion error of the end effector in spatial pose is less than the preset deviation threshold.
10. A three-level collaborative intelligent control system for a production line, characterized in that, include: The global task planning module, which runs on a cloud server, is used to process semantic parsing and logical decomposition within a first preset frequency range to generate atomic action sequences. The local spatial perception module runs on the edge computing unit on the production line side. It is used to receive atomic motion sequences at a second preset frequency range, and to construct a dynamic local map using real-time data from edge sensors. It determines the workpiece pose through point cloud registration and maps the atomic motion sequences into a continuous motion trajectory flow. The real-time motion execution module runs on the robot controller terminal. It is used to receive the instantaneous target pose in the motion trajectory stream at a third preset frequency range, read the terminal feedback data, calculate the theoretical driving torque by combining the Lagrange dynamics equation, and switch to the impedance control mode at the moment of contact to adjust the compliance and drive the actuator to complete the task. The asynchronous coordination module is used to realize asynchronous communication between the global task planning module, the local spatial perception module and the real-time action execution module through the industrial network, and to control the first preset frequency range, the second preset frequency range and the third preset frequency range to be distributed in a stepped manner.