Industrial robot flexible job control method and system

Abstract

The application discloses an industrial robot flexible operation control method and system, and the method comprises the following steps: acquiring physical interaction representation of a robot and a workpiece to be processed, combining historical workpiece representation, determining a new task operation type through a task graph neural network, including reuse, combination or new establishment; generating or selecting corresponding strategy head parameters according to the operation type, combining environment observation to generate and execute mechanical arm action instructions; when the number of strategy heads exceeds a preset capacity, triggering compression maintenance, screening and retaining representative strategy heads to form an optimized active strategy library. The application realizes rapid strategy determination and generation based on physical interaction and graph structure relationship, significantly reduces manual demonstration and training time, and improves production line change response speed; through a structured compression mechanism, the strategy diversity and global knowledge are maintained under limited storage resources, and the adaptive ability and decision efficiency in long-term operation are guaranteed.

Description

Technical Field

[0001] This invention relates to the field of industrial robot control technology, and in particular to a flexible operation control method and system for industrial robots. Background Technology

[0002] As industrial manufacturing shifts towards a flexible production model characterized by diverse varieties and small batches, industrial robots need to frequently adapt to tasks involving workpieces of different shapes and sizes. Traditional control methods face significant challenges in this scenario: whenever the production line switches to a new type of workpiece, it often relies on engineers to provide manual teaching or to conduct long-term, data-intensive specialized training for that workpiece, resulting in prolonged production line downtime and severely restricting production efficiency and response speed.

[0003] Furthermore, while image- or point cloud-based representation methods can measure the similarity of workpiece appearance to some extent, their stability is easily affected by changes in lighting, occlusion, and the diversity of workpiece postures in complex industrial environments. More importantly, such methods are difficult to characterize the coupling relationship between deep physical dynamic characteristics such as grasping force, assembly resistance, and center of gravity position, making it impossible to effectively transfer strategies for tasks with different appearances but similar physical behaviors.

[0004] On the other hand, as the variety of workpieces accumulates, the limited memory of the robot controller struggles to handle the ever-growing volume of control strategies. Without proper management, this not only risks memory overflow but also negatively impacts real-time decision-making performance due to decreased strategy retrieval efficiency. Existing strategy reuse or transfer learning methods primarily focus on improving the adaptation speed of individual tasks, lacking mechanisms for systematically and structurally compressing and maintaining the strategy library itself. Consequently, they fail to balance the contradictions between strategy diversity, knowledge preservation, and hardware resource constraints during long-term operation. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problems of low production efficiency, poor adaptability and tight storage resources in the prior art when facing tasks to be processed, such as the need for long-term manual training or teaching, the difficulty of stably capturing physical and dynamic characteristics by characterization methods that rely solely on appearance, and the lack of systematic compression and maintenance of the ever-growing control strategy library.

[0006] To address the aforementioned technical problems, this invention provides a flexible operation control method for industrial robots, comprising: The physical interaction representation of the robot interacting with the workpiece to be processed is obtained. Combined with the historical workpiece representation, the operation type of the new task control strategy is determined by the task graph neural network. The operation type includes reuse, combination or creation. Based on the determined operation type, a corresponding control strategy head parameter is generated or selected. Based on the control strategy head parameter and environmental observation, a robotic arm action command is generated. The operation is executed according to the robotic arm action command. When the determined operation type is new creation or combination, a new strategy head is generated. When the cumulative number of strategy heads exceeds the preset capacity limit for the strategy head resource pool, a compression and maintenance of the strategy head set is triggered. Representative strategy heads are retained through relationship modeling and filtering to obtain an optimized active strategy library. The robot's operation is controlled based on the strategy heads in the active strategy library.

[0007] In one embodiment of the present invention, the method for obtaining the physical interaction representation when the controlled robot interacts with the workpiece to be processed is as follows: the controlled robot uses a predefined policy network to perform multiple exploratory interactions with the workpiece to be processed, and extracts the feature vector of each interaction to obtain the physical interaction representation; wherein, the number of exploratory interactions is adjusted according to the degree of dynamic change of the sensor state distribution during the interaction process.

[0008] In one embodiment of the present invention, the method for constructing and optimizing the task graph neural network is as follows: using a variational autoencoder to encode the physical interaction representation and historical workpiece representation when the control robot interacts with the workpiece to be processed, and obtaining the potential representation of each workpiece as a node of the task relationship graph; Based on the nodes, the edge weights between any two nodes are calculated using a trainable attention network to construct a task relationship graph; The parameters of the attention network are continuously optimized by utilizing the reward signal from reinforcement learning feedback, and the structure of the task relationship graph is dynamically adjusted to complete the construction and updating of the task graph neural network.

[0009] In one embodiment of the present invention, the method for determining the operation type of the new task control strategy through the task graph neural network is as follows: inputting the encoded workpiece node to be processed and the historical workpiece node into the constructed task relationship graph. Message passing and feature aggregation are performed on the task relationship graph to obtain a graph embedding representation of the workpiece to be processed; The graph embedding representation of the workpiece to be processed is input into the classifier, and the probability distribution of the operation type is output. The operation type includes reuse, combination, or creation. The parameters of the classifier are optimized using the reward signal from reinforcement learning feedback.

[0010] In one embodiment of the present invention, the method for generating or selecting the corresponding control strategy header parameters based on the determined operation type is as follows: If the operation type is reuse, the strategy header parameters corresponding to the historical artifact are loaded directly. If the operation type is combination, the strategy header parameters of multiple historical artifacts are linearly weighted to generate combined strategy header parameters; If the operation type is "New", then the new policy header parameters are initialized and optimized in subsequent training.

[0011] In one embodiment of the present invention, the method for generating robotic arm action commands based on the control strategy head parameters and environmental observations is as follows: a recurrent neural network is used to maintain historical state information; The control strategy head parameters, current environment observations, and the hidden states of the recurrent neural network are input into the action generation network to generate robotic arm action commands.

[0012] In one embodiment of the present invention, the method for compressing and maintaining the policy head set, and obtaining an optimized active policy library by retaining representative policy heads through relation modeling and filtering, is as follows: mapping all policy head parameters to low-dimensional embeddings and constructing a policy head relation graph; Message passing and feature aggregation are performed on the policy head relationship graph to obtain the graph-aware embedding of each policy head; When the cumulative number of strategy heads exceeds the preset capacity limit for the strategy head resource pool, representative strategy heads are iteratively selected from the current strategy head set based on the distance metric embedded in the graph perception, until the number of selected strategy heads reaches the preset value. The selected strategy heads are used to form the optimized active strategy library, and the remaining strategy heads are removed. At the same time, the embedded knowledge of the removed strategy heads is archived.

[0013] The present invention also provides a flexible operation control system for industrial robots, comprising the following modules: The operation type determination module is used to acquire the physical interaction representation when the control robot interacts with the workpiece to be processed, and combine it with the historical workpiece representation to determine the operation type of the new task control strategy through the task graph neural network. The operation type includes reuse, combination or creation. The strategy execution module is used to generate or select the corresponding control strategy head parameters based on the determined operation type, generate robotic arm action instructions based on the control strategy head parameters and environmental observations, and execute the operation according to the robotic arm action instructions. When the determined operation type is new creation or combination, a new strategy head is generated. The strategy library maintenance module is used to trigger the compression and maintenance of the strategy head set when the cumulative number of strategy heads exceeds the preset capacity limit for the strategy head resource pool. It obtains an optimized active strategy library by retaining representative strategy heads through relationship modeling and filtering, and controls the robot's operation based on the strategy heads in the active strategy library.

[0014] The present invention also provides an electronic device, which includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to realize the flexible operation control method for the industrial robot.

[0015] The present invention also provides a computer storage medium, characterized in that the computer storage medium stores a computer software product, the computer software product including a plurality of instructions for causing a computer device to execute the industrial robot flexible operation control method.

[0016] The technical solution of the present invention has the following advantages compared with the prior art: The industrial robot flexible operation control method and system described in this invention introduces a task graph neural network to perform physical interaction representation and relationship modeling of the workpiece to be processed. Based on its inherent physical similarity to historical workpieces rather than its apparent characteristics, it can determine the operation type of the appropriate strategy, reducing the time required for manual teaching or specialized training when dealing with workpieces and improving the response speed and changeover efficiency of the production line. Simultaneously, an intelligent compression and maintenance mechanism for the strategy head set is designed. When the strategy library exceeds a preset capacity, the most representative strategy heads can be selected through relationship modeling to form an optimized active strategy library. This achieves systematic and structured management of massive control strategies within limited controller storage resources, ensuring stable performance and efficient decision-making during long-term system operation. Attached Figure Description

[0017] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0018] Figure 1 This is a flowchart illustrating the flexible operation control method for industrial robots in an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall framework of hierarchical reinforcement learning in the embodiment; Figure 3 This is a schematic diagram of the task graph construction process in the embodiment; Figure 4 This is a schematic diagram of the resource pool maintenance process in the embodiment; Figure 5 This is a schematic diagram of the structure of the flexible operation control system for industrial robots in an embodiment of the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0020] Example 1: like Figure 1 As shown, the present invention provides a flexible operation control method for industrial robots, comprising: The physical interaction representation of the robot interacting with the workpiece to be processed is obtained. Combined with the historical workpiece representation, the operation type of the new task control strategy is determined by the task graph neural network. The operation type includes reuse, combination or creation. Based on the determined operation type, a corresponding control strategy head parameter is generated or selected. Based on the control strategy head parameter and environmental observation, a robotic arm action command is generated. The operation is executed according to the robotic arm action command. When the determined operation type is new creation or combination, a new strategy head is generated. When the cumulative number of strategy heads exceeds the preset capacity limit for the strategy head resource pool, a compression and maintenance of the strategy head set is triggered. Representative strategy heads are retained through relationship modeling and filtering to obtain an optimized active strategy library. The robot's operation is controlled based on the strategy heads in the active strategy library.

[0021] The flexible operation control method for industrial robots provided by this invention analyzes the physical interaction between the workpiece to be processed and the historical workpieces through a task neural network, determines the operation type (reuse, combination, or new creation) of the appropriate strategy, and then generates or selects the corresponding control parameters to drive the robotic arm to perform the operation. At the same time, a resource pool management mechanism is introduced, which automatically compresses and filters when the number of strategy heads exceeds the preset capacity, retains representative strategies to form an optimized strategy library, thereby realizing rapid adaptation to new tasks and long-term, efficient maintenance of the strategy library, improving the flexible response capability and resource utilization efficiency of the production line.

[0022] This invention proposes a control method based on hierarchical reinforcement learning. For example... Figure 2 As shown, this control method constructs a hierarchical reinforcement learning framework of "high-level decision-making - low-level execution". By introducing task graph representation and dynamic maintenance mechanism of resource pool, it realizes rapid adaptation of the workpiece to be processed and sustainable management of the strategy library.

[0023] At higher levels, this method does not directly output specific motion trajectories. Instead, it uses a task graph neural network to analyze the physical interaction representations of the workpiece to be processed and historical workpieces, and determines the appropriate strategy to generate operation type. The above determination process is based on the latent representation of the workpiece extracted by the variational autoencoder and the task relationship graph constructed by the graph neural network. It is continuously optimized through reinforcement learning feedback, thereby stably capturing the physical and dynamic coupling relationship between tasks in complex industrial environments and avoiding misjudgments caused by relying solely on appearance similarity.

[0024] At the lower level, the method generates or selects corresponding strategy head parameters based on the operation type determined by the higher level. It then combines real-time environmental observations (such as joint angles and force feedback) with the historical state maintained by the recurrent neural network to generate specific robotic arm action commands, achieving precise operation. During this process, if the operation type is "new" or "combined," a new strategy head will be generated.

[0025] To address the storage and computational pressures caused by the continuous growth in the number of policies, this method also includes a resource pool management step. This step is automatically triggered when the number of policy heads exceeds the controller's preset memory capacity. It embeds the policy heads into a structured graph representation using variational graph encoding, and performs representative filtering and compression based on distance metrics of graph-aware embedding. The most diverse and comprehensive policy heads are retained to form an active policy library, while the embedding knowledge of removed policy heads is archived. This maintains long-term online learning capabilities and decision-making efficiency under limited hardware resources.

[0026] Specifically, the method for obtaining the physical interaction representation when the controlled robot interacts with the workpiece to be processed is as follows: the controlled robot uses a predefined strategy network to conduct multiple exploratory interactions with the workpiece to be processed, and extracts the feature vector of each interaction to obtain the physical interaction representation; wherein, the number of exploratory interactions is adjusted according to the degree of dynamic change of the sensor state distribution during the interaction process.

[0027] For each historical artifact task i, first utilize the predefined policy network. Control the robot and the work environment where the workpiece is located. This is a secondary interaction. A predefined policy network maintains a consistent architecture and parameter initialization method across all tasks. Its inputs are environmental state characteristics, including robot joint angles, force feedback from the end effector, and visual depth information of the workpiece. The outputs are the corresponding motion distribution parameters, such as the torque command distribution of the joint motors.

[0028] For artifact task i, at the end of the j-th exploratory interaction, extract the output of the last hidden layer of the policy network. The feature vector is used to characterize the robot's physical response to the task i in this interaction. The average of the feature vectors generated from n interactions is used to obtain the comprehensive characterization vector of task i. : .

[0029] like Figure 3 As shown, for each artifact task, the output of the policy network is a feature vector. In the workpiece task graph neural network, the relationships between different workpiece tasks are captured through node representations. Based on this, we generate a node for the new workpiece to be processed and fuse it with the existing historical workpiece graph. This process forms a new task node by calculating the average of the tentative interaction feature vectors of the new workpiece and connecting it to historical workpiece nodes to better model the physical relationship between new and old workpieces.

[0030] In real-world industrial scenarios, a fixed number of interactions It is difficult to simultaneously meet the dual requirements of production cycle efficiency and characterization accuracy: for simple workpieces with regular shapes and stable centers of gravity, excessive interaction leads to wasted time; while for flexible or irregularly shaped workpieces, insufficient interaction fails to fully capture key physical characteristics such as friction coefficient and center of gravity position. Therefore, this embodiment introduces an adaptive adjustment mechanism for interaction quotas based on the dynamic changes in sensor state distribution.

[0031] Specifically, it is first defined in the time interval. Internal sensor state distribution difference measurement : , in, It represents the empirical probability of state s in the sampled trajectory at time t. Size of the state set. Measure of state distribution difference. This reflects the physical dynamics of the workpiece during the trial operation: if A large value indicates that the sensor feedback exhibits significant instability and uncertainty when the robot handles the workpiece, suggesting that the workpiece may wobble or slip. The number of trial interactions, n, is determined by the following formula: , in Based on the number of basic interactions, This is an adjustment coefficient used to control the sensitivity of the number of interactions to changes in distribution. When the physical feedback of the workpiece is stable ( The number of interactions is close to This avoids redundant sampling when the workpiece is in a stable state, saving computing power and time; when the workpiece feedback changes significantly, the number of interactions decreases accordingly. The linear increase allows for a more refined physical characterization of the workpiece.

[0032] Furthermore, after obtaining the physical interaction representation of the workpiece generated based on the dynamic interaction quota strategy as described above, in order to achieve accurate determination of the new task operation type, it is necessary to further construct and optimize the task graph neural network.

[0033] Specifically, a pre-trained task variational autoencoder is used to extract the physical interaction representation when the robot interacts with the workpiece. The encoder encodes the representations of historical artifacts to obtain a compact representation of each artifact in the latent space. Specifically, the encoder... Map high-dimensional interaction features to latent variables that follow a Gaussian distribution. This latent representation serves as the node features in the task relationship graph, thereby achieving a low-dimensional, structured expression of the workpiece's physical characteristics. During the training phase, the decoder is used to calculate the reconstruction loss, ensuring that the latent space can accurately reconstruct the workpiece's physical characteristics. Its loss function... Including reconstruction error and KL divergence, the formula is: , in, This is the approximate posterior distribution of the encoder output. As a prior distribution, For decoder functions, The KL divergence is used. Once training is complete, the trained task variational autoencoder will be used to generate latent representations for each task. These latent representations will be used to construct the node features of the task graph.

[0034] Based on the aforementioned node features, a task relationship graph is further constructed using a trainable attention network to model the association strength between different workpiece nodes. For each node pair... Their joint representation is obtained by concatenating vectors: , in, This indicates a vector concatenation operation.

[0035] Subsequently, an attention network is used to calculate the edge weights (i.e., the association strength between workpieces) between workpiece nodes: , in, and These are trainable parameters. The adjacency matrix A is a sigmoid activation function, ensuring that the weights are in the interval [0,1]. To ensure the symmetry and stability of the graph structure, the adjacency matrix A is normalized to obtain the final adjacency matrix. : .

[0036] Building upon this, to dynamically optimize the graph structure and better reflect the physical coupling relationships between tasks, this embodiment introduces a reward mechanism based on reinforcement learning feedback to continuously update the attention network parameters. Specifically, during the robot's task execution, the system generates an immediate reward signal based on the actual task performance. This reward consists of three parts: similarity reward, decision reward, and structure regularization reward. These rewards respectively encourage nodes with similar physical characteristics in the graph to connect closely, support effective policy reuse or combination behavior, and maintain appropriate complexity and uniformity of the graph. The parameter update objective of the attention network is to maximize the long-term cumulative reward. , in, It is a discount factor. It is the immediate reward at time step t. The reward signal mainly consists of the sum of three parts: similarity reward, decision reward, and structure regularization reward.

[0037] Similarity rewards are based on the cosine similarity between workpiece nodes. Quantifying the similarity of physical properties encourages workpiece nodes with similar physical properties in the graph (such as two different sizes of cylindrical pins) to have higher connection weights, helping the attention network to connect workpieces that are very similar in the feature space more closely.

[0038] The formula for expressing similarity reward is: , in, Let i be the feature vector of node i.

[0039] Decision rewards encourage job task maps to effectively represent the collaborative relationships between tasks, making the connections in the map meaningful for actual control decisions. When strategy reuse or combination occurs (e.g., reusing the strategy of gear A to pick up gear B), if the success rate of the operation is high, the decision reward increases, thereby rewarding favorable combinations of connections between jobs.

[0040] Decision rewards are used to guide task graph neural networks to learn to make effective decisions in real-world physical tasks. Specifically, the goal is to encourage the reuse or combination of strategies that yield high success rates in real-world scenarios, thereby optimizing the practical meaning of connections between nodes in the task graph. To this end, decision rewards... The definition is directly linked to the operation type determined by the system and the final execution effect, and its calculation formula is as follows: , in, The performance score represents the actual operation performance, which is quantified by the success indicators of the robot after completing the action (such as grasping stability, assembly accuracy, task completion time, etc.).

[0041] When the system determines that the task is "reused" or "combined," if the success rate of subsequent operations is high ( If the reward value is high, a positive reward is given, thus reinforcing this type of judgment; if the task fails, the reward value is low or negative, prompting the network to adjust its decisions in similar future scenarios.

[0042] When the system determines the decision to "create a new instance," the reward is set to 0. This is because the effectiveness of the new instance strategy is highly dependent on subsequent online training optimization, and it is not appropriate to directly use the initial execution effect to evaluate the preceding decision-making process, thus maintaining a neutral incentive for exploratory decisions.

[0043] Structural regularization rewards are used to maintain graph stability, preventing excessive density or sparsity and ensuring that connections between different tasks are reasonable and representative. Graph complexity is controlled by the F-norm of the adjacency matrix and the variance of each row. The formula for expressing it is: , Where A is the adjacency matrix of the graph. It is the variance of each row of the adjacency matrix, representing the degree of unevenness in the connectivity of the graph; It is the F-norm, which measures the complexity of a graph. , This is the adjustment coefficient.

[0044] After the task graph G is constructed, message passing and aggregation will be performed on the task graph G. The update of node i at level k is represented as: , in This represents the representation of workpiece node i after the update at the k-th layer. Let i represent the set of neighbors of node i. It is a learnable weight matrix. After multiple iterations, each workpiece node obtains a final representation that integrates neighborhood information, providing input for subsequent task judgment.

[0045] Furthermore, after completing the construction of the task relationship graph and obtaining the graph embedding representation of each workpiece, the system further uses a classifier to determine the operation type of the new task.

[0046] Specifically, after message passing and feature aggregation in a multi-layer graph neural network, each workpiece node obtains its final graph embedding representation. , among which the former Each node corresponds to a representation of a historical workpiece, and the last node corresponds to a representation of the new workpiece to be processed. The classifier uses the final graph embedding representation of the workpiece to be processed. As input, the output is a three-dimensional probability distribution. These correspond to the confidence levels of the three operation types: "reuse," "combination," and "new creation," respectively. The classification process is formally modeled as a three-class classification problem, with the decision logic being: if... If it is significantly higher than the other two categories, it is determined to be a strategy header that reuses a historical artifact; if If it is dominant, it is determined to be a strategy head that combines multiple historical artifacts; if If the result is the highest, it is determined that a completely new strategy head needs to be initialized.

[0047] To improve the accuracy of the classifier in actual operations, its parameters are not trained through traditional supervised learning methods, but are continuously optimized through the reward signals of reinforcement learning feedback.

[0048] Specifically, in each training cycle, the system generates an instant reward signal based on the actual performance of the robot's tasks, and uses this signal to calculate gradients and update the weights of the classifier. To ensure that the reward signal is closely related to the robot's actual physical performance, this embodiment uses a metric based on the robotic arm's trajectory behavior characteristics to construct the reward function.

[0049] For any robotic arm motion trajectory ,in, , , They are time points Status, actions, and immediate rewards are determined through a task variational autoencoder. Extract the state features at each time step and average them along the time dimension to obtain a compact feature representation of the trajectory: .

[0050] Based on this, corresponding reward calculation methods are designed for different operation types determined by the classifier. If the system determines that a historical workpiece is being reused... The strategy head then calculates the actual trajectory of the robot when handling new workpieces. With historical artifacts Reference trajectory The feature vector, and the reuse reward is defined as: , in, This is a normalization constant. The smaller the difference in trajectory features, the higher the reward, indicating a better reuse effect.

[0051] If a new task is generated through a strategy combination of multiple historical artifacts, its reward is defined as: , in, For combination coefficients, This represents the combined historical task trajectories. The reward reflects the degree of matching between the new task trajectory and the linearly weighted combination of historical trajectory features.

[0052] If the system determines that a new strategy header has been created, the cumulative job reward on the new workpiece is recorded as follows: The reward is defined as: , in, This represents the cumulative return of the newly created strategy head. As a performance threshold, The reward values ​​are divided into two levels to encourage new strategies to reach or exceed certain performance levels. The goal of updating the classifier's parameters is to maximize the expected value of the long-term cumulative reward, guided by the reward signals based on actual physical operation results. Through this reinforcement learning mechanism that directly links classification decisions to actual operation performance, the classifier can progressively optimize its internal representation and decision boundaries. This allows it to make more accurate and efficient judgments among the three operation modes of reuse, combination, and creation when faced with new workpieces, significantly shortening the adaptation time to new tasks and improving the overall operation performance and response speed of industrial robots in flexible production environments.

[0053] Furthermore, after determining the operation type of the new task based on the task graph neural network, the process enters the strategy execution stage, where corresponding control strategy header parameters are generated or selected based on the operation type determined by the higher level.

[0054] Specifically, the lower-level motion controller generates strategy head parameters for the new workpiece in different ways based on the operation type output by the higher-level controller. .

[0055] If the operation type is "reuse", the system directly loads the historical workpiece corresponding to the judgment result from the controller's storage. strategy header parameters As Use, i.e. .

[0056] This method is suitable for scenarios where the new workpiece is highly similar to a historical workpiece in terms of physical characteristics. For example, when the new workpiece is a product from the same batch as standard part A, the stored capture parameters of standard part A can be directly called to achieve millisecond-level rapid production changeover adaptation and improve the response efficiency of the production line.

[0057] If the operation type is "combination", then based on a set of preset or learned weight coefficients, the strategy header parameters of multiple related historical artifacts are linearly weighted to generate combined strategy header parameters: , in, The number of historical artifacts involved in the combination. For the first The strategy header parameters for each historical artifact. For the corresponding weights, they usually satisfy the following conditions: and To maintain consistency in parameter scale.

[0058] This approach allows the control strategy for new workpieces to inherit and integrate the characteristics of multiple existing strategies, making it suitable for new workpieces with composite physical characteristics, such as workpieces that simultaneously possess the gripping characteristics of cylinders and the placement characteristics of planar parts.

[0059] If the operation type is "New", a new set of random strategy header parameters is initialized and gradually converged through optimization algorithms such as gradient descent during subsequent exploratory interactions or online training to adapt to the new type of workpiece task.

[0060] This approach is primarily designed for irregularly shaped parts or parts that do not have a significant similarity to historical workpieces. It involves gradually building effective control strategies through online learning.

[0061] By using the above three strategies to generate operations, the system can flexibly generate corresponding control model parameters according to the different requirements of the physical characteristics of the workpiece, thereby achieving precise connection between the operation type determined by the higher level and the execution parameters of the lower level.

[0062] Furthermore, based on the high-level management module, the operation type (reuse, combination, or new creation) of the new workpiece operation strategy is determined, and the corresponding strategy header parameters are specified. Afterwards, the lower-level motion controller enters the real-time motion generation and decision-making stage, the core task of which is to process the strategy header parameters determined by the higher level. By combining the current environmental observation information, specific and executable robotic arm action instructions are generated.

[0063] To achieve this goal, the lower-level controller adopts a time-dependent action decision mechanism. By modeling historical state information, it captures the time-dependent relationships in the operation process, thereby making more robust and continuous action decisions.

[0064] Specifically, at each time t, the input of the i-th industrial robot (i.e., the intelligent agent) consists of two parts: one is the local environmental observation at the current time. The first part includes angle readings from the joint encoder, force / torque sensor feedback from the end effector, and local visual features of the workpiece (such as region of interest descriptors in depth images); the second part is the strategy head parameters specified by the high-level task graph neural network based on the current workpiece task. Together, these two constitute the input to the lower-level controller, namely: .

[0065] This ensures that the low-level motion controller, when generating actions, not only relies on real-time sensor perception of the working environment but also incorporates decision-making guidance from higher levels. This enables the robot to dynamically switch its behavior modes based on strategy parameters provided by higher levels when facing different production line tasks, achieving seamless control from "strategy type selection" to "specific action execution."

[0066] To model long-term dependencies in action decision-making, the low-level controller uses a recurrent neural network to maintain the robot's internal hidden state. The hidden state acts as a compact memory of historical interactions for the system. Its state update process is represented as follows: , in, This represents the robot's implicit memory state at time t. The state update function for an RNN is defined by the selected policy header parameters. The influence of different strategy head parameters can correspond to different dynamic characteristics of RNNs, thus enabling the same control architecture to adapt to diverse task behavior patterns. Compared with other network structures with temporal modeling capabilities, recurrent neural networks have significant advantages in terms of computational complexity and memory overhead, making them particularly suitable for multi-robot parallel training or resource-constrained embedded controller deployment scenarios, ensuring decision continuity while maintaining overall operational efficiency.

[0067] In obtaining Then, the lower-level controller generates the action distribution through linear transformation: , Based on action distribution Sampling yields specific actions This design ensures that, under a unified network structure, different policy header parameters... Guide different robotic arm behavior patterns.

[0068] In summary, by introducing a recurrent neural network to maintain the temporal state and using high-level policy head parameters, current environmental observations, and historical hidden states as inputs for action generation, the above method achieves robust and continuous action decision-making in partially observable environments. This design enables different policy head parameters to guide differentiated robotic arm behavior patterns within a unified low-level control network architecture, truly realizing the flexible control concept of "shared control architecture, differentiated operation parameters," and significantly improving the adaptive capability and execution accuracy of industrial robots in complex and dynamic operation scenarios.

[0069] Furthermore, during the long-term operation of an industrial robot system, as new workpiece tasks continue to emerge, the strategy heads generated by the system through "new creation" or "combination" operations will gradually accumulate, leading to a continuous increase in the storage and computing burden on the controller, which may even exceed the hardware resource limits, thereby affecting the online inference speed and decision-making accuracy.

[0070] To address this issue, this embodiment designs an intelligent strategy head resource pool maintenance mechanism. This mechanism can automatically trigger a compression and filtering process when the number of strategy heads exceeds a preset capacity limit. Through structured relationship modeling and representative head selection, it constructs an optimized active strategy library that is scalable, comprehensive, and maintains diversity, thereby achieving long-term, efficient, and stable job control under limited memory conditions.

[0071] Specifically, the system sets a capacity limit for the strategy head resource pool. This upper limit is typically determined based on the industrial controller's physical memory capacity and actual computing power. Whenever the cumulative number of policy heads exceeds... When this happens, the system automatically initiates the compression and maintenance process. The goal of compression is not simply to delete policy heads in chronological order or by usage frequency, but rather to select the most representative subset from the current policy head set that best covers the distribution of existing job skills, based on a full consideration of the semantic relationships and functional differences between different policy heads, as the new active policy library.

[0072] like Figure 4 As shown, to achieve this goal, structured relationship modeling is performed on all policy heads, and each policy head... The parameter vector can be flattened into a high-dimensional vector. Where P represents the total number of parameters. The flattened strategy head parameter vector is... Figure 4 The strategy head is then mapped to a low-dimensional latent space using a head variational autoencoder to obtain low-dimensional embeddings. in The specific construction process is as follows: Figure 4 As shown in the policy graph neural network module in (a), after obtaining the low-dimensional embedding, adjacency relationships are constructed based on the attention mechanism. Following the construction method of the task graph, the latent representations of all policy heads are organized into a graph structure. This step facilitates capturing the geometric and semantic relationships between different task skills in the overall parameter space. On the constructed graph, the policy graph neural network further performs message passing and feature aggregation to obtain a structured graph-aware embedding: , in, This represents the node representation after the update at layer k. Gather for the neighbors, For activation function, The weight matrix is ​​a learnable matrix. This is an aggregation function. Unlike simple vector similarity, the graph convolutional embeddings generated by policy graph neural networks can simultaneously encode local neighborhood structure and global distribution features, thus more accurately measuring the redundancy and differences between policy heads.

[0073] To prevent the strategy head resource pool from growing indefinitely, the system sets a capacity limit on the number of strategy heads in the pool. This upper limit is typically determined by the physical memory of the industrial controller. For example... Figure 4 As shown in (b), when the current number of strategy heads exceeds When the time comes, compression is automatically triggered. The goal of compression is not simply to delete the oldest or least frequently used policy heads, but to retain a representative subset in a continuous embedding space that reflects the structural relationships between policy heads, so that it covers the underlying manifold where all policy heads are located as evenly as possible, thereby maintaining the diversity and generalization ability of the policy library with limited resources.

[0074] Specifically, the k-center greedy algorithm is used to achieve an approximate optimal solution: , , , , in, This indicates the strategy headers that already exist in the historical artifact tasks. This represents the selected set of representative strategy heads.

[0075] In the initial stage, the robot's policy pool may contain a large number of policy heads targeting workpieces with subtle differences. When the number of policy heads exceeds a set upper limit K, the system triggers a compression process. In the first round, It is the starting center, and calculates all remaining policy heads (such as...). , ...) distance to the selected set of centers. During the calculation, the system selects the distance from the starting center ( The furthest strategy head (e.g.) The system then adds the selected center to the set of centers. Next, iterates, selecting the next furthest policy head as the new center, until K center policy heads are selected. Figure 4 In (b), the selected K central strategy heads are denoted as... , , , ..., Through the above iterative process, the final selected The central strategy heads constitute the reserved strategy head pool (i.e., the optimized active strategy library). These strategy heads will be retained in the resource pool for subsequent reuse and combination; the remaining unselected strategy heads (such as...) , …) This constitutes a set of policy head pools, and policy heads in this set will be removed from the resource pool to free up storage space.

[0076] like Figure 4 As shown in (c), the graph-aware embeddings corresponding to all removed policy heads are stored in a lightweight vector archive and continue to participate as graph nodes in the training and updating of subsequent policy graph neural networks. This mechanism ensures that even after specific control parameters are deleted, the system can still retain long-term memory and distribution awareness of the job policy for that type of workpiece, thereby effectively mitigating the "catastrophic forgetting" problem that may be caused by policy compression and maintaining the system's global adaptability and knowledge continuity in long-term operation.

[0077] In summary, by introducing a strategy head resource pool maintenance mechanism based on k-center greedy compression, the industrial robot system retains at most a certain number of resource pools during runtime. A core control policy head is used, which significantly reduces the storage and computational burden on the controller. During training, both currently active policy heads and historical embeddings from the archive are incorporated, and the semantic and structural relationships between policy heads are further modeled using attention mechanisms and graph convolution.

[0078] In actual operation, the system always makes decisions and controls based on the strategy heads in the active strategy library. This ensures the diversity and representativeness of strategies even with strictly limited storage resources, significantly reducing the reliance on manual teaching and repeated training. It enables industrial robots to quickly adapt, efficiently utilize resources, and achieve stable and reliable flexible operation control in multi-variety, small-batch production scenarios.

[0079] Example 2: like Figure 5 As shown, the present invention also provides an industrial robot flexible operation control system for implementing the industrial robot flexible operation control method described in Embodiment 1, including the following modules: an operation type determination module, used to obtain the physical interaction representation when the control robot interacts with the workpiece to be processed, and combine the historical workpiece representation to determine the operation type of the new task control strategy through a task graph neural network, wherein the operation type includes reuse, combination or creation; The strategy execution module is used to generate or select the corresponding control strategy head parameters based on the determined operation type, generate robotic arm action instructions based on the control strategy head parameters and environmental observations, and execute the operation according to the robotic arm action instructions. When the determined operation type is new creation or combination, a new strategy head is generated. The strategy library maintenance module is used to trigger the compression and maintenance of the strategy head set when the cumulative number of strategy heads exceeds the preset capacity limit for the strategy head resource pool. It obtains an optimized active strategy library by retaining representative strategy heads through relationship modeling and filtering, and controls the robot's operation based on the strategy heads in the active strategy library.

[0080] Example 3: The present invention also provides an electronic device, which includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the flexible operation control method for industrial robots described in Embodiment 1.

[0081] Example 4: The present invention also provides a computer storage medium storing a computer software product, the computer software product including a plurality of instructions for causing a computer device to execute the industrial robot flexible operation control method described in Embodiment 1.

[0082] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0083] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0084] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0086] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A flexible operation control method for an industrial robot, characterized in that, include: The physical interaction representation of the robot interacting with the workpiece to be processed is obtained. Combined with the historical workpiece representation, the operation type of the new task control strategy is determined by the task graph neural network. The operation type includes reuse, combination or creation. Based on the determined operation type, a corresponding control strategy head parameter is generated or selected. Based on the control strategy head parameter and environmental observation, a robotic arm action command is generated. The operation is executed according to the robotic arm action command. When the determined operation type is new creation or combination, a new strategy head is generated. When the cumulative number of strategy heads exceeds the preset capacity limit for the strategy head resource pool, a compression and maintenance of the strategy head set is triggered. Representative strategy heads are retained through relationship modeling and filtering to obtain an optimized active strategy library. The robot's operation is controlled based on the strategy heads in the active strategy library.

2. The industrial robot flexible operation control method according to claim 1, characterized in that: The method for obtaining the physical interaction representation when the controlled robot interacts with the workpiece to be processed is as follows: the controlled robot uses a predefined strategy network to conduct multiple exploratory interactions with the workpiece to be processed, and extracts the feature vector of each interaction to obtain the physical interaction representation; wherein, the number of exploratory interactions is adjusted according to the degree of dynamic change of the sensor state distribution during the interaction process.

3. The industrial robot flexible operation control method according to claim 1, characterized in that: The method for constructing and optimizing the task graph neural network is as follows: use a variational autoencoder to encode the physical interaction representation and historical workpiece representation when the robot interacts with the workpiece to be processed, and obtain the potential representation of each workpiece as a node of the task relationship graph. Based on the nodes, the edge weights between any two nodes are calculated using a trainable attention network to construct a task relationship graph; The parameters of the attention network are continuously optimized by utilizing the reward signal from reinforcement learning feedback, and the structure of the task relationship graph is dynamically adjusted to complete the construction and updating of the task graph neural network.

4. The industrial robot flexible operation control method according to claim 3, characterized in that: The method for determining the operation type of the new task control strategy through the task graph neural network is as follows: input the encoded workpiece node to be processed and the historical workpiece node into the constructed task relationship graph. Message passing and feature aggregation are performed on the task relationship graph to obtain a graph embedding representation of the workpiece to be processed; The graph embedding representation of the workpiece to be processed is input into the classifier, and the probability distribution of the operation type is output. The operation type includes reuse, combination, or creation. The parameters of the classifier are optimized using the reward signal from reinforcement learning feedback.

5. The industrial robot flexible operation control method according to claim 1, characterized in that: The method for generating or selecting the corresponding control strategy header parameters based on the determined operation type is as follows: If the operation type is reuse, the strategy header parameters corresponding to the historical artifact are loaded directly. If the operation type is combination, the strategy header parameters of multiple historical artifacts are linearly weighted to generate combined strategy header parameters; If the operation type is "New", then the new policy header parameters are initialized and optimized in subsequent training.

6. The industrial robot flexible operation control method according to claim 1 or 5, characterized in that: The method for generating robotic arm motion commands based on the control strategy head parameters and environmental observations is as follows: a recurrent neural network is used to maintain historical state information; The control strategy head parameters, current environment observations, and the hidden states of the recurrent neural network are input into the action generation network to generate robotic arm action commands.

7. The industrial robot flexible operation control method according to claim 1, characterized in that: The method for compressing and maintaining the set of strategy heads, and obtaining an optimized active strategy library by retaining representative strategy heads through relationship modeling and filtering, is as follows: map all strategy head parameters to low-dimensional embeddings and construct a strategy head relationship graph; Message passing and feature aggregation are performed on the policy head relationship graph to obtain the graph-aware embedding of each policy head; When the cumulative number of strategy heads exceeds the preset capacity limit for the strategy head resource pool, representative strategy heads are iteratively selected from the current strategy head set based on the distance metric embedded in the graph perception, until the number of selected strategy heads reaches the preset value. The selected strategy heads are used to form the optimized active strategy library, and the remaining strategy heads are removed. At the same time, the embedded knowledge of the removed strategy heads is archived.

8. A flexible operation control system for an industrial robot, characterized in that, Includes the following modules: The operation type determination module is used to acquire the physical interaction representation when the control robot interacts with the workpiece to be processed, and combine it with the historical workpiece representation to determine the operation type of the new task control strategy through the task graph neural network. The operation type includes reuse, combination or creation. The strategy execution module is used to generate or select the corresponding control strategy head parameters based on the determined operation type, generate robotic arm action instructions based on the control strategy head parameters and environmental observations, and execute the operation according to the robotic arm action instructions. When the determined operation type is new creation or combination, a new strategy head is generated. The strategy library maintenance module is used to trigger the compression and maintenance of the strategy head set when the cumulative number of strategy heads exceeds the preset capacity limit for the strategy head resource pool. It obtains an optimized active strategy library by retaining representative strategy heads through relationship modeling and filtering, and controls the robot's operation based on the strategy heads in the active strategy library.

9. An electronic device, characterized in that, The electronic device includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the flexible operation control method for industrial robots according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer software product, the computer software product including a number of instructions for causing a computer device to execute the industrial robot flexible operation control method according to any one of claims 1 to 7.

Images