Process migration control method and device for flexible assembly task switching

Abstract

The invention discloses a process migration control method and device for flexible assembly task switching, and relates to the technical field of flexible assembly. The method comprises the following steps: acquiring original total data in a first process; taking the minimum sample loss function as an optimization target, and obtaining a distillation candidate subset from the original total data; the sample loss function is used for measuring the retention degree of the distillation candidate subset on the behavior distribution of the original total data; calculating a comprehensive score of each action in the distillation candidate subset through a distribution consistency score and an action border-crossing punishment, filtering out sample data with relatively large comprehensive scores of the actions in combination with a gating mechanism, and storing the sample data meeting the gating mechanism into a small-capacity buffer area; and finally, training the initial strategy network according to the sample data of the small-capacity buffer area and the observation data of the second process to obtain a new strategy network. The method can give consideration to both online speed and stability under the condition of limited resources.

Description

Technical Field

[0001] This application relates to the field of flexible assembly technology, and in particular to a process migration control method and apparatus for flexible assembly task switching. Background Technology

[0002] Flexible assembly lines typically feature high-variety, low-batch, and rapid changeover production, requiring frequent switching between processes such as pick-and-place, alignment, pressing, and closing of robotic arms. To shorten setup time and maintain stable yield rates, the industry is gradually introducing data-driven strategy learning and task experience utilization methods to reuse operational experience from older processes in newer ones.

[0003] In existing technologies, learning schemes that rely on full-scale experience consume a large amount of storage and computing power, have long training and deployment cycles, and are prohibitively expensive to implement on-site. Therefore, a small number of samples are usually randomly selected from the full dataset. However, this approach lacks representativeness and feasibility, and is prone to including out-of-bounds actions or distorted observations in the experience replay exploitation library, amplifying alignment biases and compression anomalies, making it difficult to guarantee the quality of the initial policy network obtained through training.

[0004] Therefore, existing technologies lack a method that balances deployment speed and stability under limited resource conditions. Summary of the Invention

[0005] Therefore, it is necessary to provide a process migration control method and apparatus for flexible assembly task switching to address the above-mentioned technical problems.

[0006] The following technical solution is adopted in this specification: This specification provides a process migration control method for flexible assembly task switching, where the flexible assembly task includes multiple processes; the method includes: In response to the switch request from the first process to the second process, the original full data of the first process is obtained; the original full data includes multiple sets of sample data obtained by interacting with the first process task using the initial policy network; each set of sample data includes the state data of the robotic arm and the corresponding action; The optimization objective is to minimize the sample loss function, and then obtain a distillation candidate subset from the original full data. The sample loss function is constructed by the kernel ridge regression reconstruction error between the original full data and the distillation candidate subset in the feature space of the initial policy network, and is used to measure the degree to which the distillation candidate subset preserves the behavior distribution of the original full data. The encoder of the initial policy network is used as a feature extractor to calculate the distribution consistency score between the distillation candidate subset and the original full data, and the out-of-bounds penalty value for each action in the distillation candidate subset is calculated according to the safe working space of the robotic arm. For any action in the distillation candidate subset, the distribution consistency score and the action's out-of-bounds penalty value are weighted to obtain a comprehensive score for the action; sample data corresponding to actions in the distillation candidate subset whose comprehensive score is less than or equal to the first gating threshold are all stored in a small-capacity buffer. The initial policy network is trained based on the sample data from the small buffer and the observation data from the second process to obtain the new policy network. The joint optimization loss function used in the training of the initial policy network is constructed based on the behavioral differences between the new policy network and the initial policy network, as well as the task performance of the new policy network.

[0007] Optionally, the sample loss function for: ; in, and These represent the state data and corresponding actions in the original full dataset, respectively. and These represent the state data and corresponding actions in the candidate distillation subset to be screened, respectively. and All are kernel matrices. express and Feature similarity between express Similarity to its own features For regularization terms, For adjustment coefficients, It is the identity matrix, and the superscript -1 indicates the inverse of the corresponding parameter.

[0008] Optionally, the encoder of the initial policy network is used as a feature extractor to calculate the distribution consistency score between the distillation candidate subset and the original full data, including: The encoder of the initial policy network is used as a feature extractor to extract features of each state data in the distillation candidate subset and the original full data, respectively. Based on the characteristics of each state data in the candidate distillation subset and the original full dataset, the statistical characteristics of the state data in the candidate distillation subset and the original full dataset are calculated respectively; the statistical characteristics include mean and covariance. The distribution consistency score is calculated based on the mean and covariance of the state data in the distillation candidate subset and the original full dataset.

[0009] Optionally, distribution consistency score The calculation formula is: ; in, and Let A and B represent the mean and covariance of the state data in the candidate distillation subset, respectively. and These represent the mean and covariance of the state data in the original full dataset, respectively. Indicates trace, Represents the L2 norm. This represents the square of the L2 norm.

[0010] Optionally, action Outbound penalty value The calculation formula is: ; in, Indicates action lower bound ， Indicates action The upper realm, Represents the L2 norm. Denotes the square of the L2 norm. Indicates the lower boundary penalty factor. This represents the out-of-bounds penalty factor.

[0011] Optionally, jointly optimize the loss function. for: ; ; ; in, The performance metrics of the new strategy network are calculated using the observation data from the second process. For distillation loss, To align weights, Expressing expectations, The weights representing the importance of states in relation to the uncertainty of the old strategy. Indicates trace, and This represents the initial policy network's prediction of actions based on sample data in a small buffer, including the mean. and variance ; and This indicates that the new policy network predicts actions based on sample data with a small buffer size, including the mean. and variance , express The reverse, express transpose, It is a positive number.

[0012] Optionally, the method further includes: For abnormal actions in the distillation candidate subset whose comprehensive score is greater than the first gate threshold, if the comprehensive score threshold of the abnormal action is greater than or equal to the second gate threshold, or if the out-of-bounds penalty value of the abnormal action is greater than or equal to the penalty threshold, then the sample data corresponding to the abnormal action is discarded; the second gate threshold is greater than the first gate threshold.

[0013] If the overall score threshold of the abnormal action is greater than the first gating threshold but less than the second gating threshold, and the out-of-bounds penalty value of the abnormal action is less than the penalty threshold, then the sample data corresponding to the abnormal action will be corrected, and the corrected sample data will be re-gated.

[0014] Optionally, the sample data corresponding to the abnormal action is corrected, including: Adjust the camera's exposure parameters, re-collect the state data corresponding to the abnormal actions, and determine the newly collected state data and corresponding actions as the corrected sample data; Alternatively, the state data corresponding to the abnormal action can be augmented, and the augmented state data and the corresponding action can be used as the corrected sample data.

[0015] Optionally, the method further includes: When the small buffer is full, the sample data with the earliest timestamp is deleted first.

[0016] This specification provides a process migration control device for flexible assembly task switching, where the flexible assembly task includes multiple processes; the device includes: The acquisition module is used to respond to the switching request from the first process to the second process and acquire the original full data under the first process; the original full data includes multiple sets of sample data obtained by interacting with the first process task using the initial policy network; each set of sample data includes the state data of the robotic arm and the corresponding action; The first screening module is used to obtain distillation candidate subsets from the original full data with the goal of minimizing the sample loss function. The sample loss function is constructed by the kernel ridge regression reconstruction error between the original full data and the distillation candidate subsets in the feature space of the initial policy network, and is used to measure the degree to which the distillation candidate subsets preserve the behavior distribution of the original full data. The second screening module is used to use the encoder of the initial policy network as a feature extractor to calculate the distribution consistency score between the distillation candidate subset and the original full data, and to calculate the out-of-bounds penalty value for each action in the distillation candidate subset based on the safe working space of the robotic arm; for any action in the distillation candidate subset, the distribution consistency score and the out-of-bounds penalty value of the action are weighted to obtain the comprehensive score of the action; and the sample data corresponding to the actions in the distillation candidate subset whose comprehensive score is less than or equal to the first gating threshold are stored in a small-capacity buffer. The training module is used to train the initial policy network based on the sample data in the small-capacity buffer and the observation data in the second process to obtain the new policy network. The joint optimization loss function used in the training process of the initial policy network is constructed based on the behavioral differences between the new policy network and the initial policy network, as well as the task performance of the new policy network.

[0017] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described process migration control method for flexible assembly task switching.

[0018] This specification provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described process migration control method for flexible assembly task switching.

[0019] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: The process migration control method for flexible assembly task switching provided in this specification, compared to existing technologies that rely heavily on all past experience or randomly sample a small number of data points, making it difficult to balance representativeness and deployment cost, ensures executable and reusable data from the source by selecting representative sample data. This allows for immediate deployment and use on the production floor, significantly improving learning efficiency and stability after task switching. Furthermore, addressing the issue that existing technologies often employ resampling or trajectory experience-based screening, lacking statistical consistency constraints for the production floor and failing to prevent out-of-bounds samples from entering the experience replay library, this specification introduces explicit gating in the distillation stage. Using both statistical indicators and feasible regions as dual standards, executableness and deployability are prioritized on the data side, adapting to complex conditions such as reflections and occlusions, significantly improving the stability and controllability of continuous learning. This method balances deployment speed and stability under limited resource conditions. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0021] Figure 1 This specification provides a flowchart illustrating a process migration control method for flexible assembly task switching. Figure 2The diagram shows a set of benchmark tasks for flexible assembly with multiple processes. (a) shows a hammering positioning task, (b) shows a pushing against the wall task, (c) shows a faucet closing task, (d) shows an item pushing backward task, (e) shows a connecting rod pulling out task, (f) shows a handle lateral pressing task, (g) shows a linear pushing task, (h) shows an item placing into a shelf task, (i) shows a window closing task, and (j) shows a pin lateral pulling out task. Figure 3 The diagram shows the data distillation process and the one-step interactive completion process for data entry in the flexible assembly state action distillation of the present invention. (a) is a schematic diagram of the data distillation process, and (b) is a flowchart of the data completion and writing process. Figure 4 The diagram shows the overall framework of the migration control system for flexible assembly task switching of the present invention, wherein (a) is a flowchart of strategy push, regression testing and online confirmation after multi-process training, and (b) is a flowchart of experience consistency distillation and replay of continuous training. Figure 5 A graph showing the trend of success rate under flexible assembly with multiple process switching as the number of training sessions increases; Figure 6 A graph showing the relationship between the size of a small buffer zone and the average success rate of flexible assembly; Figure 7 A graph showing the impact of alignment weights on the average success rate of flexible assembly; Figure 8 The following are typical on-site operation sequence diagrams for flexible assembly machine tools. (a) is the operation sequence diagram for the hammering positioning process, (b) is the operation sequence diagram for the linear pushing process, (c) is the operation sequence diagram for the pushing against the wall process, and (d) is the operation sequence diagram for the placing into the rack process. Figure 9 This is a framework diagram of the flexible assembly old process experience distillation and current process strategy learning of the present invention; Figure 10 This is a schematic diagram of a computer device for implementing a process migration control method for flexible assembly task switching, as provided in this specification. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0023] However, factors such as glare, lighting variations, and slight shading in the production line environment make the sensing and control links susceptible to interference, making it difficult to balance the efficiency of experience reuse with online stability. Existing technologies suffer from several problems: First, relying on learning schemes based on full-scale experience utilization consumes significant storage and computing power, resulting in long training and deployment cycles and prohibitive on-site resource costs. Second, the practice of randomly selecting a small number of samples lacks representativeness and feasibility review, easily leading to out-of-bounds actions or distorted observations being written into the experience playback utilization library, amplifying alignment deviations and pressing anomalies. Third, simple behavior cloning or training on a single new task neglects the maintenance of old process capabilities, often resulting in gripping force drift and trajectory fine-tuning instability after model changeovers, leading to substantial rework and parameter adjustments. Fourth, the lack of distribution consistency constraints and quality gating for the production site makes it difficult to cope with distribution drift caused by glare and shading. Fifth, the lack of an update rhythm and interface mechanism coordinated with the control link makes it difficult to stably implement and maintain learning results.

[0024] Based on this, this invention aims to address flexible assembly scenarios with multiple product types and small batches, proposing a process migration control method and device for flexible assembly task switching that can be rapidly deployed on the production floor. By selecting a small, representative experience set (distillation candidate subset) from the original full dataset, storage and computing device usage are reduced, shortening the closed-loop cycle from data acquisition to strategy update. By introducing distributed consistency gating, the observations and actions of distillation samples are constrained to remain within the assembly feasible domain, ensuring data compliance and executability, and improving the stability and controllability of experience utilization. Through an uncertainty-weighted strategy alignment mechanism, key control characteristics of the old process are maintained during the training of the new process, while ensuring continuous performance improvement of the new process, thus aligning and achieving stable migration between the old and new processes. The above method, coupled with standardized interfaces and update rhythm management, can seamlessly integrate with existing control links, enabling rapid commissioning, consistent quality, and controllable maintenance during model changeovers. It balances deployment speed, control stability, and quality consistency under limited resource conditions, and provides a unified technical foundation for future expansion.

[0025] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0026] Figure 1 This is a flowchart illustrating a process migration control method for flexible assembly task switching as described in this specification, which specifically includes the following steps: S101, in response to the switching request from the first process to the second process, obtain the original full data under the first process; the original full data includes multiple sets of sample data obtained by interacting with the first process task using the initial policy network; each set of sample data includes the state data of the robotic arm and the corresponding action.

[0027] In this process, the first and second processes can be any two adjacent processes in the flexible assembly task. The strategy network of the first process is used as the initial strategy network. Sample data from the first process is used as empirical data, and observation data from the second process is used to train the initial strategy network to obtain a new strategy network. The new strategy network retains the capabilities of the initial strategy network and adds control capabilities for the second process. For example, the flexible assembly task includes processes such as picking and placing, alignment, pressing, and closing. The first process can be the alignment process, and the second process can be the pressing process. The initial strategy network can make decisions on the actions of the picking and placing and alignment processes, and the new strategy network can make decisions on the actions of the picking, placing, alignment, and pressing processes.

[0028] In the flexible assembly task, the actions are continuous. Therefore, the input of the initial policy network is the state data of the robotic arm, and the output is the action. The action output by the initial policy network is the parameter of the action probability distribution, which may include the mean and variance. The state data can be image data collected by the camera and / or state parameters of the robotic arm collected by the sensor, representing the corresponding assembly scenario.

[0029] like Figure 2 As shown, Figure 2 This is a schematic diagram of a multi-process benchmark task set for flexible assembly. Specifically, Figure 2 Figure (a) in the diagram is a schematic diagram of the hammer-based positioning task. Figure 2 Figure (b) in the diagram is a schematic diagram of the task of pushing against the wall. Figure 2 Figure (c) in the diagram is a schematic diagram of the faucet closing task. Figure 2 Figure (d) in the diagram illustrates the task of pushing items backward. Figure 2 Figure (e) in the diagram is a schematic diagram of the linkage extraction task. Figure 2 Figure (f) in the diagram is a schematic diagram of the lateral pressing task of the handle. Figure 2 Figure (g) in the diagram is a schematic diagram of a linear push task. Figure 2 The diagram (h) in the image is a schematic diagram of the task of placing items into the shelf. Figure 2 Figure (i) in the diagram is a schematic diagram of the window closing task. Figure 2 Figure (j) in the figure is a schematic diagram of the task of pulling out the pin laterally.

[0030] S102 aims to minimize the sample loss function and obtain a distillation candidate subset from the original full data. The sample loss function is constructed by the kernel ridge regression reconstruction error between the original full data and the distillation candidate subset in the feature space of the initial policy network. It is used to measure the degree to which the distillation candidate subset preserves the behavioral distribution of the original full data.

[0031] Using the features extracted by the initial policy network as a metric, a kernel ridge regression matching is established between the original full dataset and the candidate distillation subset to be screened. The aim is to reconstruct the behavioral distribution of the original large-scale samples (the original full dataset) in the feature space from the selected small-scale samples (the candidate distillation subset). Specifically, the sample loss function... for: (1); in, and These represent the state data and corresponding actions (mean) in the original full dataset, respectively. and These represent the state data and corresponding actions (mean) in the candidate distillation subset to be screened. and All are kernel matrices. express and Feature similarity between express Similarity to its own features This is a regularization term used to prevent numerical instability during matrix inversion. For adjustment coefficients, It is an identity matrix.

[0032] To avoid the computational overhead of directly calculating high-dimensional features, we use the conjugate kernel approximation, which represents the kernel matrix as the inner product of the features extracted by the encoder of the initial policy network. For example, , Encoder extraction of the initial policy network Features Encoder extraction of the initial policy network Features for The transpose of the initial policy network has the following parameters: ; This is actually measuring the similarity between two assembly scenarios at the neural network perception level.

[0033] By minimizing the sample loss function, a small number of key samples (candidate distillation subsets) that can best explain or represent the behavioral characteristics of the original full data can be selected, thereby achieving high-fidelity data compression.

[0034] To address the challenges of frequent product turnover and rapid switching between processes like pick-and-place, alignment, pressing, and closing in flexible assembly production lines, the difficulty of reusing old operational experience in new tasks is a significant concern. Utilizing all past experience consumes substantial storage and computing power, leading to significant deployment delays and continuously escalating maintenance costs. Camera shake and reflections cause variations in observation distribution, resulting in inconsistent experience quality and unstable training and deployment. This embodiment addresses these issues by reorganizing and simplifying data within the assembly feasible domain. First, representative states and actions are selected from the old process data (original full data) to form a small-scale distillation candidate subset. Subsequently, a single interaction can be completed in the simulation environment or a controlled field to supplement the next moment's state, reward, and end marker, forming a complete experience tuple that can be directly used for backtracking. Deployed on the production field using a small-capacity buffer, a standardized data interface and writing strategy are provided to ensure rapid integration with existing controllers. This method emphasizes the synergy of representativeness, usability, and deployability, preserving the key feel of assembly operations while significantly reducing storage and computing power pressure, providing stable input for subsequent changeover learning and rapid deployment.

[0035] S103, using the encoder of the initial policy network as a feature extractor, calculates the distribution consistency score between the distillation candidate subset and the original full data, and calculates the out-of-bounds penalty value for each action in the distillation candidate subset based on the safe working space of the robotic arm; for any action in the distillation candidate subset, the distribution consistency score and the out-of-bounds penalty value of the action are weighted to obtain the comprehensive score of the action; the sample data corresponding to actions in the distillation candidate subset whose comprehensive score is less than or equal to the first gating threshold are all stored in a small-capacity buffer.

[0036] To form a complete and directly usable experience set, the selected states and actions undergo a one-step interaction in a simulation or controlled environment to obtain the next-moment state, reward, and termination flag. The observations and actions are then checked to ensure they are within the assembly feasible region (distribution consistency score and action boundary violation penalty) to intercept and isolate out-of-bounds errors such as abnormal brightness, strong reflections, high-speed jitter, and illegal fixture poses. The termination flag is a logical signal in the reinforcement learning experience tuple, used to identify whether the task has reached its endpoint state after the current flexible assembly action is completed, such as successful assembly, collision failure, or safe shutdown. It, along with state, action, and reward data, constitutes a "complete experience tuple," enabling the new policy network to accurately identify task boundaries and calculate long-term benefits when replaying data from a small buffer, thus ensuring no logical breaks or training instability occur during process transitions. The flexible assembly state / action and one-step interaction completion and data entry process is as follows: Figure 3 As shown, Figure 3 Figure (a) in the diagram is a schematic diagram of the data distillation process. Figure 3 Figure (b) shows the data completion and writing flowchart, specifically, the sample data from the original full dataset... and Data filtering was performed to obtain a candidate distillation subset. The sample data in the candidate distillation subset includes... and After obtaining the candidate distillation set, the sample data in the candidate distillation set is completed, and the reward value is calculated. Next state and termination mark Then ( , , , , The data is stored in a small buffer. However, due to the glare from metal parts, changes in lighting, and slight occlusion, the observation distribution can easily drift. If the distillation samples deviate from the actual assembly data or exceed the action boundaries, alignment errors, pressing abnormalities, and safety risks may occur. After entering the experience playback library, these unstable factors will be amplified, causing problems such as fluctuations in training effects and uncontrollable deployment on the production site. To ensure the executability of the written samples, before writing the sample data into the small buffer, the sample data in the distillation candidate subset can be intercepted and isolated using out-of-bounds interception and anomaly isolation strategies. Cases such as abnormal brightness, strong glare, high-speed jitter, and non-permitted fixture poses can be eliminated or recycled, ensuring the stability of subsequent training.

[0037] A combined strategy of capacity capping and time decay is used to complete write and eviction processes, ensuring that the buffer maintains critical coverage of older processes even at a relatively small scale. The buffer provides a unified data format, timestamps, and synchronization markers to guarantee stable interface with the host computer and controller. Data flow and module boundaries are as follows... Figure 4 As shown, Figure 4 This is a general framework diagram of a migration control system for flexible assembly task switching. Figure 4 (a) in the diagram is a flowchart of the strategy push, regression testing, and deployment confirmation process after multi-step training. Figure 4 (b) in the diagram is a flowchart of the experience consistency distillation and replay process for continuous training; specifically, in the training phase of this process in (a), for Task... t -2 to Task t +1 and a series of flexible assembly processes undergo continuous strategy iteration and learning, and the resulting control algorithm is pushed to the production site control terminal; in the verification phase, the control terminal performs regression testing on the covered multi-process tasks, and evaluates the consistency of the robot arm's operation under different assembly tasks through simulation or on-site assessment, thereby completing the final confirmation before going online, ensuring that the introduction of new processes will not lead to the degradation of the performance of existing tasks; (b) In the figure, when executing the current task Task tDuring this process, the original full data in the historical experience database is compressed with high fidelity through the experience distillation module, and the selected representative samples are stored in a small-capacity buffer. Simultaneously, as the robotic arm interacts with the working environment in real time to generate current experience, it periodically replays old process experiences from the small-capacity buffer. Through weighted fusion and joint optimization of new and old experiences, the policy network can quickly grasp the current task. t At the same time, it can effectively maintain the historical task through distribution consistency constraints. t -1 control techniques enable long-term transfer of control strategies.

[0038] Interferences such as metal reflection, time-varying illumination, and local occlusion are common in the production site, which can cause drift in the statistical characteristics of camera observation and end-effector actions. If such samples are written into a small-capacity buffer, the alignment error and pressing abnormality are easily amplified after playback.

[0039] Therefore, consistency verification and gating are performed before writing the sample data of the distillation candidate subset into the lower capacity buffer, and only samples that are statistically consistent with the field data and are executable are retained.

[0040] This embodiment serves as a quality and safety inspection gate, deployed between the distillation candidate subset and the small-capacity buffer. Using the distillation candidate subset generated in the previous step and the statistical characteristics of the original full data as a benchmark, a set of compliant experience that has undergone rigorous screening, is statistically consistent, and ensures safe operation is obtained. This data is then stored in the small-capacity buffer. The specific operations are as follows:

[0041] In one embodiment, the encoder of the initial policy network is used as a feature extractor to calculate the distribution consistency score between the distillation candidate subset and the original full data. This includes: using the encoder of the initial policy network as a feature extractor to extract features of each state data in the distillation candidate subset and the original full data respectively; calculating statistical features of the state data in the distillation candidate subset and the original full data respectively based on the features of each state data in the distillation candidate subset and the original full data; the statistical features include mean and covariance; and calculating the distribution consistency score based on the mean and covariance of the state data in the distillation candidate subset and the original full data.

[0042] Specifically, feature statistics are extracted and compared. To determine whether the current state data (image) is normal, it's not sufficient to rely solely on visual inspection; statistical comparison within the feature space of computer vision is required. This embodiment utilizes the encoder of the initial policy network as a feature extractor to calculate the statistical features of the state data in the distillation candidate subset and the original full dataset, respectively. For the distillation candidate subset... Calculate the mean of its features. Covariance :

[0043] (2); (3); in, This indicates the number of state data in the distillation candidate subset. The encoder of the initial policy network extracts the first... The characteristics of each state data point, with the superscript T indicating transpose.

[0044] Among them, for the original full data Similarly, calculate its mean. With covariance .

[0045] The mean represents the average appearance of the assembly scene. If... Leave A relatively distant location indicates either excessively strong or dim lighting, or the presence of unfamiliar background objects. Covariance represents the range of feature variation and correlation. Anomalies suggest that the current batch of data suffers from issues such as excessive high-frequency noise and texture artifacts due to strong reflections, or a lack of data diversity.

[0046] Based on the above statistics, a dual scoring system is constructed that includes distribution consistency and action safety: distribution consistency score and action out-of-bounds penalty value.

[0047] Optionally, the distribution consistency score uses the Fréchet Inception Distance (FID) to quantify the distance between the distilled candidate subset and the original full data in terms of distribution. The calculation formula is: (4); in, and Let A and B represent the mean and covariance of the state data in the candidate distillation subset, respectively. and These represent the mean and covariance of the state data in the original full dataset, respectively. Indicates trace, Represents the L2 norm. This represents the square of the L2 norm. The smaller the distribution consistency score, the closer the currently collected status data is to the real, normal production environment, and the less affected it is by abnormal interference.

[0048] To prevent the storage of dangerous actions that could lead to collisions with the robotic arm or damage to the workpiece, a penalty for actions exceeding the limit is introduced; Outbound penalty value The calculation formula is: (5); in, Indicates action lower bound ， Indicates action The upper realm, Represents the L2 norm. Denotes the square of the L2 norm. Indicates the lower boundary penalty factor. Indicates the lower boundary penalty factor; ; .

[0049] and The safe working space of the robotic arm is defined, including joint angle limits and maximum end-effector thrust constraints. Only when the action... The penalty is only applied when the action exceeds this range. It can represent the mean of the actions output by the initial policy network.

[0050] Overall score The calculation formula is: (6); in, , Weights for state importance. Overall score. The lower the value, the higher the data quality and the safer the data for that action. Indicates the score for distribution consistency. The normalization process involves mapping the Fréchet distance between the distillation candidate subset calculated using formula (4) and the original full data to a uniform scale, i.e., between 0 and 1. This is done to eliminate the influence of dimensions, thereby achieving consistency with the action boundary penalty value. The effective weighted fusion of these factors together forms a comprehensive score for evaluating data quality. .

[0051] In one embodiment, this embodiment provides a three-branch gating decision logic. Based on the comprehensive score and the gating threshold, the sample data in the distillation candidate subset is divided into three processing branches:

[0052] The first branch is for passing the threshold: when When the threshold is less than or equal to the first gating threshold, the sample set is considered uniform and the action is executable, allowing writing to a small-capacity buffer. "Statistical consistency of the sample set" means that the statistical distribution (i.e., the mean and covariance of the features) of the distilled candidate subset is consistent with that of the original full data in the feature space. This means that the selected samples can accurately retain the behavioral distribution characteristics of the original full data, and their statistical properties are not significantly shifted due to environmental interference such as metal reflection or changes in lighting.

[0053] The second branch is direct rejection (Reject): or In extreme cases, samples deemed illegal or dangerous, such as those with severely improper clamping positions or those exposed to direct strong light from lenses, are discarded immediately, and a log alert is generated to prevent contamination of the experience database.

[0054] The third branch is correctable backtracking (Retrospect): when G If the image value is greater than the first threshold but less than the second threshold, and the deviation can be corrected by resampling or exposure compensation, then the image enters a backtracking reprocessing phase. The system can instruct the camera to adjust exposure parameters and re-acquire the image, or perform preprocessing such as histogram equalization at the algorithm level, and then resubmit the image for gating check.

[0055] Therefore, after obtaining the comprehensive score of each action in the distillation candidate subset, the relationship between the comprehensive score of the action and the first gating threshold is determined, and the sample data corresponding to the actions in the distillation candidate subset whose comprehensive score is less than or equal to the first gating threshold are stored in a small-capacity buffer.

[0056] Actions with a comprehensive score greater than the first gating threshold are identified as anomalous actions. For anomalous actions in the distillation candidate subset with a comprehensive score greater than the first gating threshold, if the comprehensive score threshold of the anomalous action is greater than or equal to the second gating threshold, or if the out-of-bounds penalty value of the anomalous action is greater than or equal to the penalty threshold, then the sample data corresponding to the anomalous action is discarded. The second gating threshold is greater than the first gating threshold.

[0057] If the overall score threshold of the abnormal action is greater than the first gating threshold but less than the second gating threshold, and the out-of-bounds penalty value of the abnormal action is less than the penalty threshold, then the sample data corresponding to the abnormal action will be corrected, and the corrected sample data will be re-gated.

[0058] In one embodiment, correcting the sample data corresponding to the abnormal action includes: adjusting the camera's exposure parameters, re-acquiring the state data corresponding to the abnormal action, and determining the newly acquired state data and the corresponding action as the corrected sample data; or, enhancing the state data corresponding to the abnormal action, and determining the enhanced state data and the corresponding action as the corrected sample data. The enhancement operation includes preprocessing the image corresponding to the state data, such as histogram equalization.

[0059] In one embodiment, in order to maintain the timeliness and representativeness of experience within a limited storage space, the present invention provides a write eviction strategy: when sample data in the distillation candidate subset is written to a small-capacity buffer after passing a gating check, the sample data with the earliest timestamp is deleted first when the small-capacity buffer is full.

[0060] In this embodiment, the gating-compliant sample data is written to a small-capacity buffer and linked to a write-out policy to maintain representativeness and executability within a limited capacity.

[0061] In one embodiment, when the comprehensive scoring threshold of the abnormal action is greater than or equal to the second gating threshold, or when the out-of-bounds penalty value of the abnormal action is greater than or equal to the penalty threshold, an alarm signal is issued to block the writing and recording of batch information for traceability.

[0062] like Figure 5 As shown, Figure 5 This graph shows the trend of success rate under flexible assembly multi-process switching as the number of training iterations increases. Tasks 0 to 9 correspond to ten typical flexible assembly processes: Task 0 is hammering for positioning, Task 1 is pushing against the wall, Task 2 is closing the faucet, Task 3 is pushing the item backward, Task 4 is pulling out the linkage, Task 5 is pressing the handle laterally, Task 6 is pushing in a straight line, Task 7 is placing the item into the shelf, Task 8 is closing the window, and Task 9 is pulling out the pin laterally. The trend indicates that as the number of training iterations increases, the high success rate of the old processes can be effectively maintained while introducing new processes. This directly verifies the technical advantage of the transfer control method in this application in balancing learning efficiency and control stability in a multi-task environment.

[0063] To address the issue that reflections from metal parts, changes in lighting, and slight occlusion can easily cause observational distribution drift, distillation samples that deviate from actual assembly data or exceed action boundaries are prone to alignment errors, pressing anomalies, and safety risks. These instabilities are amplified after entering the experience playback library, leading to fluctuations in training effectiveness and uncontrollable deployment on the production floor. This embodiment designs a distribution consistency gating mechanism before the distillation candidate subset enters the small-capacity buffer. This mechanism compares the characteristics and covariance of the sample states of the distillation candidate subset with the original full data, calculates the distribution differences, and generates a comprehensive score. Only sample data that meets the comprehensive score requirement is written into the small-capacity buffer; sample data with a deviating comprehensive score are back-processed or directly removed. Simultaneously, by combining the feasible domain constraints of assembly observations and actions, key indicators such as image brightness, reflected light, fixture pose, and end-effector amplitude are verified to ensure that the data entering the buffer is both representative and executable, improving the stability of subsequent training and deployment from the source.

[0064] S104. Based on the sample data in the small-capacity buffer and the observation data in the second process, the initial policy network is trained to obtain the new policy network. The joint optimization loss function used in the training process of the initial policy network is constructed based on the behavioral differences between the new policy network and the initial policy network, as well as the task performance of the new policy network.

[0065] When switching from data crawling to assembly, if only observation data from the new process is used for training during the new process deployment phase, the initial policy network often fits the new process perfectly, quickly forgetting key skills from the old process, resulting in catastrophic forgetting, experience loss, and unstable control. The goal of this invention is to utilize the high-quality old experience (sample data in a small buffer) output from the gated inspection of flexible assembly distribution consistency, and through an algorithmic mechanism, enable the new policy network to quickly master the new process task while robustly retaining the capabilities of the old process, achieving seamless switching.

[0066] This embodiment builds upon the preceding steps, forming a complete utilization loop. It employs dual data stream joint training: one is the real-time data stream, which is the observation data generated by the robotic arm during the current new process; the other is the playback data stream, which is the sample data (the experience set from the old processes) from the distribution-consistent gating output, which has undergone rigorous screening in a small buffer. The output is an anti-forgetting initial policy network that is capable of handling the new process without forgetting old techniques; the training objective is to align the policy output distribution with the old process data and continuously optimize performance on the new process data.

[0067] Specifically, a dual-channel training mechanism is established to ensure the synchronous intake of new and old knowledge. Channel one is for new knowledge learning, which utilizes real-time observation data and rewards from the current second step, employing temporal difference or its variants as a value learning objective to drive the initial policy network to adapt to the new environment. Channel two is for reviewing old knowledge, periodically sampling data from a small-capacity experience buffer to align the distribution of the mean and variance output of the initial policy network. Note that the sample data used here is quality-checked to ensure the training process is not misled by reflective or out-of-bounds samples. An update frequency and sampling ratio are set; for example, every 10 steps of new data are collected, followed by two batches of old data for review. This achieves a computational balance between exploring new knowledge and consolidating existing knowledge. The sampling ratio of the second-step observation data to the sample data obtained from the small-capacity buffer in the initial policy network's training data is set to 10:2.

[0068] First, the policy difference (KL divergence) is calculated using sample data obtained from a small buffer. Then, the KL divergence is used to measure the new policy network. With the initial policy network Behavioral differences: (7); in, Indicates trace, and This represents the initial policy network's prediction of actions based on sample data in a small buffer, including the mean. and variance ; and This indicates that the new policy network predicts actions based on sample data with a small buffer size, including the mean. and variance , express The reverse, express The transpose of .

[0069] This formula mathematically binds the new policy. It considers the distribution of actions of the new policy in a given state, including the mean. or variance It deviates too far from the strategies of experienced teachers. It will grow larger and result in punishment.

[0070] Secondly, this embodiment introduces state importance weights based on the uncertainty of the old strategy. : (8); in, is a positive constant, which can be used to represent the smallest positive constant used to prevent the denominator from being zero and to ensure the stability of numerical calculations.

[0071] State importance weight Following the monotonic change in variance of the initial policy network on the sample data, when When the value is large, the importance weight of the state is adjusted. A small value indicates that the initial policy network is also uncertain about this state, or that it is an unknown domain. In this case, relaxing constraints allows the new policy to explore boldly, thus avoiding excessive constraints on the performance of the new process. Hourly, State Importance Weight A large value indicates that this is a key skill that the initial policy network is very confident in (such as fine-tuning during alignment). At this point, strong constraints force the new policy network to remain consistent with the initial policy network, preventing the actions from going astray.

[0072] Construction of the joint optimization loss function. The two parts described above are combined to form the final loss function. Instructions for updating network parameters:

[0073] (9); in, The performance metrics of the new strategy network, calculated using observation data from the second process, are based on reinforcement learning objectives for the second process, such as maximizing assembly success rate, and are calculated using temporal difference error. For distillation loss, To align weights, they can be adaptively set according to the scenario or manually set. This expresses the desired outcome. The formula allows the robotic arm to maintain its feel within the comfort zone of familiar processes, while actively venturing into the unknown areas of new processes.

[0074] (10); in, This represents the number of sample data points used in the initial policy network training within the small buffer. This indicates the first [number] participants in the initial policy network training within the small-capacity buffer. The state importance weight corresponding to each sample data point This indicates the first [number] participants in the initial policy network training within the small-capacity buffer. Each sample data corresponds to .

[0075] The initial strategy network update management includes value / strategy update frequency, sampling ratio, and evaluation metrics; during the deployment phase, canary releases are adopted, and metrics such as sliding window success rate or peak torque are used as rollback trigger thresholds.

[0076] In one embodiment, the relationship curves between various sizes of small-capacity buffers and the success rate of flexible assembly can be calculated through experimental simulation. A small-capacity buffer with a higher success rate and smaller size can be selected to balance storage costs and training performance. For example... Figure 6 As shown, Figure 6 This is a graph showing the relationship between the size of the small-capacity buffer and the average success rate of flexible assembly. Figure 6 The relationship between small capacity size and assembly performance is given to determine the buffer size and update rhythm. bs represents the size of the small capacity buffer (Buffer Size), that is, the number of samples stored in it. Figure 7 To set the impact curve of alignment weights on the average success rate of flexible assembly, the alignment weights are... This is used to control the constraint strength of new and old process strategies during alignment, thereby effectively preventing the forgetting of capabilities of old processes when learning new processes. Figure 7 The performance curves in multi-process scenarios are shown to illustrate the transfer and non-forgetting effects brought about by alignment and weighted fusion. Figure 8 This is a typical on-site operation sequence diagram for a flexible assembly machine. Figure 8 Figure (a) in the diagram is the operation sequence diagram of the hammering positioning process. Figure 8 Figure (b) is the operation sequence diagram of the linear push process. Figure 8 Figure (c) in the diagram is the operation sequence diagram for the process of pushing the object against the wall. Figure 8 Figure (d) in the diagram is the operation sequence diagram for the placement and shelving process. Figure 8 Present an example of online verification of a real machine, marking the location of the control interface, key points of the running results, and on-site constraints.

[0077] This manual addresses the requirement for new processes to achieve stable control within a short period of time. Relying solely on training with new data often results in the forgetting of old processes, manifesting as drift in gripping force, instability in trajectory fine-tuning, increased alignment deviation, decreased consistency across processes, and increased time for rework and parameter adjustment. This affects the efficiency and quality consistency of re-entry, making it difficult to guarantee production cycle time.

[0078] This manual establishes a dual-datastream joint training mechanism: one stream receives observation data from the current process, while the other stream receives data from a gated, small-capacity buffer. The output distribution of the control strategy is aligned on the small-capacity buffer. Uncertainty estimation is used to determine the alignment strength and state importance weights, ensuring the new strategy maintains the control style of the old process under critical operating conditions while continuously optimizing performance on new process data. Update rhythm management and interface adaptation are configured to ensure smooth integration with existing control chains. Monitoring-driven rollback and retraining triggers are provided to ensure timely suppression of performance fluctuations during migration, enabling rapid deployment and stable operation.

[0079] Compared to traditional methods that often rely on training for a single new task or behavioral cloning, which can easily lead to the degradation of capabilities of old processes or excessive constraints on the optimization of new processes, this invention implements strategy alignment and weighted fusion based on uncertainty. While maintaining the key capabilities of old processes, it releases the optimization space for new processes and ensures that the updates are feasible and maintainable through an engineered update rhythm and interface mechanism, taking into account both migration speed and stability.

[0080] This invention prioritizes usability and deployability by focusing on real-world flexible assembly processes. By building small-capacity, directly usable assembly experience, it significantly reduces storage and computing power consumption, shortens the closed-loop cycle from data acquisition to strategy updates, and can be rapidly deployed in production lines with frequent model changes. The experience building process retains key action characteristics such as picking, placing, alignment, pressing, and closing, providing a stable starting point during the introduction of new processes, reducing manual parameter adjustments and repeated trials, and improving the first-piece yield.

[0081] This invention employs a distributed consistency gating system on the data side, statistically verifying and constraining feasible regions for on-site factors such as metal reflection, lighting variations, and slight shading. Only samples conforming to the specifications are allowed into the experience playback library, preventing alignment deviations from propagating to training and deployment. On the control side, an uncertainty-weighted strategy alignment is introduced, allowing new strategies to maintain the key control style of existing processes while providing continuous optimization space. This suppresses forgetting without hindering performance improvements in new processes. The system provides standardized interfaces and update schedule management, seamlessly integrating with existing host computers and controllers, facilitating daily maintenance and version management, and reducing production line downtime and operational costs.

[0082] In one embodiment, a standardized data interface, runtime, and synchronization markers are also provided to ensure stable interface with the host computer and controller, and to support daily maintenance and version management. This application provides real-time status data, control commands, and synchronization markers through a standardized interface, ensuring that the server, host computer, and robotic arm controller exchange data under a unified time base, supporting version traceability and daily maintenance. The server, host computer, and controller exchange data and distribute policies through the standardized interface, retaining logs and parameters for auditing and maintenance.

[0083] In one embodiment, the size of the small-capacity buffer is configured according to the performance-capacity relationship curve to shorten the closed-loop cycle from data acquisition to policy update and improve transformation efficiency. On the experience building side, the size (bs) of the small-capacity buffer is not fixed, but rather depends on... Figure 6 The performance-capacity relationship curve shown is dynamically configured to compress the amount of data to the maximum extent while ensuring the assembly success rate, thereby shortening the closed-loop cycle of "acquisition-distillation-training-update".

[0084] On the process feature side, this application prioritizes extracting and retaining the feature distribution of core actions such as picking and placing, alignment, pressing, and closing, providing a logical starting point with tactile capabilities for the introduction of new processes and reducing manual parameter adjustments during the first piece debugging stage.

[0085] On the stability assurance side, this application introduces a real-time monitoring mechanism based on a sliding window. Once key indicators such as success rate or peak torque are detected to be lower than the preset threshold, the rollback logic will be automatically triggered to restore to the previous stable strategy version and start targeted retraining to ensure the quality consistency and online stability of the production site.

[0086] In one embodiment, such as Figure 9 As shown, Figure 9 A framework diagram for the distillation of experience from old processes and the learning of current process strategies in flexible assembly. Figure 9 The system is divided into a "Experience Distillation" module on the left and a "Policy Learning" module on the right. In the "Experience Distillation" module, the raw full dataset is first obtained from the "Historical Experience Base," and then a sample loss function is applied. Minimize as the optimization objective, and select representative state data. and corresponding actions These samples perform one interaction within the "previous process operation environment" to supplement the state at the next moment. ,award With termination mark This forms a complete experience tuple and stores it in a small buffer. In the "Policy Learning" module on the right, a dual-datastream joint training mechanism is employed: one stream receives real-time online data generated by the "current process operation environment" and stored in the "current process experience buffer," used to calculate the reinforcement learning loss reflecting the performance of the new process task. Another small buffer samples prior to the previous process to calculate the distillation loss based on policy alignment. , Through the analysis of and Joint optimization is carried out so that the new strategy network can adapt to the current process operation environment while retaining key skills from the historical experience base.

[0087] In one embodiment, this embodiment also proposes a process migration control method for flexible assembly task switching, which includes the following steps: S100 performs representative extraction (distillation candidate subset) and one-step interactive completion of existing process data within the assembly feasible domain to construct a small-capacity and directly usable complete set of experience tuples for changeover training and online startup.

[0088] Representative extraction employs feature space metrics to improve information density and deployability, while distillation of candidate subsets maintains small-scale high coverage through a comprehensive strategy of capacity capping, time decay, and quality control scoring.

[0089] S200 sets a distribution consistency gate before writing experience, calculates the distribution difference based on the characteristic statistics of candidate samples and on-site / original samples and forms a consistency score, and combines the feasible domain constraints of observation and action to allow only qualified samples to be included in the database.

[0090] In the new process training phase, S300 employs dual-data-stream joint training. One stream connects to the current process's online data, while the other stream connects to a small-capacity of old process experience that has been gated. The strategy output distribution is aligned on the old process data, and the alignment strength is weighted according to the teacher's uncertainty. This alignment is then weighted and fused with the reinforcement learning objective to obtain a new process control strategy that is not forgotten and can be deployed.

[0091] S400 features update rhythm management, gray-scale release and threshold rollback mechanisms. When the performance indicators do not meet the standards, it rolls back to the stable version and triggers retraining. When the standards are met, it is released.

[0092] In one or more embodiments of this specification, the executing entity may be a server, which may be a server set up on a business platform, or a device such as a desktop computer or laptop computer that can execute the scheme of this specification.

[0093] When applying the process migration control method for flexible assembly task switching provided in this manual, it is not necessary to consider... Figure 1 The steps shown are executed in sequence. The specific execution order of each step can be determined as needed, and this manual does not impose any restrictions on it.

[0094] The above describes a process migration control method for flexible assembly task switching, provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding process migration control device for flexible assembly task switching, where the flexible assembly task includes multiple processes; the device includes: The acquisition module is used to respond to the switching request from the first process to the second process and acquire the original full data under the first process; the original full data includes multiple sets of sample data obtained by interacting with the first process task using the initial policy network; each set of sample data includes the state data of the robotic arm and the corresponding action; The first screening module is used to obtain distillation candidate subsets from the original full data with the goal of minimizing the sample loss function. The sample loss function is constructed by the kernel ridge regression reconstruction error between the original full data and the distillation candidate subsets in the feature space of the initial policy network, and is used to measure the degree to which the distillation candidate subsets preserve the behavior distribution of the original full data. The second screening module is used to use the encoder of the initial policy network as a feature extractor to calculate the distribution consistency score between the distillation candidate subset and the original full data, and to calculate the out-of-bounds penalty value for each action in the distillation candidate subset based on the safe working space of the robotic arm; for any action in the distillation candidate subset, the distribution consistency score and the out-of-bounds penalty value of the action are weighted to obtain the comprehensive score of the action; and the sample data corresponding to the actions in the distillation candidate subset whose comprehensive score is less than or equal to the first gating threshold are stored in a small-capacity buffer. The training module is used to train the initial policy network based on the sample data in the small-capacity buffer and the observation data in the second process to obtain the new policy network. The joint optimization loss function used in the training process of the initial policy network is constructed based on the behavioral differences between the new policy network and the initial policy network, as well as the task performance of the new policy network.

[0095] Specific limitations regarding the process migration control device for flexible assembly task switching can be found in the limitations of the process migration control method for flexible assembly task switching described above, and will not be repeated here. Each module in the aforementioned process migration control device for flexible assembly task switching can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0096] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 A process migration control method for flexible assembly task switching is provided.

[0097] This instruction manual also provides Figure 10 The schematic diagram of the computer device shown is as follows: Figure 10 At the hardware level, the computer device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 A process migration control method for flexible assembly task switching is provided.

[0098] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0099] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A process migration control method for flexible assembly task switching, characterized in that, Flexible assembly tasks involve multiple processes; the method includes: In response to the switch request from the first process to the second process, the original full data of the first process is obtained; the original full data includes multiple sets of sample data obtained by interacting with the first process task using the initial policy network; each set of sample data includes the state data of the robotic arm and the corresponding action; The optimization objective is to minimize the sample loss function, and then obtain a distillation candidate subset from the original full data. The sample loss function is constructed by the kernel ridge regression reconstruction error between the original full data and the distillation candidate subset in the feature space of the initial policy network, and is used to measure the degree to which the distillation candidate subset preserves the behavior distribution of the original full data. The encoder of the initial policy network is used as a feature extractor to calculate the distribution consistency score between the distillation candidate subset and the original full data, and the out-of-bounds penalty value for each action in the distillation candidate subset is calculated according to the safe working space of the robotic arm. For any action in the distillation candidate subset, the distribution consistency score and the action's out-of-bounds penalty value are weighted to obtain the action's comprehensive score; the sample data corresponding to actions in the distillation candidate subset whose comprehensive score is less than or equal to the first gating threshold are all stored in a small-capacity buffer. The initial policy network is trained based on the sample data from the small buffer and the observation data from the second process to obtain the new policy network. The joint optimization loss function used in the training of the initial policy network is constructed based on the behavioral differences between the new policy network and the initial policy network, as well as the task performance of the new policy network.

2. The method according to claim 1, characterized in that, Sample loss function for: ； in, and These represent the state data and corresponding actions in the original full dataset, respectively. and These represent the state data and corresponding actions in the candidate distillation subset to be screened, respectively. and All are kernel matrices. express and Feature similarity between express Similarity to its own features For regularization terms, For adjustment coefficients, It is the identity matrix, and the superscript -1 indicates the inverse of the corresponding parameter.

3. The method according to claim 1, characterized in that, Using the encoder of the initial policy network as a feature extractor, a distribution consistency score between the distillation candidate subset and the original full dataset is calculated, including: The encoder of the initial policy network is used as a feature extractor to extract features of each state data in the distillation candidate subset and the original full data, respectively. Based on the characteristics of each state data in the candidate distillation subset and the original full dataset, the statistical characteristics of the state data in the candidate distillation subset and the original full dataset are calculated respectively; the statistical characteristics include mean and covariance. The distribution consistency score is calculated based on the mean and covariance of the state data in the candidate distillation subset and the original full dataset.

4. The method according to claim 3, characterized in that, Distribution Consistency Score The calculation formula is: ； in, and Let A and B represent the mean and covariance of the state data in the candidate distillation subset, respectively. and These represent the mean and covariance of the state data in the original full dataset, respectively. Indicates trace, Describing the L2 norm, This represents the square of the L2 norm.

5. The method according to claim 1, characterized in that, action Outbound penalty value The calculation formula is: ； in, Indicates action lower bound ， Indicates action The upper realm, Describing the L2 norm, Denotes the square of the L2 norm. Indicates the lower boundary penalty factor. This represents the out-of-bounds penalty factor.

6. The method according to claim 1, characterized in that, Joint optimization loss function for: ； ； ； in, The performance metrics of the new strategy network are calculated using the observation data from the second process. For distillation loss, To align weights, Expressing expectations, The weights representing the importance of states in relation to the uncertainty of the old strategy. Indicates trace, and This represents the initial policy network's prediction of actions based on sample data in a small buffer, including the mean. and variance ; and This indicates that the new policy network predicts actions based on sample data with a small buffer size, including the mean. and variance , express The reverse, express transpose, It is a positive number.

7. The method according to claim 1, characterized in that, The method further includes: For abnormal actions in the distillation candidate subset whose comprehensive score is greater than the first gate threshold, if the comprehensive score threshold of the abnormal action is greater than or equal to the second gate threshold, or the out-of-bounds penalty value of the abnormal action is greater than or equal to the penalty threshold, then the sample data corresponding to the abnormal action is discarded; the second gate threshold is greater than the first gate threshold. If the overall score threshold of the abnormal action is greater than the first gating threshold but less than the second gating threshold, and the out-of-bounds penalty value of the abnormal action is less than the penalty threshold, then the sample data corresponding to the abnormal action will be corrected, and the corrected sample data will be re-gated.

8. The method according to claim 7, characterized in that, Correct the sample data corresponding to this abnormal action, including: Adjust the camera's exposure parameters, re-collect the state data corresponding to the abnormal actions, and determine the newly collected state data and corresponding actions as the corrected sample data; Alternatively, the state data corresponding to the abnormal action can be augmented, and the augmented state data and the corresponding action can be used as the corrected sample data.

9. The method according to claim 1, characterized in that, The method further includes: When the small buffer is full, the sample data with the earliest timestamp is deleted first.

10. A process migration control device for flexible assembly task switching, characterized in that, The flexible assembly task includes multiple processes; the device includes: The acquisition module is used to respond to the switching request from the first process to the second process and acquire the original full data under the first process; the original full data includes multiple sets of sample data obtained by interacting with the first process task using the initial policy network; each set of sample data includes the state data of the robotic arm and the corresponding action; The first screening module is used to obtain a distillation candidate subset from the original full data with the goal of minimizing the sample loss function. The sample loss function is constructed by the kernel ridge regression reconstruction error between the original full data and the distillation candidate subset in the feature space of the initial policy network, and is used to measure the degree to which the distillation candidate subset preserves the behavior distribution of the original full data. The second screening module is used to use the encoder of the initial policy network as a feature extractor to calculate the distribution consistency score between the distillation candidate subset and the original full data, and to calculate the out-of-bounds penalty value for each action in the distillation candidate subset based on the safe working space of the robotic arm; for any action in the distillation candidate subset, the distribution consistency score and the out-of-bounds penalty value of the action are weighted to obtain the comprehensive score of the action; and the sample data corresponding to the actions in the distillation candidate subset whose comprehensive score is less than or equal to the first gating threshold are stored in a small-capacity buffer. The training module is used to train the initial policy network based on the sample data in the small-capacity buffer and the observation data in the second process to obtain the new policy network. The joint optimization loss function used in the training process of the initial policy network is constructed based on the behavioral differences between the new policy network and the initial policy network, as well as the task performance of the new policy network.

Images