A dynamic reconfiguration multi-machine collaborative control method supporting digital twins

Abstract

A dynamic reconstruction multi-machine collaborative control method supporting digital twins is characterized by: first, encapsulating the device's protocols and interfaces to construct a data acquisition and control module; second, establishing a dynamic reconstruction module consisting of four libraries; third, dynamically allocating tasks driven by quality, safety, and efficiency based on the data acquisition and control module and the dynamic reconstruction module; and fourth, before executing the assigned tasks, the system first performs online detection or offline correction of the position of the occupied position and the position of the processed parts to ensure the accuracy and consistency of task execution. By encapsulating the data acquisition and control module and establishing a dynamic reconstruction module, the present invention realizes dynamic reconstruction and registration of the equipment, performs offline programming and parsing tasks, and realizes dynamic and efficient allocation of processing tasks, ensuring improved processing accuracy and efficiency.

Description

Technical Field

[0001] This invention relates to a multi-machine collaborative control method that supports dynamic reconstruction of digital twins. In particular, it addresses the implementation of dynamic digital twin simulation collaborative control for diverse workshop scenarios, multiple processing devices, and non-unique workpieces. This method can be used to address the dynamic reconstruction of scenarios, devices, and workpieces in simulation or twin environments. Background Art

[0002] With the rapid development of manufacturing toward intelligent and digitalization, digital twin technology has become a key means of achieving intelligent manufacturing. By building virtual models of physical devices and systems, digital twin technology achieves a simultaneous mapping of the virtual and real worlds, enabling real-time monitoring, simulation, and optimization of various changes in the production process.

[0003] In traditional manufacturing systems, equipment typically operates in isolation, unable to achieve real-time collaborative work. This results in poor resource utilization and low production efficiency. The introduction of digital twin technology enables collaborative control between manufacturing equipment. Through virtual simulation and real-time data exchange, different devices can be rehearsed, optimized, and operated in virtual space, thereby improving overall production efficiency.

[0004] In modern manufacturing environments, production tasks often require the coordinated operation of multiple processing devices. Furthermore, the changing nature of different workshop scenarios, processing techniques, and workpiece types requires manufacturing systems to be highly flexible and adaptable. Traditional multi-machine collaborative control methods are typically based on static, pre-set models and are unable to cope with dynamically changing production demands and complex on-site environments. The limitations of this approach are primarily manifested in the following aspects:

[0005] First, the response speed is insufficient. Traditional control systems react slowly to changes in the production site and are unable to adjust the coordination strategy between equipment in a timely manner, resulting in low production efficiency.

[0006] Secondly, there is a lack of dynamic reconstruction capabilities. In the actual production process, equipment configuration, workpiece status and processing requirements may change frequently, and traditional control methods are difficult to adjust and optimize in real time.

[0007] Finally, the collaboration is highly complex. The collaborative work between multiple devices requires precise synchronization and efficient communication. Traditional methods are prone to improper coordination or waste of resources when dealing with complex scenarios.

[0008] Existing multi-machine collaborative control methods are typically based on pre-set static models and are difficult to adapt to the dynamic changes that frequently occur during the manufacturing process. For example, different shop floor scenarios, workpiece types, and changes in processing equipment configuration often require the system to be able to dynamically adjust and optimize during operation. Traditional methods often exhibit problems such as delayed response and lack of flexibility in these complex and changing production environments, making them unable to meet the needs of modern intelligent manufacturing.

[0009] Currently, most digital twin simulation and collaborative control systems on the market are typically developed for a single scenario. These systems are typically designed for fixed processing equipment and specific workpieces, with their structure and functionality determined at the time of development. Consequently, these systems often exhibit fixed and single-minded applicability, making them inflexible in responding to evolving production demands. For example, when equipment configuration or workpiece types change on a production floor, existing simulation systems often fail to adapt quickly, resulting in suboptimal simulation results.

[0010] Furthermore, these single-scenario digital twin simulation systems exhibit significant control inefficiencies and lags in response when faced with the coordinated operation of multiple devices and machines. Because the systems were not originally designed for multi-machine collaboration, their control algorithms and system architecture struggle to operate effectively in complex collaborative scenarios. Coordination between devices often experiences delays, making it difficult to ensure synchronization and continuity across the entire production process, impacting production efficiency and product quality.

[0011] These issues are particularly acute in complex manufacturing environments. In modern manufacturing environments, the increasing diversity and complexity of production tasks, coupled with the frequent changes in process flows, workpiece states, and equipment configurations, require simulation systems to possess greater flexibility and adaptability. However, existing digital twin simulation systems, due to their design limitations, struggle to cope with these dynamic changes and are unable to provide real-time, multi-scenario simulation support.

[0012] Based on the above problems, a new solution is urgently needed to improve the flexibility, adaptability and response speed of digital twin simulation collaborative control systems. Summary of the Invention

[0013] This paper addresses the shortcomings of traditional simulation systems in adaptability, flexibility, and responsiveness by proposing a dynamic reconstruction solution and multi-machine collaborative control method for virtual simulation or digital twin environments. This approach aims to address the current challenges faced in dynamic simulation display and multi-machine collaborative control across multiple scenarios, multiple devices, and multiple workpieces.

[0014] The technical solution of the present invention is:

[0015] A dynamic reconfiguration multi-machine collaborative control method supporting digital twins is characterized by:

[0016] First, the device protocols and interfaces are encapsulated to construct a data acquisition and control module. This encapsulation enables modular management of the underlying device structure, allowing the upper-level control system to flexibly and uniformly access these device resources. Through this standardized design, the integrated control system can dynamically register any number of sensor modules, forming a highly scalable and adaptable collaborative control system that supports dynamic reconfiguration in multiple scenarios and flexible scheduling of multiple devices.

[0017] Secondly, a dynamic reconstruction module is established, which includes four main libraries: workshop library (production line library), product library (for storing related products or parts), equipment library (including automation equipment and tooling tools), and personnel library. These libraries are designed to support the dynamic reconstruction of the system. Each library mainly stores digital models, parameter models and corresponding databases related to its corresponding objects. For example, the workshop library stores digital models and configuration parameters of different production lines, and the product library stores detailed model information of various products and parts. Through this classified storage and management, when performing dynamic registration, the system can quickly find the required model data and load it into the digital twin simulation or control environment to provide support for subsequent dynamic reconstruction.

[0018] Third, based on the aforementioned data acquisition and control module and dynamic reconstruction module, the system is able to achieve dynamic task allocation driven by quality, safety, and efficiency. Specifically, during the task planning phase, the system will first comprehensively plan and simulate the tasks through offline programming software and generate corresponding multi-station tasks. These tasks will then obtain the status information of each robot unit through the centralized control system, and then dynamically allocate the tasks to the most suitable robot unit for execution based on real-time data. This dynamic task allocation mechanism ensures that the workload of each device is optimized in a multi-device collaborative working environment. At the same time, when multiple devices are working simultaneously, a coarse / fine bounding box dynamic algorithm is used to improve overall production efficiency and quality control accuracy.

[0019] Fourth, before executing the assigned task, the system will first perform online detection or offline correction of the position of the placeholder and the position of the processed parts to ensure the accuracy and consistency of the task execution. If the system is offline, it will first perform offline simulation to simulate and verify every detail of the task. After the simulation is completed, the system will send the generated output results to the equipment for actual operation. If the system is online, it will accurately match the position of the processed parts through online detection, and then perform real-time online correction of the processing trajectory. During the processing, due to the large number of running devices, coarse and fine bounding boxes are added based on the device model. The coarse bounding box is used for range detection. When an object is detected within the range, the coarse bounding box is converted into a fine bounding box for anti-collision detection. If the fine bounding box detects an object, the current movement needs to be stopped and the state adjusted to ensure the consistency of operation safety, processing accuracy and product quality in the collaborative process of multiple devices.

[0020] It can be seen that the present invention can realize dynamic reconstruction and collaborative control of multiple scenes, multiple devices, and multiple workpieces, effectively overcome the shortcomings of existing systems, and meet the simulation needs in complex production environments.

[0021] The beneficial effects of the present invention are:

[0022] The present invention solves the challenges faced in the dynamic display of simulation of multiple scenes, multiple devices, and multiple workpieces and the collaborative control of multiple machines. In view of the shortcomings of traditional simulation systems in terms of adaptability, flexibility, and response speed, a dynamic reconstruction scheme and a multi-machine collaborative control method for virtual simulation or digital twin environments are proposed. This method can encapsulate the data acquisition control module, establish a dynamic reconstruction module, realize dynamic reconstruction and registration of equipment, offline programming and parsing tasks, realize dynamic and efficient allocation of processing tasks, and correct the trajectory before executing the task to ensure processing accuracy. This method has the following significant advantages:

[0023] First, this approach encapsulates device protocols and interfaces and modularizes various sensor types to create a highly flexible control system. This system can dynamically register and schedule different device modules, enabling it to quickly adapt to changes in production lines, including changes in equipment configuration, workpiece type, and production tasks. This flexibility significantly enhances the adaptability of the manufacturing system, enabling it to cope with complex and changing production environments.

[0024] Second, the proposed dynamic reconfiguration module, combined with the data acquisition and control module, enables the system to efficiently manage and schedule multiple devices. By collecting and analyzing real-time data, the system dynamically allocates tasks based on the status of each device, ensuring that each device can operate in optimal conditions. This not only improves the efficiency of multi-machine collaborative operations but also reduces resource waste and production delays caused by improper device scheduling.

[0025] Third, this method dynamically adjusts the machining trajectory before task execution through online detection and offline correction. Whether the system is online or offline, precise simulation and detection ensure machining accuracy. In particular, the online detection and matching function enables real-time correction of the machining trajectory, ensuring consistent and high-precision product quality and significantly reducing errors during the production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Overall technical solution for the reconstruction of the automatic grinding and spraying control system of the entire machine.

[0027] Figure 2 Architecture of dynamic reconfiguration method for multi-robot integrated control system.

[0028] Figure 3 Dynamic and efficient task allocation method for multi-machine collaboration.

[0029] Figure 4 Technical approaches to multi-robot collaboration and task allocation methods.

[0030] Figure 5 A holistic approach to dynamically reconfigurable multi-machine collaborative control. DETAILED DESCRIPTION

[0031] The present invention will be further described below with reference to the accompanying drawings and examples.

[0032] like Figure 1-5 shown.

[0033] A dynamic reconstruction multi-machine collaborative control method supporting digital twins. The overall process is as follows Figure 5 shown.

[0034] Below Figure 1 The method of the present invention is described using the aircraft complete machine grinding and spraying system as an example, and specifically comprises the following steps:

[0035] The first step is to encapsulate the device's protocols and interfaces and build a data acquisition and control module. These interfaces include data acquisition interfaces for multi-source sensors and processing equipment control interfaces. Correspondingly, the interface transmission protocols primarily encompass those for non-real-time information networks and real-time control networks. Defining the interfaces and protocols as standard unit modules enables the integrated control system to dynamically register any number of standard modules, implementing a multi-machine collaborative motion control architecture and supporting the construction, reorganization, expansion, and maintenance of complete grinding and spraying systems for tasks of varying sizes and complexity.

[0036] Take the grinding and spraying of the entire aircraft as an example:

[0037] (1.1) Define robots, AGVs, end effectors (for grinding and spraying), and various sensors as standard unit modules;

[0038] (1.2) Encapsulate protocols such as TCP / IP, OPC UA, EtherCAT, and IO-Link;

[0039] (1.3) During dynamic registration, a connection is quickly established with device sensors through encapsulated protocols to read or transmit data and complete reconstruction and simulation.

[0040] Step 2, such as Figure 2 As shown, a dynamic reconstruction module is established, including a workshop library F = {f0, f1, ... f n-1}(production line library), product library P={P0,P1,…P m-1} (store corresponding products or parts), equipment library D = {D0, D1, ... D q-1}(including automation equipment and tooling), personnel database S={S0,S1,…S j-1 The workshop library is selected during initialization or specified by the configuration file. The workshop library is the basis of the reconstructed control system and is the overall spatial description of a specific production line or workstation. Other products and equipment are placed within the scope of the workshop.

[0041] The product library stores parts, and a single production line (workshop) may support the polishing and painting of multiple products. The equipment library stores the equipment that may be used by each production line, including information such as digital models, kinematic pairs, and DH models. During dynamic registration, the system can quickly locate the required model data, load it into the simulation or control environment, and quickly complete device connection motion and information data communication through packaged interfaces and protocols. Pre-packaged model kinematic pairs and DH information enable real-time device motion display in a simulated environment.

[0042] For the personnel library, its various character models and their joint motion pairs are pre-packaged. Each character model contains a SQLite data table, which contains the position information of all joints of the character model. When the scene detects that a person has entered, the character model is loaded; through visual detection, the identified character joint information is written into the SQLite data table, and the character model completes the implementation of the person's simulated actions through the data table information.

[0043] Step 3: Based on the data acquisition control module and the dynamic reconstruction module, the processing tasks are dynamically and efficiently allocated. Figure 3 As shown, the overall global task R is imported into the offline programming system or the global task R is written through the offline programming system. The global task R is decomposed into R = {r0, r1, ... r l-1}, where r i For a certain process in the overall task, further process r i Divide the tasks, i ={T i,0 ,T i,1 ,…T i,j-1}where i is the process number, j is the number of tasks under the process, T i,0 Contains not only the task content, but also the task status

[0044]

[0045] Then the global task R can be expressed as:

[0046]

[0047] The task status Rs can be expressed as:

[0048]

[0049] The device status is represented by:

[0050] DS={Ds0,Ds1,…Ds q-1} (4)

[0051] in:

[0052]

[0053] like Figure 4 As shown, its task allocation is R (i,) →D,R (i,) Represents the task of the i-th step process; when the number of tasks is greater than |D|, task R i,j In its corresponding placeholder device D q When multiple places can be occupied for tasks, they are queued at the same time. The task status is read before the task is read. Only when the task Ts i,j = -1, the device performs the operation on the task; Rs (i,) =0 means all tasks of this process are completed and the next process R is carried out (i+1,) Task. Each AGV only accepts one processing task at a time, and its constraints are:

[0054] S i-1,q +P i-1,q i,q (6)

[0055] Among them, S i,q represents the starting processing time of the i-th task of the q-th AGV, P i,q ​represents the processing time of the i-th task of the q-th AGV; in order to avoid interference during processing that affects processing quality and safety, the AGVs at two adjacent stations cannot perform processing tasks at the same time, and the constraints are:

[0056] Ds i-1 +Ds i+1 <1 (7)

[0057] In multi-AGV batch task allocation, task waiting time refers to the time difference between task release and actual execution. Since each AGV may accept multiple tasks at a time, the length of the task queue directly affects the waiting time of tasks in the queue. The total task waiting time T is a key comprehensive indicator that directly affects the system's task execution efficiency and overall scheduling effect. Its calculation formula is as follows:

[0058]

[0059]

[0060]

[0061]

[0062] T is the total task waiting time; w is the task queue position number; For the kth task r k The task waiting time is calculated using the Hammond distance. As the starting point of the task, The end point of the mission, is the initial position of the device.

[0063] During task allocation, if some AGVs remain busy for extended periods while others remain idle, the workload will be extremely unevenly distributed across the AGVs. This situation not only reduces the overall efficiency of the system but can also lead to overuse and wear and tear on some AGVs. To avoid this problem, the load balancing indicator f2 is introduced. Its purpose is to maximize the balance of workload between each AGV during task allocation, ensuring a more balanced distribution of tasks across all AGVs and thus improving system stability and efficiency. Its calculation formula is as follows:

[0064]

[0065]

[0066]

[0067]

[0068] eunload With e load is the energy consumption of the equipment when it is idle and during the processing; the value of cv is between [0,1]. The closer its value is to 0, the more balanced the equipment workload is.

[0069] This dynamic task allocation mechanism ensures that the workload of each device is optimized in a multi-device collaborative working environment, thereby improving overall production efficiency and quality control accuracy.

[0070] Step 4: Perform trajectory detection and correction. The device obtains the assigned task and enters the corresponding station. Each measurement system detects the positioning posture of the station and determines the deviation between the actual position P′={p′1, p′2, p′3} and the theoretical position P={p1, p2, p3}. The system adopts online or offline compensation to correct the deviation:

[0071] (4.1) If the offline correction method is used, the offline programming software obtains the measured posture, corrects the position of the points of the offline program instructions, and chooses whether to perform offline simulation verification as needed, and then sends the corrected position task instructions to the device for execution.

[0072] (4.2) If the online correction method is adopted, the master control system of the unit equipment will make position corrections to the points of the offline task instructions based on the obtained measured posture, and then send the corrected position task instructions to the equipment for execution (the online correction method can achieve the greatest degree of automation).

[0073] After the device completes the execution of the current instruction, the system proceeds to the next step based on the completion status of the entire station task list. If all tasks are completed, the process task is completed. Otherwise, the device waits for the system to dynamically assign the next task.

[0074] In step 5, during multi-device machining, based on the device model, a fine bounding box is added, using the adjacent kinematic pairs of the corresponding device as nodes. The fine bounding box is generated based on the joints of the two adjacent kinematic pairs, with the starting joint as the center of motion and including the connecting objects between the two joints. A coarse bounding box is generated based on the device's working range and the size of the AGV, covering its working range and model size. During the motion process, a task tree approach is used to achieve a synchronized depth-first traversal of the two bounding box trees, which helps improve the algorithm's execution efficiency and ensure the safety of multi-machine collaborative work.

[0075] Take the polishing robot arm as an example:

[0076] (5.1) Get the joints of the robotic arm model i and range of motion;

[0077] (5.2) In the joint iand joint i+1 Add a rectangular bounding box between the connecting rods to complete the addition of the fine bounding box. Add the rectangular bounding box of the AGV where the robot arm is located and the circular bounding box formed by the working range to form a coarse bounding box.

[0078] (5.3) First, open the coarse bounding box and close the fine bounding box. During the movement, the task tree method is adopted. When the coarse bounding box detects that an object has entered, the coarse bounding box is closed and the fine bounding box is opened. When the fine bounding box detects that an object has entered, the task execution of the robot arm is stopped.

[0079] This patented method can also be applied to training scenarios for various types of equipment. By dynamically reconfiguring different equipment types and quantities, as well as participating personnel, a specific virtual training environment can be formed, thereby completing the corresponding training in a virtual simulation or digital twin system. Specifically, for example, in a lunar rover driving training scenario, if astronauts need to drive different types of lunar rovers, operate multiple astronauts, and coordinate with each other, this dynamic reconstruction collaborative control method can be used to achieve training in a digital twin environment.

[0080] The parts not involved in the present invention are the same as the existing technology or can be implemented by using the existing technology.

Claims

1. A dynamic reconfiguration multi-machine collaborative control method supporting digital twins, characterized by: First, encapsulate the device's protocol and interface to build a data acquisition control module; Secondly, a dynamic reconstruction module was established. This module includes four libraries: a workshop library (i.e., a production line library), a product library for storing related products or parts, an equipment library containing automation equipment and tooling, and a personnel library. Each library stores digital models, parametric models, and corresponding databases related to its corresponding objects. Third, based on the above-mentioned data acquisition control module and dynamic reconstruction module, dynamic allocation of tasks driven by quality, safety and efficiency is carried out; Fourth, before executing the assigned task, the system will first perform online detection or offline correction of the position of the occupied position and the position of the processed part; if the system is in an offline state, it will first perform offline simulation to simulate and verify every detail of the task; after the simulation is completed, the system will send the generated output results to the device for actual operation; if the system is in an online state, it will accurately match the position of the processed part through online detection, and then perform real-time online correction of the processing trajectory; based on the device model, coarse and fine bounding boxes are added. The coarse bounding box is used for range detection. When an object is detected within the range, the coarse bounding box is converted into a fine bounding box for anti-collision detection. If the fine bounding box detects an object, the current movement needs to be stopped and the state needs to be adjusted; When performing dynamic allocation, first import the overall global task R into the offline programming system or write the global task R through the offline programming system, and then decompose the global task R into R={r0,r1,…r l-1 }, where r i For a certain process in the overall task, further process r i Divide the tasks, i ={T i,0 ,T i,1 ,…T i,j-1 }where i is the process number, j is the number of tasks under the process, T i,0 It contains not only the task content, but also the task status: Then the global task R is expressed as: The task status Rs is expressed as: The device status is represented by: DS={Ds0,Ds1,…Ds q-1 } (4) in: The task is assigned to R (i,) →D,R (i,) Represents the task of the i-th step process; when the number of tasks is greater than |D|, task R i,j In its corresponding placeholder device D q When multiple places can execute tasks, they are queued at the same time. The task status is read before the task is read. Only when the task Ts i,j = -1, the device performs the operation on the task; Rs (i,) =0 means all tasks of this process are completed and the next process R is carried out (i+1,) Task: Each AGV only accepts one processing task at a time, and its constraints are: S i-1,q +P i-1,q i,q (6) Among them, S i,q represents the starting processing time of the i-th task of the q-th AGV, P i,q represents the processing time of the i-th task of the q-th AGV; in order to avoid interference during processing that affects processing quality and safety, the AGVs at two adjacent stations cannot perform processing tasks at the same time, and the constraints are:​ Ds i-1 +Ds i+1 <1 In multi-AGV batch task allocation, task waiting time refers to the time difference between task release and actual execution. Since each AGV may accept multiple tasks at a time, the length of the task queue directly affects the waiting time of tasks in the queue. The total task waiting time T is a key comprehensive indicator that directly affects the system's task execution efficiency and overall scheduling effect. Its calculation formula is as follows: T is the total task waiting time; w is the task queue position number; For the kth task r k The task waiting time is calculated using the Hammond distance. As the starting point of the task, The end point of the mission, is the initial position of the device; To this end, the load balancing index f2 is introduced. Its purpose is to maximize the balance of the workload of each AGV during task allocation, ensuring a more reasonable task allocation for each AGV, thereby improving the stability and efficiency of the system. Its calculation formula is as follows: e unload With e load is the energy consumption of the equipment when it is idle and during the processing; the value of f2 is between [0,1]. The closer its value is to 0, the more balanced the equipment workload is.

2. The method according to claim 1, wherein: The workshop library stores digital models and configuration parameters of different production lines, while the product library stores detailed model information of various products and parts.

3. The method according to claim 1, wherein: During dynamic allocation, during the task planning phase, the system will first conduct comprehensive planning and simulation of the task through offline programming software and generate corresponding multi-station tasks; These tasks will then obtain the status information of each robot unit through the centralized control system, and then dynamically assign the tasks to the most suitable robot unit for execution based on real-time data.

Images