Digital twinning-based auxiliary material matching rehearsal method and system

Abstract

The invention relates to the field of auxiliary material distribution, in particular to an auxiliary material distribution rehearsal method and system based on digital twinning, and the method comprises the steps: collecting the real-time data of a physical environment through deploying a sensor network, and constructing a synchronous high-fidelity digital twinning body; generating a plurality of distribution schemes based on an improved genetic algorithm; parallel rehearsal schemes are carried out in the digital twinborn body, and evaluation is carried out through a multi-dimensional constraint rule base; adopting a multi-objective optimization algorithm to select an optimal scheme; compiling the optimal scheme into an equipment executable instruction; the physical state and the virtual state are compared in real time in the execution process, and closed-loop verification and dynamic correction are achieved. According to the method, virtual rehearsal and optimization of the distribution scheme are realized by constructing the digital twinborn bodies, physical conflicts and efficiency bottlenecks can be discovered and avoided in advance, and the problems that the prior art depends on artificial experience, conflicts cannot be predicted, information splitting, simulation and control disjunction and the like are effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of auxiliary material tray, in particular to an auxiliary material tray pre-rehearsal method and system based on digital twinning. BACKGROUND

[0002] In the auxiliary material tray operation in the field of discrete manufacturing, the current mainly relies on the personal experience of dispatchers for manual planning. This method has obvious technical limitations when facing the application scenarios of various types of materials, different sizes and shapes, and complex stacking constraints. Due to the lack of scientific and systematic planning method, it is difficult for manual to quickly develop a tray scheme that is optimal in space utilization, center of gravity stability and operation convenience. More seriously, the existing technical system lacks the ability to conduct comprehensive virtual verification before the actual execution of the scheme, and cannot predict possible physical conflicts such as unstable material stacking, motion interference between devices, and blocked transportation path. These problems are often exposed until the actual operation stage, resulting in operation interruption, rework adjustment and even safety accidents. Although some manufacturing enterprises try to introduce simulation technology to assist planning, the existing simulation system is mostly offline static simulation, which is seriously out of touch with the real-time changing physical environment. It cannot obtain real-time state data of the production line, nor can it convert the simulation results into executable control instructions for the device, forming a technical gap between virtual planning and physical execution. This technical status leads to the problems of low efficiency, insufficient reliability, high safety risk and other outstanding problems in the auxiliary material tray operation, which seriously restricts the improvement of the overall efficiency of intelligent manufacturing. SUMMARY

[0003] The purpose of the present application is to provide an auxiliary material tray pre-rehearsal method and system based on digital twinning, to solve the problems of unscientific planning, inability to predict physical conflicts, information fragmentation between systems, and simulation and real-time control disconnection caused by relying on manual experience in the existing auxiliary material tray process.

[0004] To achieve the above-mentioned purpose, the following technical solutions are adopted.

[0005] The auxiliary material tray pre-rehearsal method based on digital twinning comprises the following steps: S1, collecting real-time data of auxiliary material trays, automated guided vehicles and shelves through a sensor network deployed in the physical environment, constructing a high-fidelity digital twin body synchronized with the physical environment based on a lightweight incremental update algorithm, and obtaining a real-time virtual scene; S2, receiving a tray task from a production management system using the current state of the real-time virtual scene, generating a plurality of initial tray schemes coded with material picking sequence, placement position and transportation path based on an improved genetic algorithm; S3, preform the plurality of initial dispensing schemes in the digital twin in parallel, comprehensively evaluate each scheme by integrating a multi-dimensional constraint rule library of geometric constraints, physical constraints and business constraints, and obtain comprehensive evaluation data of each scheme; S4, select an optimal dispensing scheme from the plurality of initial dispensing schemes based on the comprehensive evaluation data by using a multi-objective optimization algorithm; S5, input the optimal dispensing scheme into an instruction compiler, and automatically parse and compile into a bottom layer control instruction sequence that can be directly executed by an execution device; S6, in the process of executing the bottom layer control instruction sequence, compare physical sensor feedback data with digital twin expected state in real time, and once a deviation beyond a tolerance is detected, trigger a dynamic re-planning program to generate an adjustment scheme, so as to realize closed-loop verification and deviation correction.

[0006] Optionally, the step S1 specifically comprises: Real-time position and attitude data of the auxiliary material tray, the automatic guided vehicle and the shelf are collected by deploying an ultra-wideband positioning tag and an inertial measurement unit sensor in the physical environment, multi-sensor fusion algorithm is used to fuse absolute position data provided by the ultra-wideband positioning tag and relative motion data provided by the inertial measurement unit sensor, measurement errors of individual sensors are corrected through data complementation, accurate three-dimensional coordinates and spatial orientation of each tracked entity in the virtual space are calculated, and high-precision real-time pose information is obtained; Meanwhile, identity data of objects are collected by deploying radio frequency identification devices at key nodes of the warehouse, the identity data are spatio-temporally associated and mapped with the real-time pose information, and accurate corresponding relationship between the virtual model and the physical entity is established; Based on the real-time pose information and the corresponding relationship, a lightweight three-dimensional engine incremental update algorithm is used to continuously monitor state changes of physical data, a state change recognition mechanism is set to detect differences between the physical data and the stored state of the virtual world, only the targeted virtual objects with state changes are updated in direction and pose, while the virtual objects with unchanged states remain unchanged, a high-fidelity digital twin highly consistent with the real environment is constructed, and a virtual scene accurately reflecting real-time state of the physical environment is obtained.

[0007] Optionally, the step S2 specifically comprises: The current state of the real-time virtual scene provided by the digital twin receives dispensing tasks from a manufacturing execution system or a warehouse management system, and obtains material type, quantity information in the task list and current available resource state data; The improved genetic algorithm generates multiple initial palletizing schemes by encoding the picking order of materials, the placement position and orientation on the pallet, and the transportation path of the automated guided vehicle as a chromosome data structure, using a diversity initialization strategy to generate an initial population, and introducing an adaptive genetic operator to dynamically adjust the crossover and mutation probabilities according to the evolution state of the population to balance the global search and local search capabilities. The improved genetic algorithm iteratively optimizes the initial population, selects excellent individuals according to the fitness evaluation results in each iteration process, and generates a new generation of population through the adaptive genetic operator, and finally outputs multiple initial palletizing schemes with differences.

[0008] Optionally, the step S3 comprises: Simultaneously pre-playing multiple initial palletizing schemes in the digital twin using a parallel computing architecture, and comprehensively evaluating each scheme through a multi-dimensional constraint rule library; Among them, the geometric constraint simulates the interference between materials and materials, materials and carriers, automated guided vehicles and the environment in the virtual environment through the integrated continuous collision detection algorithm, and detects and records the collision events in the scheme execution process in real time; The physical constraint calculates the dynamic center of gravity position of the pallet in real time in the virtual environment by defining the weight attribute of the material and setting the center of gravity offset threshold, and judges whether the center of gravity projection exceeds the stable area of the pallet, and identifies the placement scheme that may cause overturning; The business constraint evaluates the business rationality of the scheme by calculating the space utilization, operation time consumption and material correlation, wherein the space utilization is evaluated by calculating the ratio of the total projection area of the placed materials to the available area of the pallet, the operation time consumption is based on the pre-defined action time model to accumulate the total operation time of the scheme, and the material correlation is evaluated by logically binding the associated materials of the same production work order and calculating the proximity of the associated materials in the placement position; By comprehensively considering the geometric constraint, the physical constraint and the business constraint, the comprehensive evaluation data of each scheme in terms of geometric feasibility, physical stability and business rationality is obtained.

[0009] Optionally, the step S4 comprises: Based on the comprehensive evaluation data obtained in step S3, a multi-objective optimization algorithm is used to select the optimal palletizing scheme from the multiple initial palletizing schemes, wherein the multi-objective optimization algorithm includes a weighted scoring method and a Pareto optimal frontier analysis method, the weighted scoring method assigns different weight values to multiple evaluation dimensions such as space utilization, operation time consumption, balance and collision-free property, calculates the comprehensive score of each scheme by linear weighted summation, and sorts according to the score; The Pareto optimal frontier analysis method forms a Pareto optimal solution set containing multiple schemes by identifying non-dominated solutions between multiple optimization objectives without absolute superior-inferior relationship; the system selects the final optimal dispensing scheme from the ranking result obtained from the weighted scoring method or the Pareto optimal solution set according to the optimization preference set by the user.

[0010] Optionally, the step S5 comprises: inputting the optimal dispensing scheme selected in the step S4 into an instruction compiler, the instruction compiler being built-in with an extensible device instruction driving library; the instruction compiler first parses the high-level task description in the optimal dispensing scheme, decomposes high-level operations such as material handling and placement into basic operation units; then, according to the instruction templates predefined in the device instruction driving library, the basic operation units are compiled into low-level control instruction sequences executable directly by specific devices, wherein the control instruction of the automated guided vehicle includes a coordinate sequence of navigation path points, a motion speed parameter and an obstacle avoidance strategy, and the control instruction of the mechanical arm includes a grasping pose parameter, a joint motion trajectory and an end effector action instruction; the compiled low-level control instruction sequences are issued to the corresponding physical execution devices through an industrial network, and an instruction state tracking mechanism is established to provide accurate instruction execution benchmarks for subsequent closed-loop verification.

[0011] Optionally, the step S6 comprises: during the execution of the low-level control instruction sequences, the real-time state data of the automated guided vehicle, the mechanical arm and the auxiliary material tray are continuously collected through the physical sensor network, and these physical sensor feedback data are compared with the expected state of the virtual device in the digital twin in real time; when it is detected that there is a deviation between the actual position of the automated guided vehicle and the expected path exceeding the allowed range, or the device is stalled for more than a preset time threshold due to obstacles and other reasons, the system triggers a dynamic re-planning program; the dynamic re-planning program takes the actual state of the current physical world as a new initial condition, quickly re-enacts a local adjustment scheme or a global backup scheme in the digital twin, and inputs the adjustment scheme into the instruction compiler to compile a new control instruction sequence and issue it to the physical device.

[0012] The auxiliary material dispensing pre-enactment system based on digital twin comprises: A physical perception layer, comprising ultra-wideband positioning tags and inertial measurement unit sensors deployed on auxiliary material trays, automated guided vehicles and shelves, and industrial network devices deployed at key nodes in the workshop, for collecting real-time motion data of entities in the physical environment and transmitting them in real time through the industrial network; A digital twin construction module connected to the physical perception layer, receiving the real-time motion data, processing the real-time motion data to obtain accurate pose information using a multi-sensor fusion algorithm, and constructing a high-fidelity digital twin synchronized with the physical environment using a lightweight three-dimensional engine incremental update algorithm; A parallel pre-visualization and optimization engine module connected to the digital twin construction module, receiving the current state of the digital twin and the picking task from the production management system, generating multiple initial picking plans based on an improved genetic algorithm, pre-visualizing the plans in the digital twin in parallel and evaluating them through a multi-dimensional constraint rule base; An instruction compiling module connected to the parallel pre-visualization and optimization engine module, built-in an extensible device instruction driver library, automatically parsing and compiling the optimal picking plan into a sequence of low-level control instructions executable by the automated guided vehicle and the robotic arm; A closed-loop verification and dynamic deviation correction module connected to the instruction compiling module and the physical perception layer respectively, real-time comparing the physical sensor feedback data with the expected state of the digital twin, and triggering a dynamic re-planning procedure when detecting an out-of-limit deviation.

[0013] Optionally, the physical perception layer further includes radio frequency identification devices deployed at the warehouse entrances and exits, shelf areas, and picking stations, for collecting object identity data and associating them with pose information; The system adopts an edge-cloud collaborative computing architecture, in which the edge server deploys the digital twin construction module and the instruction compiling module, responsible for handling data synchronization and instruction compiling tasks with high real-time requirements, and the cloud server deploys the parallel pre-visualization and optimization engine module, responsible for executing computationally intensive plan generation and optimization tasks.

[0014] Optionally, the parallel pre-visualization and optimization engine module includes: A dimensional constraint rule base storing evaluation rules for geometric constraints, physical constraints, and business constraints, wherein the geometric constraints integrate a continuous collision detection algorithm, the physical constraints define a barycentric stability evaluation criterion, and the business constraints include space utilization, operation time consumption, and material association calculation logic; An improved genetic algorithm unit implementing a plan generation mechanism based on an adaptive genetic operator, representing the picking plan through chromosome encoding, and initializing the population using a diversity preservation strategy; A parallel computing unit utilizing a multi-core processor to concurrently execute simulation evaluations of multiple plans in the digital twin; A multi-objective optimization unit processing the evaluation results using a weighted scoring method and a Pareto optimal frontier analysis method, and outputting an optimal plan set; The closed-loop verification and dynamic deviation correction module includes: A data comparison unit receiving physical sensor feedback data in real-time and comparing them with the expected state of the digital twin within a tolerance; A multi-level response unit sequentially starting an audible and visual alarm, device suspension, and dynamic re-planning when detecting an out-of-limit deviation; The dynamic replanning unit regenerates the adjustment plan based on the current physical state and issues new instructions through the instruction compilation module.

[0015] Compared with the prior art, the present invention has the following beneficial effects: This application, by constructing a high-fidelity digital twin and establishing a complete virtual pre-simulation and closed-loop control process, achieves comprehensive pre-simulation and optimization of various palletizing schemes in a virtual environment. It can proactively identify and avoid potential physical conflicts and efficiency bottlenecks, effectively solving the problems of unscientific planning, unpredictable physical conflicts, information fragmentation between systems, and disconnect between simulation and real-time control caused by reliance on manual experience in existing palletizing processes. Multi-sensor data fusion and a lightweight incremental update algorithm ensure high synchronization between the digital twin and the physical environment, providing an accurate virtual environment for pre-simulation. An improved genetic algorithm generates diverse initial schemes, providing rich candidates for optimization. A multi-dimensional constraint rule base comprehensively evaluates the feasibility of schemes from geometric, physical, and business perspectives. A multi-objective optimization algorithm ensures that the final scheme achieves the best balance among multiple competing objectives. Automatic instruction compilation technology connects the key links from virtual decision-making to physical execution. Real-time closed-loop verification and dynamic correction mechanisms endow the system with adaptive capabilities to cope with environmental uncertainties. This application establishes a complete closed-loop control system from virtual space to physical space, significantly improving the overall efficiency, reliability, and safety of palletizing operations. Attached Figure Description

[0016] Figure 1 This is a schematic flowchart of an embodiment of the auxiliary material preparation and pre-simulation method based on digital twins according to the present invention.

[0017] Figure 2 This is a schematic diagram of a module of an embodiment of the auxiliary material batching and pre-simulation system based on digital twins according to the present invention. Detailed Implementation

[0018] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0019] The following detailed description is exemplary and intended to provide further detailed explanation of the invention. Unless otherwise specified, all technical terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention.

[0020] Example 1

[0021] like Figure 1 As shown in Example 1, a method for pre-simulating auxiliary material batching based on digital twins. This embodiment is applied to auxiliary material traying operations in discrete manufacturing scenarios, such as the selection, matching, and transportation of auxiliary materials in electronic assembly or automobile manufacturing workshops. It utilizes digital twin technology to achieve virtual pre-simulation of the traying scheme, multi-objective optimization, and closed-loop control of physical execution. The specific implementation process is as follows: First, data acquisition of the physical environment and the construction of a high-fidelity digital twin are carried out. Within the physical space involved in the palletizing operation, sensing devices are deployed for all entities requiring tracking: ultra-wideband (UWB) positioning tags and inertial measurement unit (IMU) sensors are installed on pallets of auxiliary materials awaiting palletizing, automated guided vehicles (AGVs) used for transportation, forklifts assisting in handling, and shelves storing auxiliary materials. The UWB positioning tags are used to acquire the absolute position information of the entities, while the IMU sensors are used to collect the motion attitude data of the entities, including acceleration and angular velocity, to achieve real-time perception of the entity's position and attitude. Simultaneously, industrial network equipment is deployed at key nodes in the workshop (such as warehouse entrances and exits, next to the palletizing station, and turns in the AGV's travel path). 5G CPEs or industrial Wi-Fi 6 base stations can be selected to build a low-latency, highly reliable industrial communication network, ensuring that the massive amounts of sensor data collected by UWB and IMU can be transmitted back to the data processing unit in real time.

[0022] The returned sensor data is fused and analyzed to establish an accurate correspondence between the virtual model and the physical entity. A multi-sensor fusion algorithm (such as Kalman filtering) is used to fuse the absolute position data provided by UWB positioning tags with the relative motion data provided by IMU sensors. This data complementarity corrects for measurement errors from individual sensors—for example, when UWB signals drift due to obstruction, the relative displacement data from the IMU is used for compensation; when the IMU accumulates errors due to prolonged operation, it is calibrated using the absolute position data from UWB. Ultimately, the precise three-dimensional coordinates and spatial orientation of each tracked entity in the virtual space are calculated. If RFID devices are deployed in the workshop (e.g., RFID readers are placed at warehouse entrances / exits, shelf areas, and palletizing stations, and RFID tags are affixed to pallets and material boxes), the object identification data collected by RFID (including material type, batch, attributes, etc.) is spatiotemporally mapped with the calculated real-time pose information to ensure that each model in the virtual scene accurately corresponds to a specific entity in the physical world.

[0023] Based on the processed data, a high-fidelity digital twin synchronized with the physical environment is constructed using a lightweight 3D engine incremental update algorithm. First, based on the actual layout of the workshop (e.g., the location and dimensions of walls, columns, shelves, aisles, and workstations), a virtual scene containing all static elements is constructed using a real-time 3D engine. 3D models with dimensions and appearances consistent with the physical entities are created for AGVs, pallets, and various auxiliary materials. During virtual-real synchronization, instead of fully refreshing the entire 3D scene every frame, the currently received physical data is continuously compared with the previous state stored in the virtual world. Only changed target objects and their attributes (e.g., entity position coordinates, rotation angles, and working status indicators) are identified and transmitted. Oriented pose updates are performed on these changed virtual objects, while the original data remains unchanged for virtual objects whose states have not changed. In this way, the synchronization delay between the physical world and the digital twin is controlled within a preset range, ensuring a high degree of consistency between the real-time virtual scene and the real physical environment.

[0024] Upon receiving a palletizing task from a production management system (such as a Manufacturing Execution System (MES) or a Warehouse Management System (WMS), multiple initial palletizing schemes are generated based on the current state of the real-time virtual scene provided by the digital twin. The palletizing task includes the type and quantity of the materials to be palletized, the corresponding production order information, and the current status of available resources (such as the idle status of AGVs, the working status of robotic arms, and the storage location of materials on the shelves). An improved genetic algorithm is used to generate the initial schemes: each palletizing scheme is encoded as a "chromosome" data structure, which includes the picking order of materials (the order in which materials are picked from the shelves), the placement and orientation of materials on the pallet (such as the specific placement point within the X / Y coordinate area of ​​the pallet and the rotation angle of the materials), and the AGV's transportation path (the route from the shelf to the palletizing station). When generating the initial population, a diversity initialization strategy is adopted to avoid excessive concentration of initial schemes. An adaptive genetic operator is introduced to dynamically adjust the crossover probability and mutation probability according to the evolutionary state of the population. When the population diversity is low, the mutation probability is increased to increase the scheme differentiation. When the population tends to converge, the mutation probability is reduced to stabilize the excellent scheme. By iteratively optimizing and balancing the global search and local search capabilities, dozens or even hundreds of initial disk matching schemes with differences are finally output.

[0025] Multiple initial disk configuration schemes are simulated in parallel within a digital twin, and a comprehensive evaluation is performed using a multi-dimensional constraint rule base. Utilizing multi-core computing resources deployed in the workshop, all initial schemes are rapidly simulated simultaneously in a virtual scenario. During the simulation, geometric, physical, and business constraints are rigorously applied to evaluate each scheme in real time. The implementation of geometric constraints involves: establishing accurate 3D models and simplified collision bodies for each material and carrier (pallet, AGV); integrating a continuous collision detection (CCD) algorithm to simulate motion interference between materials, between materials and carriers, and between AGVs and the workshop environment (such as walls, columns, and other equipment) during the execution of the scheme; detecting and recording collision events in real time; and marking the scheme as non-compliant with geometric constraints if a collision occurs. The implementation of physical constraints involves: predefining the weight attributes of each material; setting a center of gravity offset threshold for the pallet; calculating the overall dynamic center of gravity of the pallet after it carries materials in real time during the simulation; and determining whether the projection of the center of gravity on the horizontal plane exceeds the stable area of ​​the pallet (such as the area formed by the inward contraction of the pallet edge); if it does, marking the placement scheme as potentially causing overturning. The implementation of business constraints includes three aspects: first, calculating space utilization by evaluating the utilization efficiency of pallet space through the ratio of the total projected area of ​​placed materials to the usable area of ​​the pallet; second, statistically analyzing operation time based on predefined action time models (such as the standard time for manually picking up an auxiliary material, the action time for a robotic arm to grasp materials, and the time for AGVs to...). The evaluation criteria include: 1) the time taken to travel a unit distance; 2) the total operation time from material picking to placement for each solution; and 3) the assessment of material correlation, which involves logically binding strongly related materials from the same production order (such as screws and washers required to assemble a component) and calculating the proximity of these related materials on the pallet. The lower the proximity, the lower the material correlation score for the solution. By combining the evaluation results of the above three types of constraints, a comprehensive evaluation of each initial palletizing solution in terms of geometric feasibility, physical stability, and business rationality is obtained.

[0026] Based on comprehensive evaluation data, a multi-objective optimization algorithm is used to select the optimal pallet arrangement scheme from multiple initial schemes. The multi-objective optimization algorithm includes a weighted scoring method and a Pareto optimal front analysis method: In the weighted scoring method, weights are assigned to different evaluation dimensions according to the actual needs of the workshop (such as space utilization, operation time, physical balance, collision-free operation, etc.), and the comprehensive score of each scheme is calculated by linear weighted summation, and the schemes are ranked from highest to lowest score; In the Pareto optimal front analysis method, "non-dominated solutions" are identified where there is no absolute superiority or inferiority relationship among multiple optimization objectives—that is, no other scheme is superior to this one in all objectives. These non-dominated solutions form the Pareto optimal solution set. The system determines the final optimal pallet arrangement scheme from the weighted scoring ranking results or the Pareto optimal solution set based on the user-defined optimization preferences (such as prioritizing the scheme with the shortest operation time during peak production seasons, and prioritizing the scheme with the best physical balance when materials are prone to tipping over).

[0027] The optimal disk configuration is input into the instruction compiler, which automatically parses and compiles it into a sequence of low-level control instructions that can be directly executed by the device. The instruction compiler has a built-in extensible device instruction driver library, which includes the instruction formats and communication protocols of mainstream AGV, robotic arms, and automated equipment brands. The compilation process consists of two steps: First, the high-level task descriptions in the optimal palletizing scheme (such as "moving a certain type of auxiliary material from a designated shelf to a pallet at a specific palletizing station" or "placing the auxiliary material at a designated position on the pallet") are parsed and decomposed into a series of basic operation units (such as "the AGV moves from its current position to the target shelf", "the robotic arm grabs the auxiliary material from the shelf", "the AGV carries the auxiliary material to the palletizing station", or "the robotic arm places the auxiliary material at a designated position on the pallet"). Second, based on the predefined instruction templates in the device instruction driver library, each basic operation unit is converted into the corresponding low-level control instructions for the physical device. For example, for the AGV, navigation path point coordinate sequences, driving speed parameters, and obstacle avoidance trigger conditions are generated; for the robotic arm, grasping pose parameters (such as the three-dimensional coordinates and orientation of the end effector), joint motion trajectories (rotation angle and speed of each joint), and end effector action instructions (such as grasping force and opening / closing time) are generated. The compiled low-level control instruction sequence is sent to the corresponding execution device through the industrial network, and an instruction status tracking mechanism is established to record the issuance time, expected execution duration and execution status of each instruction, providing a benchmark for subsequent closed-loop verification.

[0028] During the execution of the underlying control command sequence, closed-loop verification and dynamic correction are performed in real time. The sensor network of the physical perception layer continuously collects real-time status data of the AGV, robotic arm, and auxiliary material tray (such as the actual position of the AGV, the joint angle of the robotic arm, and the change of the center of gravity of the tray), and compares these physical sensor feedback data with the expected state of the virtual device in the digital twin (the motion trajectory and state changes of the virtual device generated based on the issued commands) in real time. When a deviation exceeding the preset tolerance is detected (e.g., the deviation between the actual position of the AGV and the expected path exceeds a set threshold, or the equipment stagnates for more than a preset time due to dynamic obstacles), the system immediately activates a multi-level response mechanism: First, it triggers an audible and visual alert to notify on-site personnel; if the deviation persists or shows an increasing trend, it automatically suspends the operation of the relevant equipment to prevent collisions or overturning accidents; simultaneously, it initiates a dynamic replanning program—this program uses the current physical world's actual state (e.g., the current position of the AGV, the real-time position of obstacles, and the remaining quantity of materials) as new initial conditions, and quickly re-enacts local adjustment schemes (e.g., adjusting the AGV's travel path to bypass obstacles) or global backup schemes in the digital twin, inputs the generated adjustment scheme into the instruction compiler to compile it into a new sequence of control instructions, and sends it to the physical equipment to correct the execution deviation, ensuring that the palletizing task is ultimately completed smoothly.

[0029] Example 2

[0030] like Figure 2 As shown, this embodiment provides a digital twin-based auxiliary material batching pre-simulation system to implement the batching pre-simulation method in Embodiment 1 above. This system is deployed in a discrete manufacturing workshop and works collaboratively with the workshop's existing production management system (MES / WMS). Specifically, it includes a physical sensing layer, a digital twin construction module, a parallel pre-simulation and optimization engine module, an instruction compilation module, and a closed-loop verification and dynamic correction module. The composition and function of each module are as follows: The physical sensing layer is the foundation for the system's interaction with the physical world, responsible for collecting real-time data on entities related to palletizing operations and transmitting it to upper-layer modules. Its components include sensing devices and industrial network equipment: Sensing devices include UWB positioning tags and IMU sensors installed on auxiliary material pallets, AGVs, forklifts, and shelves, as well as RFID readers deployed at warehouse entrances / exits, shelf areas, and palletizing workstations. UWB positioning tags and IMU sensors work together to achieve real-time perception of entity positions and orientations. RFID readers are used to collect identification data (including material attributes, batch information, etc.) from pallets and material boxes. Some workshops may also add industrial cameras as needed to assist in verifying the placement of materials and the operational status of equipment. Industrial network equipment includes 5G CPEs or industrial Wi-Fi 6 base stations deployed at key nodes in the workshop, building a low-latency, highly reliable wireless communication network to ensure that the massive amounts of data collected by sensing devices (such as UWB location data, IMU orientation data, and RFID identification data) can be transmitted in real-time and stably to subsequent data processing modules, avoiding impacts on system real-time performance due to data transmission delays or losses.

[0031] The digital twin construction module connects to the physical sensing layer. Its core function is to map physical entities and states into a virtual scene, generating a high-fidelity digital twin synchronized with the physical environment. This module includes a data fusion unit and a lightweight incremental update unit: The data fusion unit receives raw sensor data transmitted from the physical sensing layer and uses multi-sensor fusion algorithms (such as Kalman filtering) to fuse the absolute position data of the UWB and the relative motion data of the IMU. By using data complementarity to correct the measurement errors of a single sensor, it calculates the precise three-dimensional coordinates and spatial orientation of each tracked entity in the virtual space. At the same time, this unit performs spatiotemporal correlation mapping between the identification data collected by RFID and the calculated real-time pose information to ensure that the virtual model corresponds one-to-one with the physical entity, avoiding identity mismatch or pose deviation. The lightweight incremental update unit, based on the real-time pose information output by the data fusion unit and the correspondence between the entities, uses a 3D engine to construct a virtual scene. First, it creates 3D models of static elements (walls, columns, shelves, aisles) according to the actual layout of the workshop, and creates dynamic models for AGVs, pallets, and auxiliary materials that are consistent with the physical entities. During the virtual-real synchronization process, this unit does not use a full scene refresh method, but continuously compares the current physical data with the historical state of the virtual world. It only identifies and updates target objects whose states have changed and their attributes (such as position coordinates, rotation angle, and working status), while keeping the data unchanged for objects whose states have not changed. This reduces the computational and communication burden, controls the virtual-real synchronization delay within a preset range, and ensures that the virtual scene can accurately reflect the real-time state of the physical environment.

[0032] The parallel pre-simulation and optimization engine module is the core of the system's intelligent decision-making, connecting the digital twin construction module and the production management system. It is responsible for generating initial pallet allocation schemes, conducting parallel pre-simulation evaluations, and selecting the optimal solution. This module includes a multi-dimensional constraint rule library, an improved genetic algorithm unit, a parallel computing unit, and a multi-objective optimization unit. The multi-dimensional constraint rule library stores evaluation rules for geometric, physical, and business constraints. The geometric constraint rules integrate a continuous collision detection (CCD) algorithm, defining collision judgment criteria between materials, between materials and carriers, and between equipment and the environment. The physical constraint rules define the weight attribute data of materials and the center-of-gravity offset threshold of the pallet, clarifying the judgment conditions for the center-of-gravity projection exceeding the stable area. The business constraint rules include the calculation logic for space utilization (the ratio of the projected area of ​​placed materials to the usable area of ​​the pallet), the statistical logic for operation time consumption (a total time accumulation method based on the action time model), and the evaluation logic for material correlation (the calculation method for the placement proximity of related materials in the same work order). An improved genetic algorithm unit is used to generate initial palletizing schemes: the palletizing scheme is encoded as a "chromosome" (containing picking order, placement position, and transportation path), an initial population is generated using a diversity initialization strategy, and the population is iteratively optimized using adaptive genetic operators (dynamically adjusting crossover and mutation probabilities) to output multiple differentiated initial schemes. A parallel computing unit utilizes multi-core processor resources to simultaneously perform rapid simulations of multiple initial schemes in a virtual scene generated by the digital twin construction module. During the simulation, a multi-dimensional constraint rule library is invoked to evaluate each scheme in real time, recording the evaluation results of geometric feasibility, physical stability, and business rationality. A multi-objective optimization unit receives the evaluation data from the parallel computing unit, processes the data using a weighted scoring method (assigning weights to evaluation dimensions and calculating a comprehensive score) and Pareto optimal front analysis (identifying non-dominated solutions to form an optimal solution set), and outputs a set of optimal palletizing schemes based on user optimization preferences for the user to choose the final scheme.

[0033] The instruction compilation module connects to the parallel pre-simulation and optimization engine module, responsible for converting the optimal palletizing scheme into low-level instructions executable by physical devices. The core of this module is a built-in extensible device instruction driver library, which stores instruction formats, communication protocols, and control parameters for mainstream AGVs, robotic arms, and automated equipment. The library's content can be expanded based on new equipment types added to the workshop. Upon receiving the finalized optimal palletizing scheme, the instruction compilation module first parses the high-level task description in the scheme, decomposing it into a series of independent basic operation units (such as AGV movement, robotic arm grasping, and material placement). Then, based on predefined templates in the device instruction driver library, it compiles each basic operation unit into the corresponding low-level control instructions for the device—for example, generating navigation point coordinates, travel speed, and obstacle avoidance parameters for the AGV, and grasping pose, joint motion trajectory, and end effector action instructions for the robotic arm. After compilation, the low-level instruction sequence is sent to the corresponding physical execution devices via the industrial network, and an instruction status tracking mechanism is established to record the instruction issuance time, expected execution status, and progress, providing benchmark data for closed-loop verification.

[0034] The closed-loop verification and dynamic correction modules are connected to the instruction compilation module and the physical perception layer, respectively, to achieve closed-loop control of physical execution and virtual pre-simulation, ensuring stable execution of the palletizing task. This module includes a data comparison unit, a multi-level response unit, and a dynamic replanning unit: The data comparison unit receives real-time equipment and material status data (such as the actual position of the AGV, the joint angle of the robotic arm, and the center of gravity of the pallet) transmitted from the physical perception layer, and simultaneously obtains the expected state of the instructions issued by the instruction compilation module (such as the expected path of the AGV and the expected trajectory of the robotic arm). It compares the two in real time, calculates the deviation value, and compares it with preset tolerances (such as position deviation thresholds and dwell time thresholds). The multi-level response unit triggers corresponding responses based on the data comparison results: if the deviation is within the tolerance range, the equipment continues to operate normally; if the deviation exceeds the tolerance range, an audible and visual warning is first activated to alert on-site personnel; if the deviation continues or shows an increasing trend, a pause command is automatically sent to the relevant equipment to prevent accidents. The dynamic replanning unit is activated when the equipment is paused or the deviation exceeds the limit. It uses the current physical world state (such as the current position of the equipment, the position of obstacles, and the amount of remaining material) as the initial conditions, calls the real-time virtual scene of the digital twin construction module, and quickly re-previews to generate local adjustment plans (such as adjusting the AGV path to bypass obstacles) or global backup plans. The new plan is then compiled into new low-level instructions by the input instruction compilation module and sent to the physical equipment to correct the deviation, ensuring that the palletizing task can continue to be executed until it is completed.

[0035] Furthermore, the system adopts an edge-cloud collaborative computing architecture to balance real-time performance and computing power requirements: edge servers are deployed near the workshop, running digital twin building modules and instruction compilation modules, responsible for handling tasks with high real-time requirements (such as virtual-real synchronization and instruction compilation), reducing data transmission latency; cloud servers or local central servers run parallel pre-simulation and optimization engine modules, responsible for handling computationally intensive tasks (such as generating a large number of initial schemes, parallel pre-simulation evaluation, and multi-objective optimization), utilizing the powerful computing resources of the cloud or central servers to improve the efficiency of scheme generation and optimization, ensuring that the system can still quickly output the optimal scheme when facing complex disk configuration tasks.

[0036] As is known from common technical knowledge, this invention can be implemented through other embodiments that do not depart from its spirit or essential characteristics. Therefore, the disclosed embodiments described above are merely illustrative in all respects and are not the only ones. All modifications within the scope of this invention or its equivalents are included in this invention.

Claims

1. A digital-twin-based over-the-counter dispensing plate pre-performance method, characterized in that, The method comprises the following steps: S1, collecting real-time data of the auxiliary material tray, the automatic guided vehicle and the shelf through a sensor network deployed in the physical environment, constructing a high-fidelity digital twin synchronized with the physical environment based on a lightweight incremental update algorithm, and obtaining a real-time virtual scene; S2, receiving a dispensing task from a production management system using the current state of the real-time virtual scene, and generating a plurality of initial dispensing schemes encoded with material picking sequences, placement positions and transportation paths based on an improved genetic algorithm; S3, pre-playing the plurality of initial dispensing schemes in the digital twin, and comprehensively evaluating each scheme through a multi-dimensional constraint rule library integrating geometric constraints, physical constraints and business constraints to obtain comprehensive evaluation data of each scheme; S4, selecting an optimal dispensing scheme from the plurality of initial dispensing schemes based on the comprehensive evaluation data using a multi-objective optimization algorithm; S5, inputting the optimal dispensing scheme into an instruction compiler, and automatically parsing and compiling into a bottom-layer control instruction sequence executable directly by an execution device; S6, comparing physical sensor feedback data with digital twin expected state in real time during execution of the bottom-layer control instruction sequence, and triggering a dynamic re-planning program to generate an adjustment scheme once a deviation beyond a tolerance is detected, to realize closed-loop verification and correction.

2. The digital-twin-based in-process inventory tray walkthrough method of claim 1, wherein, The step S1 specifically comprises: Collecting real-time position and attitude data of the auxiliary material tray, the automatic guided vehicle and the shelf through ultra-wideband positioning tags and inertial measurement unit sensors deployed in the physical environment, fusing absolute position data provided by the ultra-wideband positioning tags with relative motion data provided by the inertial measurement unit sensors using a multi-sensor fusion algorithm, correcting measurement errors of individual sensors through data complementation, calculating accurate three-dimensional coordinates and spatial orientations of each tracked entity in the virtual space, and obtaining high-precision real-time pose information; Meanwhile, collecting identity data of objects through radio frequency identification devices deployed at key nodes of the warehouse, and performing spatio-temporal correlation mapping of the identity data and the real-time pose information to establish accurate correspondence between virtual models and physical entities; Based on the real-time pose information and the correspondence, continuously monitoring state changes of physical data using a lightweight three-dimensional engine incremental update algorithm, detecting differences between physical data and stored states of the virtual world through a state change recognition mechanism, only updating directed poses of target virtual objects with recognized state changes, and keeping virtual objects with unchanged states unchanged, to construct a high-fidelity digital twin highly consistent with the real environment, and obtain a virtual scene accurately reflecting real-time states of the physical environment.

3. The digital-twin-based over-the-counter dispensing plate pre-performance method according to claim 1, characterized in that, The step S2 specifically comprises: Receiving a dispensing task from a manufacturing execution system or a warehouse management system using the current state of the real-time virtual scene provided by the digital twin, and obtaining material type, quantity information and current available resource state data in the task list; The improved genetic algorithm generates multiple initial palletizing schemes by encoding the picking sequence of materials, the placement position and orientation on the pallet, and the transportation path of the automated guided vehicle as a chromosome data structure, using a diversity initialization strategy to generate an initial population, and introducing an adaptive genetic operator to dynamically adjust the crossover and mutation probabilities according to the evolution state of the population to balance the global search and local search capabilities. The improved genetic algorithm is used to iteratively optimize the initial population, select excellent individuals according to the fitness evaluation results in each iteration process, and generate a new generation of population through the adaptive genetic operator, and finally output multiple initial palletizing schemes with differences.

4. The digital-twin-based over-the-counter dispensing plate pre-performance method according to claim 1, characterized in that, The step S3 comprises: Simultaneously pre-play multiple initial palletizing schemes in the digital twin using a parallel computing architecture, and comprehensively evaluate each scheme through a multi-dimensional constraint rule library; Among them, the geometric constraint simulates the interference between materials, materials and carriers, automated guided vehicles and the environment in the virtual environment by integrating a continuous collision detection algorithm, and detects and records collision events in real time during the execution of the scheme; The physical constraint calculates the dynamic center of gravity position of the pallet in real time in the virtual environment by defining the weight attribute of the material and setting the center of gravity offset threshold, and judges whether the center of gravity projection exceeds the stable area of the pallet, and identifies the placement scheme that may cause overturning; The business constraint evaluates the business rationality of the scheme by calculating the space utilization, operation time consumption and material correlation, wherein the space utilization is evaluated by calculating the ratio of the total projection area of the placed materials to the available area of the pallet, the operation time consumption is based on the pre-defined action time model to accumulate the total operation time of the scheme, and the material correlation is evaluated by logically binding the associated materials of the same production work order and calculating the proximity of the associated materials in the placement position; By comprehensively considering the geometric constraint, the physical constraint and the business constraint, the comprehensive evaluation data of each scheme in terms of geometric feasibility, physical stability and business rationality is obtained.

5. The digital-twin-based over-the-counter dispensing plate pre-performance method according to claim 1, characterized in that, The step S4 comprises: Based on the comprehensive evaluation data obtained in step S3, an optimal palletizing scheme is selected from the multiple initial palletizing schemes using a multi-objective optimization algorithm, wherein the multi-objective optimization algorithm includes a weighted scoring method and a Pareto optimal frontier analysis method, the weighted scoring method assigns different weight values to multiple evaluation dimensions such as space utilization, operation time consumption, balance and collision-free property, calculates the comprehensive score of each scheme by linear weighted summation, and sorts according to the score; The Pareto optimal frontier analysis method forms a Pareto optimal solution set containing multiple schemes by identifying non-dominated solutions that do not have an absolute superior-inferior relationship between multiple optimization objectives; the system selects the final optimal palletizing scheme from the ranking result obtained by the weighted scoring method or the Pareto optimal solution set according to the user's set optimization preference.

6. The digital-twin-based over-the-counter dispensing plate pre-performance method according to claim 1, characterized in that, The step S5 comprises: inputting the optimal dispensing scheme selected in step S4 into an instruction compiler, the instruction compiler being built-in with an extensible device instruction driving library; the instruction compiler first parses the high-level task description in the optimal dispensing scheme, decomposes the high-level operations such as material handling and placing into basic operation units; then, according to the pre-defined instruction templates in the device instruction driving library, the basic operation units are compiled into low-level control instruction sequences executable by specific devices, wherein the control instruction of the automated guided vehicle comprises a coordinate sequence of navigation path points, a motion speed parameter and an obstacle avoidance strategy, and the control instruction of the mechanical arm comprises a grasping pose parameter, a joint motion trajectory and an end effector action instruction; the compiled low-level control instruction sequences are issued to the corresponding physical execution devices through an industrial network, and an instruction state tracking mechanism is established to provide accurate instruction execution benchmarks for subsequent closed-loop verification.

7. The digital-twin-based over-the-counter dispensing plate rehearsal method according to claim 1, characterized by, The step S6 comprises: during the execution of the low-level control instruction sequences, the real-time state data of the automated guided vehicle, the mechanical arm and the auxiliary material tray are continuously collected through the physical sensor network, and the physical sensor feedback data are compared with the expected state of the virtual device in the digital twin in real time; when it is detected that there is a deviation between the actual position of the automated guided vehicle and the expected path, which exceeds the allowed range, or the device is stalled due to an obstacle for more than a preset time threshold, the system triggers a dynamic re-planning program; the dynamic re-planning program takes the actual state of the current physical world as a new initial condition, quickly re-plays in the digital twin to generate a local adjustment scheme or a global backup scheme, and inputs the adjustment scheme into the instruction compiler to compile new control instruction sequences and issue them to the physical devices.

8. A digital-twin-based pre-presentation system for excipients, for implementing the digital-twin-based pre-presentation method for excipients according to any one of claims 1 to 7, characterized in that, Comprise: a physical perception layer comprising ultra-wideband positioning tags and inertial measurement unit sensors deployed on auxiliary material trays, automated guided vehicles and shelves, and industrial network devices deployed at key nodes in the workshop, for collecting real-time motion data of entities in the physical environment and transmitting them in real time through an industrial network; a digital twin construction module connected to the physical perception layer, receiving the real-time motion data, processing the real-time motion data to obtain accurate pose information using a multi-sensor fusion algorithm, and constructing a high-fidelity digital twin synchronized with the physical environment using a lightweight three-dimensional engine incremental update algorithm; a parallel pre-play and optimization engine module connected to the digital twin construction module, receiving dispensing tasks from a production management system and the current state of the digital twin, generating multiple initial dispensing schemes based on an improved genetic algorithm, pre-playing the schemes in the digital twin in parallel and evaluating them through a multi-dimensional constraint rule library; an instruction compilation module connected to the parallel pre-play and optimization engine module, built-in with an extensible device instruction driving library, automatically parsing and compiling the optimal dispensing scheme into low-level control instruction sequences executable by automated guided vehicles and mechanical arms; a closed-loop verification and dynamic correction module connected to the instruction compilation module and the physical perception layer respectively, comparing physical sensor feedback data with expected states of the digital twin in real time, and triggering a dynamic re-planning program when an out-of-limit deviation is detected.

9. The digital twin-based in-process inventory tray walk-through system of claim 8, wherein, The physical perception layer further comprises radio frequency identification devices deployed at warehouse entrances and exits, shelf areas, and tray stations, for collecting identity data of objects and associating the data with pose information; The system adopts an edge-cloud collaborative computing architecture, wherein the edge server deploys the digital twin construction module and the instruction compiling module, and is responsible for processing data synchronization and instruction compiling tasks with high real-time requirements; the cloud server deploys the parallel pre-play and optimization engine module, and is responsible for performing computationally intensive scheme generation and optimization tasks.

10. The digital twin-based in-process inventory tray walk-through system of claim 8, wherein, The parallel pre-play and optimization engine module comprises: a dimension constraint rule library that stores evaluation rules of geometric constraints, physical constraints, and business constraints, wherein the geometric constraints integrate a continuous collision detection algorithm, the physical constraints define a barycenter stability evaluation criterion, and the business constraints include space utilization, operation time consumption, and material association calculation logic; an improved genetic algorithm unit that implements a scheme generation mechanism based on an adaptive genetic operator, encodes a tray scheme by using a chromosome, and initializes a population by using a diversity preservation strategy; a parallel computing unit that uses a multi-core processor to concurrently execute simulation evaluation of multiple schemes in the digital twin; a multi-objective optimization unit that uses a weighted scoring method and a Pareto optimal front analysis method to process evaluation results and output an optimal scheme set; The closed-loop verification and dynamic correction module comprises: a data comparison unit that receives physical sensor feedback data in real time and compares the data with a tolerance range of an expected state of the digital twin; a multi-level response unit that sequentially starts an audible and visual alarm, device suspension, and dynamic re-planning when a deviation is detected to be out of limits; a dynamic re-planning unit that generates an adjustment scheme again based on a current physical state and issues new instructions through the instruction compiling module.

Images