Intelligent automatic loading technical method and management system

Abstract

The invention relates to the cross technical field of artificial intelligence and automatic control, and discloses an intelligent automatic loading technical method and a management system. According to the method, a high-fidelity digital twin model including vehicles, cargos and environments is constructed, and a loading space-time action sequence considering space utilization rate, loading balance and efficiency is generated in combination with a multi-objective optimization algorithm; a multi-agent collaborative architecture is adopted to decompose and execute an instruction, a tail end sensor is utilized to feed back state deviation in real time, and closed-loop re-planning is triggered. The system comprises a perception modeling unit, an optimization planning unit, a collaborative execution unit and a closed-loop correction unit, and is provided with a visual strategy configuration interface. According to the method, autonomous perception, intelligent decision making, accurate execution and dynamic optimization in the loading process are realized, and the operation efficiency, robustness and system configurability are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence and automation control, and specifically relates to an intelligent automatic loading technology method and management system. Background Technology

[0002] As my country's logistics and warehousing management modernization deepens, loading operations, as a crucial link connecting warehousing and transportation, directly impact the overall supply chain efficiency through their level of intelligence, collaboration, and precision. Even with the introduction of some automated equipment, traditional loading methods still heavily rely on human experience for task planning, vehicle verification, and resource scheduling, lacking dynamic perception and intelligent decision-making capabilities regarding order information, vehicle status, and warehousing resources. This model struggles to meet the demands of high-frequency, multi-category, and rapid-response modern logistics, resulting in low loading efficiency, high error rates, significant resource waste, and potential safety hazards.

[0003] Intelligent automated loading technology focuses on achieving closed-loop control of the entire process, from vehicle identification and spatial modeling to material scheduling and loading execution, through multi-source sensing, system integration, and real-time optimization. This technological direction aims to break down information barriers between warehouse management systems, transportation scheduling platforms, and automated loading equipment, building unified data standards and communication interfaces. Leveraging technologies such as 3D vision, artificial intelligence, and edge computing, it digitally models and dynamically adjusts the loading process, thereby improving the accuracy, safety, and flexibility of loading operations.

[0004] Existing technologies still face multiple bottlenecks in practical applications:

[0005] First, vehicle identification and parking status assessment rely heavily on manual visual inspection or simple image recognition, which cannot accurately match orders and vehicle parameters, making it difficult to generate optimal loading plans. Second, the heterogeneous protocols and data silos between the loading system and upstream and downstream information systems lead to delays in task instruction transmission and resource scheduling mismatches. Third, the loading process lacks real-time and comprehensive monitoring of cargo location, equipment operating status, and loading progress, resulting in delayed responses to abnormal events and a high risk of safety accidents or operational interruptions. Finally, the system's policy configuration is highly rigid, requiring professional intervention to adjust for new vehicle models, new cargo types, or temporary scheduling changes, leading to high on-site operational complexity and poor management usability.

[0006] The aforementioned problems are particularly prominent in modern warehousing and logistics centers with high throughput and diverse scenarios, and there is an urgent need for an integrated intelligent loading technology solution that deeply integrates perception, decision-making, execution and monitoring capabilities to solve them. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide an intelligent automatic loading technology method and management system, which aims to overcome the technical bottlenecks in the prior art, such as loading planning relying on human experience, information silos between heterogeneous systems, lack of process monitoring and real-time optimization capabilities, and poor system adaptability and configurability.

[0008] To address the aforementioned technical problems, this invention provides an intelligent automated loading technology method, comprising:

[0009] Acquire multi-source heterogeneous sensing data and construct a digital twin model of the loading operation scenario, including:

[0010] By deploying a structured light scanning array at the loading bay, a three-dimensional scan of the vehicle's outer contour and the interior space of the cargo compartment is performed on the vehicle to be loaded, and high-density point cloud data is obtained. The high-density point cloud data is processed by the Poisson surface reconstruction algorithm to generate a parametric three-dimensional mesh model of the vehicle to be loaded. This model accurately describes the internal length, width, height, irregular structure of the cargo compartment, and the door opening boundary.

[0011] By setting up online multi-dimensional sensor stations on the cargo conveyor line, each cargo unit to be loaded is continuously detected. The online multi-dimensional sensor station integrates a dynamic weighing module, a laser triangulation measurement module and a high-frequency radio frequency identification reader to obtain the precise weight, length, width and height dimensions and unique electronic identification code of the cargo unit.

[0012] Using a time-of-flight lidar fixedly installed in the loading area, the global environmental point cloud of the work area is periodically scanned and generated. The voxel grid algorithm is then used to spatially register and fuse the vehicle 3D mesh model, the geometric model of the cargo unit, and the environmental point cloud data to construct a real-time digital twin model containing all physical entities in a unified coordinate system.

[0013] The multi-source heterogeneous sensing data includes the three-dimensional structural data of the vehicle to be loaded, the physical attribute data of the cargo to be loaded, and the environmental point cloud data of the loading and unloading area.

[0014] Based on the aforementioned digital twin model, and combined with real-time loading order information, an optimized spatiotemporal sequence of loading actions is generated, including:

[0015] The loading planning problem is formalized as a multi-objective optimization problem. The set of optimization objective functions includes maximizing the utilization rate of the cargo compartment volume, minimizing the deviation between the vehicle load centroid and the geometric center, minimizing the total loading operation time, and minimizing the movement and stacking risks of highly fragile goods. A heuristic optimization algorithm combining genetic and simulated annealing is used to solve the multi-objective optimization problem.

[0016] The loading time-space action sequence defines the target three-dimensional coordinates and attitude of each cargo unit to be loaded when it is moved into the cargo compartment of the vehicle by the loading and unloading equipment at a predetermined time point.

[0017] The loading time-space action sequence is decomposed into a subset of execution instructions for each independent loading and unloading equipment unit, and then sent to the corresponding loading and unloading equipment unit through a multi-agent collaborative control network.

[0018] During the execution of the subset of execution instructions by the loading and unloading equipment unit, its physical execution status is monitored in real time, and the physical execution status is compared with the predetermined status in the digital twin model to generate a status deviation amount.

[0019] When the state deviation exceeds the preset deviation threshold, the digital twin model is updated based on the current physical execution state, and the loading time and space action sequence of the remaining goods to be loaded is regenerated based on the updated digital twin model.

[0020] Furthermore, after constructing the digital twin model, the process also includes attribute labeling of the physical entities in the model, specifically including:

[0021] The unique electronic identification code of the acquired cargo unit is linked with the cargo information database in the warehouse management system to mark the corresponding cargo geometric model in the digital twin model with its material code, stacking layer limit, fragility level and specified loading orientation and other constraint attributes.

[0022] Based on the parametric 3D mesh model of the vehicle, combined with a pre-set vehicle physical parameter library, the vehicle's unloaded center of gravity position, maximum load capacity, and allowable load range of the front and rear axles are calculated and labeled.

[0023] Furthermore, generating the optimal loading time-space action sequence also includes:

[0024] The chromosome encoding scheme of the genetic algorithm is the loading order of the cargo units to be loaded, and the fitness function is composed of the weighted sum of the optimization objective function. In each generation iteration of the genetic algorithm, a simulated annealing mechanism is introduced to perform local search on the offspring generated by the crossover and mutation operators to escape local optima.

[0025] The output of the genetic algorithm is an optimal cargo loading order. Then, for each cargo unit in this order, a three-dimensional packing heuristic algorithm based on the lowest horizontal plane filling strategy is used to determine the precise three-dimensional coordinates and orientation of the cargo unit in the virtual cargo space of the digital twin model, which is collision-free and satisfies all constraints.

[0026] The optimal loading sequence, placement coordinates, and orientation of all cargo units are combined with the capacity parameters of the loading and unloading equipment to finally compile a loading spatiotemporal action sequence in the form of a directed acyclic graph.

[0027] Furthermore, the loading time-space action sequence is decomposed into a subset of execution instructions for each independent loading and unloading equipment unit, and then distributed to the corresponding loading and unloading equipment unit through a multi-agent cooperative control network, including:

[0028] The physical devices in the loading system, including multi-degree-of-freedom loading robots, cargo conveyor lines, and vehicle fixing devices, are abstracted into independent software intelligent agents.

[0029] The directed acyclic graph of the loading time-space action sequence is published to a real-time message bus based on a publish-subscribe pattern;

[0030] Each software agent subscribes to the action nodes associated with it in the message bus; when the state of all the preceding dependent nodes of the action node becomes "completed", the agent parses the node information and converts it into low-level hardware control instructions that it can execute.

[0031] The intelligent agent of the multi-degree-of-freedom loading robot, after receiving motion node information including cargo unit identifier, starting pose and target pose, calls the inverse kinematics solver to calculate the joint angles corresponding to the target pose, and uses a fifth-order polynomial interpolation algorithm to plan a smooth and shock-free motion trajectory in the joint space, and finally generates a pulse sequence instruction to be sent to the servo driver.

[0032] Furthermore, its physical execution state is monitored in real time, and the physical execution state is compared with the predetermined state in the digital twin model to generate a state deviation, including:

[0033] A six-dimensional torque sensor and a miniature depth camera are integrated into the end effector of a multi-degree-of-freedom loading robot;

[0034] At the moment each cargo placement action is completed, the actual three-dimensional position and attitude of the placed cargo in the cargo compartment are obtained by a miniature depth camera and compared with the theoretical target pose defined in the loading time-space action sequence. The Euclidean distance and attitude quaternion error of the two in the three-dimensional Cartesian coordinate system are calculated, which together constitute the geometric deviation vector.

[0035] Meanwhile, the data from the six-dimensional torque sensor during placement is recorded and analyzed. By setting thresholds for contact force and torque, it is determined whether the placement process is stable and whether there are any unforeseen collisions or slippages.

[0036] The magnitude of the geometric deviation vector is compared with a preset geometric deviation threshold, and the torque sensor data analysis result is compared with a preset mechanical stability threshold. If either comparison result exceeds the threshold, it is determined that the state deviation exceeds the preset deviation threshold.

[0037] Furthermore, the geometric deviation threshold is set to a position error of 25 mm and an attitude error of 5 degrees; the contact force threshold is set to 50 Newtons and the torque threshold is set to 5 Newton-meters; if the force or torque in any direction exceeds the threshold for more than 100 milliseconds during placement, it is determined that there is an unforeseen collision or slippage.

[0038] Furthermore, the spatial resolution of the multi-source heterogeneous sensing data reaches the millimeter level, the point cloud data update cycle is no more than 500 milliseconds, the scanning frequency of the structured light scanning array is 20 frames per second, and the scanning frequency of the time-of-flight lidar is 10 Hz.

[0039] This invention also provides an intelligent automated loading technology management system, which uses the aforementioned intelligent automated loading technology method to achieve intelligent automated loading technology management, including:

[0040] The perception and 3D modeling unit is configured to acquire multi-source heterogeneous perception data of the vehicle to be loaded, the cargo to be loaded, and the working environment through a structured light scanning array, an online multi-dimensional sensing station, and a time-of-flight lidar. It also uses Poisson surface reconstruction algorithm and voxel grid fusion technology to construct a real-time digital twin model of the loading operation scenario.

[0041] The multi-objective optimization planning unit is configured to generate the optimal loading spatiotemporal action sequence in the form of a directed acyclic graph by running a heuristic optimization algorithm that combines genetic and simulated annealing with a three-dimensional bin packing heuristic algorithm, based on a digital twin model and loading order information.

[0042] The multi-agent collaborative execution unit is configured to abstract each physical device in the system into an independent software agent. Through a real-time message bus based on a publish-subscribe pattern, it parses and distributes the loading time-space action sequence into low-level hardware control instructions that can be executed by each agent, and calls inverse kinematics solving and trajectory planning algorithms to control the precise movement of the loading and unloading equipment.

[0043] The real-time status synchronization and closed-loop correction unit is configured to monitor the actual physical state of the goods in real time by integrating a six-dimensional torque sensor and a miniature depth camera at the end of the loading and unloading equipment. It compares the actual physical state of the goods with the predetermined state in the digital twin model to quantify the state deviation. When the deviation exceeds a preset threshold, it triggers a multi-objective optimization planning unit to replan the remaining loading tasks based on the updated real-time state.

[0044] Furthermore, the perception and 3D modeling unit is also configured to annotate the attributes of physical entities in the digital twin model, including linking the unique electronic identification code of the cargo unit with the cargo information database in the warehouse management system to annotate the material code, stacking layer limit, fragility level and specified loading orientation, and calculating and annotating the unloaded center of gravity position, maximum load capacity and allowable load range of the front and rear axles based on the parametric 3D mesh model of the vehicle.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] 1. By constructing a high-fidelity digital twin model, this invention completely and accurately maps the loading operation process in the physical world to the digital space, providing a globally unified data foundation for intelligent planning and decision-making, and solving the problem of information silos between heterogeneous systems.

[0047] 2. A hybrid heuristic optimization algorithm is adopted to perform multi-objective collaborative optimization on the loading planning problem. The generated loading scheme has achieved quantitative improvement in space utilization, vehicle balance and operation efficiency, and has eliminated the dependence on human experience.

[0048] 3. Based on a multi-agent collaborative control architecture, the system achieves automated instruction decomposition and distribution from high-level planning to low-level equipment execution. Combined with real-time sensing feedback from the end effector and a closed-loop replanning mechanism, the system has the ability to respond quickly to and autonomously correct abnormal situations on site, greatly enhancing the robustness and intelligence of the loading process.

[0049] 4. A visual strategy configuration unit is provided, which enables managers without a technical background to flexibly define new vehicle models and new goods through a graphical interface, and adjust and optimize strategies, significantly improving the system's adaptability and maintainability, and reducing long-term operating costs. Attached Figure Description

[0050] Figure 1 This is a schematic diagram of the overall technical solution architecture of an intelligent automatic loading technology method and management system proposed in this invention;

[0051] Figure 2 This is a schematic diagram of the core principle framework for generating loading spatiotemporal action sequences based on digital twins and multi-objective optimization in this invention.

[0052] Figure 3 This is a logical flowchart of the multi-source heterogeneous sensing data fusion and real-time digital twin model construction in this invention.

[0053] Figure 4 This is a logical flowchart of the decomposition of the spatiotemporal action sequence during vehicle loading and the multi-agent collaborative control execution in this invention.

[0054] Figure 5 This is a logical flowchart of the physical execution status monitoring, deviation comparison and closed-loop replanning mechanism in this invention;

[0055] Figure 6 This is a schematic diagram of the multi-level interaction relationship and data flow between the terminal operator and the system policy configuration unit in this invention. Detailed Implementation

[0056] Please refer to Figures 1 to 6 This invention provides an intelligent automated loading technology method and management system. Its core lies in constructing a high-fidelity digital twin model covering the vehicle, cargo, and operating environment; combining this with a multi-objective optimization algorithm to generate the optimal loading spatiotemporal action sequence; and relying on a multi-agent collaborative control architecture to achieve automated closed-loop control from global planning to equipment execution. During physical execution, the system collects status data in real time through end-point sensors and compares it with predetermined states in the digital twin model. Once a deviation exceeds a preset threshold, a replanning mechanism is immediately triggered, thereby ensuring that the entire loading process possesses a high degree of autonomy, robustness, and adaptability.

[0057] The intelligent automated loading technology method includes the following steps:

[0058] S1, acquire multi-source heterogeneous sensing data and construct a digital twin model of the loading operation scenario;

[0059] S2, Based on the digital twin model and combined with real-time loading order information, generate the optimal loading time and space action sequence;

[0060] S3, decompose the loading time-space action sequence into a subset of execution instructions for each independent loading and unloading equipment unit, and send them to the corresponding loading and unloading equipment unit through a multi-agent collaborative control network;

[0061] S4. During the execution of the subset of execution instructions by the loading and unloading equipment unit, its physical execution status is monitored in real time, and the physical execution status is compared with the predetermined status in the digital twin model to generate a status deviation amount.

[0062] S5, when the state deviation exceeds the preset deviation threshold, the digital twin model is updated based on the current physical execution state, and the loading time and space action sequence of the remaining goods to be loaded is regenerated based on the updated digital twin model.

[0063] In step S1, multi-source heterogeneous sensing data is acquired and a digital twin model of the loading operation scenario is constructed. Specifically, this involves using a structured light scanning array deployed at the loading bay to perform a 3D scan of the vehicle's overall outline and the interior space of the cargo compartment, acquiring high-density point cloud data. The structured light scanning array consists of multiple high-resolution projectors and synchronous industrial cameras, arranged alternately along the vertical and horizontal directions of the loading bay to ensure comprehensive, blind-spot-free coverage of the entering vehicle. The scanning frequency is set to 20 frames per second, with a single complete scan taking no more than 3 seconds. The acquired raw point cloud data contains no fewer than 5 million spatial coordinate points, achieving a spatial resolution at the millimeter level. Subsequently, the high-density point cloud data is processed using a Poisson surface reconstruction algorithm to generate a parametric 3D mesh model of the vehicle to be loaded. This model not only accurately describes the internal length, width, and height dimensions of the cargo compartment but also fully preserves the irregular structures present inside the cargo compartment, such as reinforcing ribs, fixing hooks, ventilation holes, and geometric features of the door opening boundaries. In the Poisson reconstruction process, the gradient field is constructed using an octree adaptive subdivision strategy, and the surface normal vector is estimated through the local neighborhood covariance matrix. The number of vertices in the final output mesh model is controlled to be less than 100,000 in order to balance accuracy and subsequent computational efficiency.

[0064] Simultaneously, each cargo unit awaiting loading is continuously monitored by an online multi-dimensional sensing station installed on the cargo conveyor line. The online multi-dimensional sensing station integrates a dynamic weighing module, a laser triangulation module, and a high-frequency RFID reader. The dynamic weighing module uses a high-precision strain gauge sensor with a sampling frequency of 1 kHz, enabling real-time acquisition of the precise weight of the cargo unit during transport, with an error range not exceeding ±50 grams. The laser triangulation module consists of two sets of through-beam laser profilometers, arranged laterally and longitudinally along the conveyor line respectively. It calculates the length, width, and height dimensions of the cargo unit using triangulation principles, with a measurement repeatability error of less than 0.5 mm. The high-frequency RFID reader operates in the 134 kHz band and can non-contactly read the electronic tags embedded in the bottom or side of the cargo unit as it passes, obtaining a unique electronic identification code. All sensor data is timestamped and transmitted in real-time to the central processing unit via industrial Ethernet.

[0065] In addition, a time-of-flight lidar fixedly installed on the top of the loading area periodically scans to generate a global environmental point cloud for the work area. The time-of-flight lidar has a scanning frequency of 10 Hz, a horizontal field of view of 360 degrees, and a vertical field of view of 40 degrees, and a single scan can cover the entire space within a radius of 30 meters. The acquired environmental point cloud is used to identify the presence of obstacles, personnel, or other mobile devices in the work area, and serves as a background reference for subsequent spatial registration. The central processing unit uses a voxel grid algorithm to spatially register and fuse the vehicle's 3D mesh model, the geometric model of the cargo unit, and the environmental point cloud data. The registration process first aligns the vehicle point cloud with the environmental point cloud using an iterative nearest-point algorithm, and then transforms the geometric model of the cargo unit to a unified world coordinate system based on its position information on the conveyor line. The final constructed digital twin model is a real-time updated 3D scene representation in a unified coordinate system that includes all physical entities, with an update cycle of no more than 500 milliseconds.

[0066] After constructing the digital twin model, the method further includes attribute labeling of the physical entities in the model. Specifically, the unique electronic identification code of the acquired cargo unit is linked to the cargo information database in the warehouse management system. The warehouse management system provides cargo information through a standard application programming interface, including constraint attributes such as material code, stacking layer limit, fragility level, and specified loading orientation. These attributes are directly bound to the metadata fields of the corresponding cargo geometry model in the digital twin model. For the vehicle model, based on its parametric 3D mesh model and combined with a preset vehicle physical parameter library, the vehicle's unloaded center of gravity position, maximum load capacity, and allowable load range of the front and rear axles are calculated and labeled. The vehicle physical parameter library stores standard parameters for common vehicle models; for new vehicle models, these parameters are manually entered or imported by the operator through a graphical interface.

[0067] In step S2, based on the digital twin model and combined with real-time loading order information, an optimized loading spatiotemporal action sequence is generated. The loading order information is pushed by the upstream order management system and includes business constraints such as a list of goods to be loaded, customer priority, and delivery time requirements. The system formalizes the loading planning problem into a multi-objective optimization problem, whose set of optimization objective functions includes maximizing cargo compartment volume utilization, minimizing the deviation between the vehicle's load centroid and geometric center, minimizing the total loading operation time, and minimizing the movement and stacking risks of highly fragile goods. The specific definitions of each objective function are as follows:

[0068] The objective function for volume utilization rate is defined as the ratio of the total volume of loaded goods to the usable volume of the cargo compartment, and its mathematical expression is:

[0069] ;

[0070] in, For the first The volume of a cargo unit This refers to the usable volume of the cargo compartment.

[0071] The load balancing objective function is defined as the weighted sum of the longitudinal and lateral offsets between the vehicle's overall centroid and the geometric center of the cargo box when the vehicle is fully loaded. Its mathematical expression is:

[0072] ;

[0073] in,( , ( ) represents the calculated coordinates of the vehicle's center of mass under full load. , () represents the coordinates of the geometric center of the cargo compartment. and These are the vertical and horizontal weighting coefficients.

[0074] The objective function for operation time is defined as the total time required to complete all loading tasks, and its calculation depends on the kinematic model and path planning results of the loading and unloading equipment.

[0075] The fragile risk objective function is defined as the sum of the weighted product of the number of handling operations and the stacking height experienced by all cargo units with a fragile level higher than the threshold during loading.

[0076] A heuristic optimization algorithm combining genetics and simulated annealing is used to solve the multi-objective optimization problem. The population size of the genetic algorithm is set to 200, and the chromosome encoding scheme is the loading order of the cargo units to be loaded, i.e., the length is... The algorithm uses an integer sequence, where each element represents a unique identifier for a cargo unit. The fitness function is a weighted sum of the four objective functions mentioned above, with the weight coefficients dynamically set by the visualization strategy configuration unit. In each iteration of the genetic algorithm, after performing selection, crossover, and mutation operations, a simulated annealing mechanism is introduced to perform local searches on some offspring individuals. The initial temperature of the simulated annealing is set to 100°C, and the cooling rate is set to 0.95. The local search employs a neighborhood exchange strategy, randomly exchanging the positions of two adjacent or non-adjacent genes in the chromosome each time, and evaluating the fitness of the new solution. If the new solution is better, it is accepted; otherwise, a suboptimal solution is accepted with a certain probability to avoid getting trapped in local optima. The maximum number of iterations in the algorithm is set to 500 generations, or it may terminate early if the fitness does not improve significantly for 50 consecutive generations.

[0077] The algorithm outputs an optimal cargo loading order. Subsequently, for each cargo unit in this order, a 3D packing heuristic algorithm based on a minimum horizontal plane filling strategy is employed to determine its precise 3D coordinates and orientation within the virtual cargo space of the digital twin model, ensuring collision-free placement and satisfying all constraints. The core idea of ​​the minimum horizontal plane filling strategy is to maintain a dynamically updated list of available space, selecting the current lowest horizontal plane as a candidate placement surface each time, and searching for a position on this surface that satisfies the cargo size and orientation constraints without interfering with other already placed cargo. The selection of the placement orientation must consider the specified loading orientation and stacking stability of the cargo, prioritizing orientations with flat bottoms and low centers of gravity. All placement decisions are verified through simulation in the digital twin model to ensure no geometric conflicts.

[0078] Finally, the optimal loading order, placement coordinates, and orientation of all cargo units are compiled into a loading spatiotemporal action sequence in the form of a directed acyclic graph (DAG). Each node in the graph represents an atomic action, such as "robot picks up cargo A," "robot moves to cargo compartment coordinates X, Y, Z," and "robot places cargo A." Directed edges between nodes represent the sequential dependencies between actions. Each node also includes the time window required to execute the action, equipment resource requirements, and expected physical state changes.

[0079] In step S3, the loading spatiotemporal action sequence is decomposed into a subset of execution instructions for each independent loading / unloading equipment unit, and then distributed to the corresponding loading / unloading equipment unit through a multi-agent collaborative control network. The system abstracts each physical device in the loading process, including the multi-degree-of-freedom loading robot, cargo conveyor line, and vehicle securing device, into an independent software agent. Each agent encapsulates the kinematic model, control interface, state machine, and communication logic of the corresponding device. The directed acyclic graph of the loading spatiotemporal action sequence is published to a real-time message bus based on a publish-subscribe pattern. This message bus employs a zero-copy memory sharing mechanism, with a message latency of less than one millisecond.

[0080] Each software agent subscribes to action nodes related to it in the message bus. For example, the loading robot agent subscribes to all nodes involving "grab," "move," and "place"; the conveyor line agent subscribes to nodes such as "start," "stop," and "position"; and the vehicle securing agent subscribes to nodes such as "clamp" and "release." When all the preceding dependent nodes of an action node become "completed," the agent parses the node information and converts it into low-level hardware control instructions that it can execute.

[0081] For a multi-degree-of-freedom loading robot agent, after receiving motion node information including the cargo unit identifier, initial pose, and target pose, it first queries the digital twin model for the precise geometric model and mass distribution of the cargo unit, which is used for subsequent trajectory planning and force control parameter setting. Then, it calls the inverse kinematics solver to calculate the joint angles corresponding to the target pose. The inverse kinematics solution uses a numerical iterative method, with initial guesses provided by the joint angles from the previous moment, and a convergence tolerance set to 0.01 degrees. After obtaining the initial and target joint angles, a smooth, shock-free motion trajectory is planned in the joint space using a 5th-order polynomial interpolation algorithm. The form of the 5th-order polynomial is:

[0082] ;

[0083] Among them, coefficient to The trajectory is uniquely determined by the initial and target positions, velocities, and acceleration boundary conditions. The time span of the trajectory is dynamically calculated based on the device's maximum speed and acceleration to ensure that joint movements do not exceed limits. Finally, the trajectory is discretized into a series of time-angle pairs and converted into pulse sequence commands sent to the servo driver.

[0084] In step S4, during the execution of the subset of execution instructions by the loading / unloading equipment unit, its physical execution state is monitored in real time, and the physical execution state is compared with a predetermined state in the digital twin model to generate a state deviation. The monitoring primarily relies on a six-dimensional torque sensor and a miniature depth camera integrated into the end effector of the multi-degree-of-freedom loading robot. The six-dimensional torque sensor has a sampling frequency of 1 kHz and can simultaneously measure force and torque in three directions. The miniature depth camera has a resolution of 640×480, a depth accuracy of 1 mm, and a frame rate of 30 Hz.

[0085] At the instant each cargo placement action is completed, the miniature depth camera triggers a snapshot to capture the actual 3D position and orientation of the placed cargo within the cargo compartment. The image processing module employs a point cloud registration algorithm to match the captured cargo point cloud with the theoretical model of the cargo in the digital twin model, calculating the Euclidean distance and orientation quaternion error between the two in a 3D Cartesian coordinate system. Position error vector The attitude error is calculated by using quaternion differences to obtain the rotation angle. The geometric deviation vector is defined as follows: .

[0086] Meanwhile, a six-dimensional torque sensor records the contact force and torque data throughout the placement process. The data analysis module sets the contact force threshold to 50 Newtons and the torque threshold to 5 Newton-meters. If the force or torque in any direction exceeds the threshold for more than 100 milliseconds during placement, an unforeseen collision or slippage is determined to have occurred.

[0087] The magnitude of the geometric deviation vector is compared with a preset geometric deviation threshold, which is set to a position error of 25 mm and an attitude error of 5 degrees. The torque sensor data analysis results are then compared with a preset mechanical stability threshold. If any comparison result exceeds the threshold, the state deviation is determined to exceed the preset deviation threshold.

[0088] In step S5, when the state deviation exceeds a preset deviation threshold, the system immediately pauses the current loading process and updates the digital twin model based on the current physical execution state. The update operation includes correcting the position and orientation of the actually placed goods to sensor measurements and marking their status as "confirmed." Subsequently, based on the updated digital twin model, the system extracts the remaining goods to be loaded list and re-executes the multi-objective optimization planning process in step S2 to generate a new spatiotemporal sequence of loading actions for the remaining goods. The replanning process considers the actual distribution of the loaded goods to ensure compatibility between the new plan and the existing layout. After the replanning is completed, the system resumes execution and continues to complete the remaining loading tasks.

[0089] The intelligent automated loading technology management system includes a perception and 3D modeling unit, a multi-objective optimization planning unit, a multi-agent collaborative execution unit, a real-time state synchronization and closed-loop correction unit, and a visualization strategy configuration unit. The perception and 3D modeling unit is configured to acquire multi-source heterogeneous perception data of the vehicles to be loaded, the goods to be loaded, and the operating environment through a structured light scanning array, an online multi-dimensional sensing station, and a time-of-flight lidar. It then uses a Poisson surface reconstruction algorithm and voxel grid fusion technology to construct a real-time digital twin model of the loading operation scenario.

[0090] The multi-objective optimization planning unit is configured to generate an optimal loading spatiotemporal action sequence represented in the form of a directed acyclic graph based on the digital twin model and loading order information, by running a heuristic optimization algorithm that combines genetic and simulated annealing, and in conjunction with a three-dimensional bin packing heuristic algorithm. The multi-agent cooperative execution unit is configured to abstract each physical device in the system into an independent software agent, and through a real-time message bus based on a publish-subscribe model, parse and distribute the loading spatiotemporal action sequence into low-level hardware control instructions executable by each agent, and invoke inverse kinematics solving and trajectory planning algorithms to control the precise movement of the loading and unloading equipment.

[0091] The real-time status synchronization and closed-loop correction unit is configured to monitor the actual physical state of the goods in real time using a six-dimensional torque sensor and a miniature depth camera integrated at the end of the loading and unloading equipment. It compares this physical state with a predetermined state in a digital twin model to quantify the state deviation. When the deviation exceeds a preset threshold, the multi-objective optimization planning unit is triggered to replan the remaining loading tasks based on the updated real-world state. The visualization strategy configuration unit provides a graphical user interface that allows operators to import or scan and generate new 3D vehicle models and annotate them with constraints; allows operators to define the physical and loading constraint attributes of new goods units; and allows operators to adjust the weight coefficients of each objective in the multi-objective optimization function to change the focus of the loading planning strategy to adapt to different business needs.

[0092] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An intelligent automated loading technology method, characterized in that, include: Acquire multi-source heterogeneous sensing data and construct a digital twin model of the loading operation scenario, including: By deploying a structured light scanning array at the loading bay, a three-dimensional scan of the vehicle's outer contour and the interior space of the cargo compartment is performed on the vehicle to be loaded, and high-density point cloud data is obtained. The high-density point cloud data is processed by the Poisson surface reconstruction algorithm to generate a parametric three-dimensional mesh model of the vehicle to be loaded. This model accurately describes the internal length, width, height, irregular structure of the cargo compartment, and the door opening boundary. By setting up online multi-dimensional sensor stations on the cargo conveyor line, each cargo unit to be loaded is continuously detected. The online multi-dimensional sensor station integrates a dynamic weighing module, a laser triangulation measurement module and a high-frequency radio frequency identification reader to obtain the precise weight, length, width and height dimensions and unique electronic identification code of the cargo unit. Using a time-of-flight lidar fixedly installed in the loading area, the global environmental point cloud of the work area is periodically scanned and generated. The voxel grid algorithm is then used to spatially register and fuse the vehicle 3D mesh model, the geometric model of the cargo unit, and the environmental point cloud data to construct a real-time digital twin model containing all physical entities in a unified coordinate system. The multi-source heterogeneous sensing data includes the three-dimensional structural data of the vehicle to be loaded, the physical attribute data of the cargo to be loaded, and the environmental point cloud data of the loading and unloading area. Based on the aforementioned digital twin model, and combined with real-time loading order information, an optimized spatiotemporal sequence of loading actions is generated, including: The loading planning problem is formalized as a multi-objective optimization problem. The set of optimization objective functions includes maximizing the utilization rate of the cargo compartment volume, minimizing the deviation between the vehicle load centroid and the geometric center, minimizing the total loading operation time, and minimizing the movement and stacking risks of highly fragile goods. A heuristic optimization algorithm combining genetic and simulated annealing is used to solve the multi-objective optimization problem. The loading time-space action sequence defines the target three-dimensional coordinates and attitude of each cargo unit to be loaded when it is moved into the cargo compartment of the vehicle by the loading and unloading equipment at a predetermined time point. The loading time-space action sequence is decomposed into a subset of execution instructions for each independent loading and unloading equipment unit, and then sent to the corresponding loading and unloading equipment unit through a multi-agent collaborative control network. During the execution of the subset of execution instructions by the loading and unloading equipment unit, its physical execution status is monitored in real time, and the physical execution status is compared with the predetermined status in the digital twin model to generate a status deviation amount. When the state deviation exceeds the preset deviation threshold, the digital twin model is updated based on the current physical execution state, and the loading time and space action sequence of the remaining goods to be loaded is regenerated based on the updated digital twin model.

2. The intelligent automatic loading technology method according to claim 1, characterized in that, After constructing the digital twin model, the process also includes attribute labeling of the physical entities in the model, specifically including: The unique electronic identification code of the acquired cargo unit is linked with the cargo information database in the warehouse management system to mark the corresponding cargo geometric model in the digital twin model with its material code, stacking layer limit, fragility level and specified loading orientation and other constraint attributes. Based on the parametric 3D mesh model of the vehicle, combined with a pre-set vehicle physical parameter library, the vehicle's unloaded center of gravity position, maximum load capacity, and allowable load range of the front and rear axles are calculated and labeled.

3. The intelligent automatic loading technology method according to claim 2, characterized in that, Generating the optimal loading time-space action sequence also includes: The chromosome encoding scheme of the genetic algorithm is the loading order of the cargo units to be loaded, and the fitness function is composed of the weighted sum of the optimization objective function. In each generation iteration of the genetic algorithm, a simulated annealing mechanism is introduced to perform local search on the offspring generated by the crossover and mutation operators to escape local optima. The output of the genetic algorithm is an optimal cargo loading order. Then, for each cargo unit in this order, a three-dimensional packing heuristic algorithm based on the lowest horizontal plane filling strategy is used to determine the precise three-dimensional coordinates and orientation of the cargo unit in the virtual cargo space of the digital twin model, which is collision-free and satisfies all constraints. The optimal loading sequence, placement coordinates, and orientation of all cargo units are combined with the capacity parameters of the loading and unloading equipment to finally compile a loading spatiotemporal action sequence in the form of a directed acyclic graph.

4. The intelligent automatic loading technology method according to claim 3, characterized in that, The loading time-space action sequence is decomposed into a subset of execution instructions for each independent loading and unloading equipment unit, and then distributed to the corresponding loading and unloading equipment unit through a multi-agent cooperative control network, including: The physical devices in the loading system, including multi-degree-of-freedom loading robots, cargo conveyor lines, and vehicle fixing devices, are abstracted into independent software intelligent agents. The directed acyclic graph of the loading time-space action sequence is published to a real-time message bus based on a publish-subscribe pattern; Each software agent subscribes to action nodes associated with it in the message bus; when the state of all the preceding dependent nodes of the action node becomes "completed", the agent parses the node information and converts it into low-level hardware control instructions that it can execute. The intelligent agent of the multi-degree-of-freedom loading robot, after receiving motion node information including cargo unit identifier, starting pose and target pose, calls the inverse kinematics solver to calculate the joint angles corresponding to the target pose, and uses a fifth-order polynomial interpolation algorithm to plan a smooth and shock-free motion trajectory in the joint space, and finally generates a pulse sequence instruction to be sent to the servo driver.

5. The intelligent automatic loading technology method according to claim 4, characterized in that, Real-time monitoring of its physical execution status, and comparison of the physical execution status with predetermined states in the digital twin model to generate state deviation quantities, including: A six-dimensional torque sensor and a miniature depth camera are integrated into the end effector of a multi-degree-of-freedom loading robot; At the moment each cargo placement action is completed, the actual three-dimensional position and attitude of the placed cargo in the cargo compartment are obtained by a miniature depth camera and compared with the theoretical target pose defined in the loading time-space action sequence. The Euclidean distance and attitude quaternion error of the two in the three-dimensional Cartesian coordinate system are calculated, which together constitute the geometric deviation vector. Meanwhile, the data of the six-dimensional torque sensor during the placement process is recorded and analyzed. By setting the thresholds for contact force and torque, it is determined whether the placement process is stable and whether there are any unforeseen collisions or slippages. The magnitude of the geometric deviation vector is compared with a preset geometric deviation threshold, and the torque sensor data analysis result is compared with a preset mechanical stability threshold. If either comparison result exceeds the threshold, it is determined that the state deviation exceeds the preset deviation threshold.

6. The intelligent automatic loading technology method according to claim 5, characterized in that, The geometric deviation threshold is set to a position error of 25 mm and an attitude error of 5 degrees; the contact force threshold is set to 50 N and the torque threshold is set to 5 Nm; if the force or torque in any direction exceeds the threshold for more than 100 milliseconds during placement, an unforeseen collision or slippage is determined to exist.

7. The intelligent automatic loading technology method according to claim 6, characterized in that, The spatial resolution of multi-source heterogeneous sensing data reaches the millimeter level, the point cloud data update cycle is no more than 500 milliseconds, the scanning frequency of the structured light scanning array is 20 frames per second, and the scanning frequency of the time-of-flight lidar is 10 Hz.

8. An intelligent automated loading technology management system, characterized in that, Intelligent automatic loading technology management is achieved using an intelligent automatic loading technology method according to any one of claims 1 to 7, including: The perception and 3D modeling unit is configured to acquire multi-source heterogeneous perception data of the vehicle to be loaded, the cargo to be loaded, and the working environment through a structured light scanning array, an online multi-dimensional sensing station, and a time-of-flight lidar. It also uses Poisson surface reconstruction algorithm and voxel grid fusion technology to construct a real-time digital twin model of the loading operation scenario. The multi-objective optimization planning unit is configured to generate the optimal loading spatiotemporal action sequence in the form of a directed acyclic graph by running a heuristic optimization algorithm that combines genetic and simulated annealing with a three-dimensional bin packing heuristic algorithm, based on a digital twin model and loading order information. The multi-agent collaborative execution unit is configured to abstract each physical device in the system into an independent software agent. Through a real-time message bus based on a publish-subscribe pattern, it parses and distributes the loading time-space action sequence into low-level hardware control instructions that can be executed by each agent, and calls inverse kinematics solving and trajectory planning algorithms to control the precise movement of the loading and unloading equipment. The real-time status synchronization and closed-loop correction unit is configured to monitor the actual physical state of the goods in real time by integrating a six-dimensional torque sensor and a miniature depth camera at the end of the loading and unloading equipment. It compares the actual physical state of the goods with the predetermined state in the digital twin model to quantify the state deviation. When the deviation exceeds a preset threshold, it triggers a multi-objective optimization planning unit to replan the remaining loading tasks based on the updated real-time state.

9. The intelligent automatic loading technology management system according to claim 8, characterized in that, The perception and 3D modeling unit is also configured to label the attributes of physical entities in the digital twin model, including linking the unique electronic identification code of the cargo unit with the cargo information database in the warehouse management system to label the material code, stacking layer limit, fragility level and specified loading orientation, and calculating and labeling the unloaded center of gravity position, maximum load capacity and allowable load range of the front and rear axles based on the parametric 3D mesh model of the vehicle.

Images