Multi-camera vision stacking pose planning method and system for intelligent loading and unloading robot

Abstract

The invention relates to the field of industrial automation, in particular to a multi-camera vision stacking pose planning method and system for an intelligent loading and unloading robot. The method comprises the following steps: constructing a collaborative sensing system through a plurality of industrial cameras deployed in a target space, and collecting dynamic environment data of a stacking area in real time; generating a self-adaptive stacking pose based on a hierarchical pose planning mechanism: analyzing an initial pose parameter through a theme control layer and generating a preliminary placement instruction, and verifying the feasibility of the preliminary placement instruction based on environmental data of the collaborative awareness system through a visual verification layer, and a final placement coordinate is optimized by combining the accurate positioning layer with a virtual box matching algorithm. The problems that in the prior art, real-time environment sensing capacity is lacked, pose planning is disjointed with the actual environment, and dynamic change scenes cannot be processed are solved.

Description

Technical Field

[0001] The present invention relates to the field of industrial automation, and in particular to a multi-camera visual stacking posture planning method and system for an intelligent loading and unloading robot. Background Art

[0002] In the field of industrial automation, robotic box packing systems have been widely used, particularly in logistics, warehousing, and manufacturing. However, existing robotic box packing systems still face numerous challenges in practical applications. First, these systems generally lack real-time environmental perception capabilities and are unable to accurately detect dynamic changes in the stacking area. For example, during multi-layer stacking, the robot may place goods in unstable or unsafe locations due to its inability to monitor subtle surface deformations or obstacles in real time, resulting in the goods tipping over or being damaged. Second, existing pose planning systems are disconnected from the actual environment and are unable to adjust their plans based on real-time environmental data. This makes it difficult for the robot to generate reasonable placement poses in complex stacking scenarios, which in turn affects packing efficiency and quality. Furthermore, existing systems are unable to handle dynamically changing stacking scenarios, making it impossible for the robot to adjust its movements in a timely manner once the environment changes. Finally, existing systems lack a mechanism to verify the rationality of the planned poses, failing to ensure that the generated poses are safe and feasible in actual operation. Therefore, a multi-camera vision-based pose planning method and system for intelligent loading and unloading robots is urgently needed. Summary of the Invention

[0003] The purpose of the present invention is to provide a multi-camera visual stacking posture planning method and system for an intelligent loading and unloading robot, so as to solve the problems in the prior art of being unable to perceive environmental changes in real time, the disconnection between posture planning and the actual environment, and the inability to handle dynamic scenes.

[0004] To achieve the above objectives, the following technical solutions are adopted.

[0005] A multi-camera visual stacking pose planning method for an intelligent loading and unloading robot is characterized by comprising the following steps:

[0006] A collaborative perception system is built by deploying multiple industrial cameras in the target space to collect dynamic environmental data of the stacking area in real time;

[0007] Generate adaptive stacking poses based on a hierarchical pose planning mechanism, including: parsing initial pose parameters and generating preliminary placement instructions through a subject control layer, verifying the feasibility of the preliminary placement instructions based on environmental data of the collaborative perception system through a visual verification layer, and optimizing the final placement coordinates through a precise positioning layer combined with a virtual box matching algorithm;

[0008] Executing the virtual box matching algorithm includes: constructing a virtual box model that matches the target cargo size, performing three-dimensional matching between the virtual box model and the actual spatial data collected by the collaborative sensing system, optimizing the contact surface parameters between the virtual box and the actual stacking area through the gradient descent method, and generating a final pose instruction that meets a preset safety margin.

[0009] Optionally, the construction of the collaborative perception system includes:

[0010] Deploy at least three industrial cameras above, to the sides, and at the bottom of the stacking area. Each camera uses a time synchronization protocol to synchronize frame-level data.

[0011] Establish a spatial coordinate system conversion model between cameras and eliminate blind spots in multi-view data collection through feature point matching algorithms;

[0012] The depth data and RGB data collected by multiple cameras are fused in real time to generate a 3D environment topology map that includes space occupancy status, obstacle distribution, and stacking surface morphology.

[0013] Optionally, the feasibility verification of the visual verification layer includes:

[0014] Detecting a spatial occupancy state of a target area corresponding to the preliminary placement instruction based on the three-dimensional environment topology map;

[0015] Calculate the contact area ratio between the virtual box model and the stacking surface topography. If the ratio is lower than the preset threshold, mark it as an infeasible pose.

[0016] Evaluate the reachability of the robotic arm at the target position and determine whether there is a risk of joint overruns based on the robot workspace model.

[0017] Optionally, the optimization process of the virtual cabinet matching algorithm includes:

[0018] Construct a virtual box model with elastic boundaries according to the physical size parameters of the target goods;

[0019] Projecting the virtual box model to a candidate placement area of ​​the three-dimensional environment topology map, and screening a set of candidate poses that meet a minimum safety distance through a collision detection algorithm;

[0020] The posture with the smallest angle between the normal vector of the contact surface and the direction of gravity is selected from the candidate posture set as the optimization benchmark, and the rotation angle and translation amount of the virtual box are iteratively adjusted until the preset stability evaluation index is met.

[0021] Optionally, the calculation of the stability evaluation index includes:

[0022] Obtain the normal vector distribution of the contact points on the stacking surface and calculate the offset between the normal vector distribution and the projection of the center of gravity of the virtual box;

[0023] Based on the contact point pressure distribution model, the slip probability of the virtual box under dynamic load is predicted;

[0024] The stability prediction model is trained based on historical stacking data, and the output is a comprehensive evaluation parameter including the static stability coefficient and dynamic anti-disturbance capability.

[0025] Optionally, generating the initial pose of the subject control layer includes:

[0026] Receive cargo stacking sequence instructions from the upstream scheduling system and analyze the size, weight, and stacking constraints of the target cargo;

[0027] Generate a preliminary stacking layout plan based on the close-packing algorithm and calculate the theoretical placement coordinates of each item in the stacking coordinate system;

[0028] The theoretical placement coordinates are subjected to inverse kinematics solution according to the kinematic model of the robotic arm to generate an initial posture instruction set including a joint angle sequence.

[0029] Optionally, the method further includes an exception handling mechanism:

[0030] When the visual verification layer detects that the pose is infeasible, the dynamic replanning module is triggered to regenerate candidate poses based on the current environment topology map;

[0031] Establish an abnormal posture backtracking database to record the environmental characteristic parameters that lead to posture failure and optimize the constraints of subsequent posture planning;

[0032] When the number of consecutive replanning exceeds a preset threshold, the manual intervention protocol is initiated and environmental reconstruction suggestions are output.

[0033] Multi-camera vision stacking pose planning system for intelligent loading and unloading robots, including:

[0034] A multi-camera collaborative perception module, consisting of multiple spatially deployed industrial cameras, a data synchronization unit, and a 3D reconstruction engine, is used to generate dynamic environmental topology maps in real time.

[0035] A hierarchical pose planning module includes a subject control unit, a visual verification unit, and a precise positioning unit. The subject control unit is used to generate initial pose instructions, the visual verification unit is used to perform feasibility verification based on the environment topology map, and the precise positioning unit is used to execute the virtual box matching algorithm.

[0036] The robot control interface module is used to convert the optimized posture instructions into a motion trajectory that can be executed by the robot and monitor the posture deviation during the execution process in real time.

[0037] Optionally, the precise positioning unit includes:

[0038] A virtual modeling component, used to generate a deformable virtual box model according to target cargo parameters;

[0039] A spatial matching engine for aligning the virtual box model with real-time environmental data in multiple dimensions;

[0040] The optimization calculation component is used to iteratively optimize the contact surface parameters using a physics-based simulation algorithm and output the final pose coordinates that meet the mechanical constraints.

[0041] Optionally, the visual verification unit includes:

[0042] The obstacle detection submodule is used to identify dynamic obstacles in the environment topology map based on a deep learning model;

[0043] The stability prediction submodule is used to calculate the static stability coefficient of the candidate pose through finite element analysis;

[0044] The reachability analysis submodule is used to verify the reachability space range of the target posture in combination with the robot DH parameter model.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The present invention solves the problems in the prior art of lacking real-time environmental perception capabilities, the disconnection between posture planning and the actual environment, and the inability to handle dynamically changing scenarios through a multi-camera collaborative perception system and a hierarchical posture planning mechanism. The multi-camera collaborative perception system can achieve all-round monitoring, ensuring that the robot can perceive the dynamic changes in the stacking area in real time, thereby improving the accuracy of packaging positioning. The hierarchical posture planning mechanism ensures that posture planning can adapt to dynamically changing stacking scenarios through visual verification and precise positioning optimization, thereby enhancing the system's environmental adaptability. Through visual verification and optimization algorithms, it is ensured that the generated posture is safe and feasible in actual operation, thereby improving the overall reliability of the system.

[0047] By deploying at least three industrial cameras above, to the sides, and below the stacking area, this system achieves comprehensive monitoring of the stacking area, eliminating visual blind spots. Using a time synchronization protocol and a spatial coordinate system conversion model, it achieves frame-level synchronization and fusion of multi-camera data, generating a three-dimensional topological map of the environment that includes spatial occupancy status, obstacle distribution, and stacking surface topography, providing more accurate environmental information for posture planning. By fusing depth data with RGB data in real time, a detailed three-dimensional topological map of the environment is generated, providing richer environmental information for posture planning, thereby improving posture planning accuracy.

[0048] This invention ensures that the generated pose is safe and feasible in actual operation by detecting the spatial occupancy of the target area based on a three-dimensional environmental topology map, calculating the contact area percentage, and evaluating the manipulator's accessibility. By evaluating the manipulator's accessibility for the target pose and combining it with the robot's workspace model to determine whether there is a risk of joint overrun, the risk of robot failure during task execution is reduced. By detecting spatial occupancy and contact area percentage, the placement of goods in unstable or unsafe locations is avoided, improving the safety of the stacking process.

[0049] A virtual box matching algorithm, combined with a gradient descent method to optimize contact surface parameters, generates final pose instructions that meet a preset safety margin, ensuring stable and safe cargo placement. A collision detection algorithm screens candidate poses that meet the minimum safety distance and selects the pose with the smallest angle between the contact surface normal and the direction of gravity as the optimization benchmark, further improving stacking stability. The system adapts to complex stacking scenarios by iteratively adjusting the virtual box's rotation angle and translation until the preset stability evaluation criteria are met.

[0050] By calculating the offset between the contact point normal vector distribution and the projection of the virtual box's center of gravity, the slip probability of the virtual box under dynamic load is predicted, providing a scientific basis for posture optimization. A stability prediction model is trained using historical stacking data, outputting comprehensive evaluation parameters including static stability coefficients and dynamic anti-disturbance capabilities to further optimize posture planning. By combining a contact point pressure distribution model with finite element analysis, the accuracy of posture stability prediction is improved, ensuring the stability and reliability of the stacking process.

[0051] This method receives stacking sequence instructions from an upstream scheduling system, analyzes the target cargo's size, weight, and stacking constraints, and generates precise initial pose instructions. A preliminary stacking layout plan is generated based on a close-packing algorithm, and the theoretical placement coordinates of each cargo item in the stacking coordinate system are calculated, improving pose planning efficiency. An inverse kinematics solution generates an initial pose instruction set containing a sequence of joint angles, ensuring the robot can perform complex stacking tasks.

[0052] When the visual verification layer detects an infeasible pose, the dynamic replanning module is triggered to regenerate candidate poses based on the current environmental topology, improving the system's flexibility and adaptability. By establishing a database of abnormal poses, recording the environmental characteristic parameters that led to pose failures, and optimizing the constraints of subsequent pose planning, the system's intelligence is further enhanced. When the number of consecutive replanning attempts exceeds a preset threshold, a manual intervention protocol is initiated and environmental reconstruction recommendations are output, ensuring that the system can respond promptly to extreme situations.

[0053] This invention integrates a multi-camera collaborative perception module, a hierarchical pose planning module, and a robot control interface module to form a complete adaptive stacking pose planning system, improving the system's overall performance. The robot control interface module converts optimized pose instructions into robot-executable motion trajectories and monitors pose deviations in real time during execution to ensure accurate stacking. This modular design enables full automation of the entire process, from environmental perception to pose planning and robot control, improving the automation level of the box packing robot system.

[0054] The present invention uses a virtual modeling component and a spatial matching engine to align the virtual box model with the real-time environmental data in multiple dimensions, ensuring the precise matching of the virtual box with the actual stacking area. The contact surface parameters are iteratively optimized by the optimization calculation component using a physics-based simulation algorithm, and the final posture coordinates that meet the mechanical constraints are output, thereby further improving the stability and reliability of the stacking. Through an efficient optimization algorithm, it is ensured that the optimized posture instructions can be quickly generated in complex scenarios, thereby improving the computational efficiency of the system. The obstacle detection submodule is used to identify dynamic obstacles in the environmental topology map based on a deep learning model, thereby improving the system's adaptability to complex environments. The static stability coefficient of the candidate posture is calculated using finite element analysis through the stability prediction submodule, further optimizing the posture planning and ensuring the stability of the stacking. The reachability analysis submodule is combined with the robot DH parameter model to verify the reachability space range of the target posture, ensuring that the robot can safely execute the posture instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a schematic diagram of the steps of an embodiment of the multi-camera visual stacking pose planning method for an intelligent loading and unloading robot of the present invention.

[0056] Figure 2 This is a module diagram of an embodiment of the multi-camera visual stacking pose planning system for an intelligent loading and unloading robot of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0058] The following detailed description is all exemplary description, and is intended to provide further detailed description of the present invention. Unless otherwise indicated, all technical terms adopted in the present invention have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs. The terms used in the present invention are only for describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present invention.

[0059] Example 1

[0060] like Figure 1 As shown, the present invention provides a multi-camera visual stacking pose planning method for intelligent loading and unloading robots. This method achieves dynamic environmental perception and adaptive pose planning of the stacking area through a multi-camera collaborative perception system and a hierarchical pose planning mechanism. The following are the specific implementation steps of this method:

[0061] Construction of multi-camera collaborative perception system

[0062] Deploy at least three industrial cameras above, to the sides, and at the bottom of the stacking area, including:

[0063] A bird's-eye view camera is deployed above the stacking area to obtain an overall bird's-eye view image of the stacking area, which can clearly observe the outline and overall layout of the stacking area.

[0064] A side-view camera is deployed on both sides of the stacking area to obtain side images of the stacking area, which can observe the height changes of the stacking area and the obstacles on the side.

[0065] An upward-looking camera is deployed at the bottom of the stacking area to obtain the bottom image of the stacking area, so as to observe the bottom support structure and the occupancy of the bottom space.

[0066] Each camera achieves frame-level data synchronization through a time synchronization protocol, ensuring temporal consistency of data collected by multiple cameras. Specifically, through the Network Time Protocol (NTP) or hardware clock synchronization module, the frame acquisition times of all cameras are aligned, ensuring that data collected by each camera at the same time is consistent with each other.

[0067] A spatial coordinate system conversion model is established between cameras, and a feature point matching algorithm is used to eliminate blind spots in multi-view data collection. Specifically, a corner detection algorithm (such as Harris corner detection) is used to extract feature points from each camera's image. Then, a spatial coordinate conversion algorithm (such as a 3D spatial transformation matrix) is used to map feature points from different viewpoints to the same coordinate system, thereby achieving multi-view data fusion.

[0068] The depth data and RGB data collected by multiple cameras are fused in real time to generate a three-dimensional environment topology map that includes the space occupancy status, obstacle distribution, and stacking surface morphology. Specifically, a depth sensor (such as Kinect or ToF camera) is used to obtain depth data, while an RGB camera is used to obtain color image data. A three-dimensional point cloud map containing depth information and color information is generated through a fusion algorithm of depth data and RGB data (such as a depth and color image alignment algorithm). Then, a point cloud processing algorithm (such as voxelization and surface reconstruction) is used to generate a three-dimensional environment topology map that can clearly show the space occupancy status, obstacle distribution, and stacking surface morphology of the stacking area.

[0069] Hierarchical pose planning mechanism

[0070] Theme control layer: Receives cargo stacking sequence instructions from the upstream scheduling system and analyzes the size, weight, and stacking constraints of the target cargo. Specifically, the stacking sequence and related parameters of the cargo are received through the interface with the warehouse management system (WMS) or production scheduling system (MES). A preliminary stacking layout plan is generated based on the close-packing algorithm, and the theoretical placement coordinates of each cargo in the stacking coordinate system are calculated. Specifically, a two-dimensional or three-dimensional packing algorithm (such as a genetic algorithm or a heuristic algorithm) is used to generate the optimal stacking layout plan based on the size and weight of the cargo, and the theoretical placement coordinates of each cargo in the stacking coordinate system are calculated. The theoretical placement coordinates are inversely kinematically solved based on the kinematic model of the manipulator to generate an initial pose instruction set containing a joint angle sequence. Specifically, the kinematic model of the manipulator (such as the DH parameter model) is used to convert the theoretical placement coordinates into a joint angle sequence of the manipulator to generate an initial pose instruction set.

[0071] Visual verification layer: Based on the generated three-dimensional environment topology map, detect the spatial occupancy status of the target area corresponding to the preliminary placement instruction. Specifically, by searching the point cloud data of the target area in the three-dimensional environment topology map, determine whether the area is occupied. Calculate the contact area ratio between the virtual box model and the stacking surface topography, and mark it as an infeasible posture when the ratio is lower than the preset threshold. Specifically, by calculating the overlapping area between the virtual box model and the stacking surface, and comparing it with the bottom area of ​​the virtual box, if the overlapping area ratio is lower than the preset threshold (such as 50%), the posture is considered infeasible. Evaluate the reachability of the manipulator of the target posture, and combine the robot workspace model to determine whether there is a risk of joint overrun. Specifically, use the workspace model of the manipulator (such as the reachability map or inverse kinematics solution) to determine whether the target posture is within the working range of the manipulator and whether there is a risk of joint overrun.

[0072] Precise positioning layer: Execute the virtual box matching algorithm to build a virtual box model that matches the size of the target goods. Specifically, based on the size parameters of the target goods (such as length, width, and height), a virtual box model is generated in the computer. The virtual box model is matched in three dimensions with the actual spatial data collected by the collaborative perception system, and the contact surface parameters between the virtual box and the actual stacking area are optimized by the gradient descent method to generate the final posture instruction that meets the preset safety margin. Specifically, the gradient descent method (such as the least squares method or the gradient descent algorithm in the optimization algorithm library) is used to adjust the position and posture of the virtual box so that the contact surface parameters (such as contact area, contact point distribution, etc.) between the virtual box and the actual stacking area are optimized, and the final posture instruction that meets the preset safety margin (such as the contact area is greater than a certain threshold, the contact points are evenly distributed, etc.) is generated.

[0073] Optimization process of virtual cabinet matching algorithm

[0074] A virtual box model with elastic boundaries is constructed based on the physical dimensions of the target goods. Specifically, the boundaries of the virtual box model can be adjusted based on the actual dimensions of the goods to accommodate goods of varying shapes and sizes. The virtual box model is projected onto a candidate placement area within the 3D environment topology map, and a collision detection algorithm is used to select a set of candidate poses that meet a minimum safety distance. Specifically, a collision detection algorithm (such as the separating axis theorem or a spatial partitioning algorithm) is used to detect collisions between the virtual box and the surrounding environment, and a set of candidate poses that meet a minimum safety distance (e.g., 10 mm) is selected. From the candidate poses, the pose with the smallest angle between the contact surface normal vector and the direction of gravity is selected as the optimization benchmark. The rotation angle and translation of the virtual box are iteratively adjusted until a preset stability evaluation criterion is met. Specifically, the angle between the contact surface normal vector and the direction of gravity is calculated, and the pose with the smallest angle is selected as the optimization benchmark. An iterative algorithm (such as a gradient descent method or a genetic algorithm) is then used to adjust the rotation angle and translation of the virtual box until a preset stability evaluation criterion (e.g., a contact area greater than a certain threshold, uniform distribution of contact points, etc.) is met.

[0075] Calculation of stability evaluation index

[0076] The normal vector distribution of contact points on the stacking surface is obtained, and the offset between these normal vectors and the projection of the virtual box's center of gravity is calculated. Specifically, the normal vectors of the contact points are extracted from the 3D environment topology map, and the offset between these normal vectors and the projection of the virtual box's center of gravity is calculated to assess the stability of cargo placement. Based on the contact point pressure distribution model, the slip probability of the virtual box under dynamic loads is predicted. Specifically, a contact point pressure distribution model (such as a finite element analysis model or a mechanical model) is used to predict the slip probability of the virtual box under dynamic loads (such as the weight of the cargo, the movement of the robotic arm, etc.). A stability prediction model is trained with historical stacking data to output a comprehensive evaluation parameter comprising a static stability coefficient and dynamic anti-disturbance capability. Specifically, a stability prediction model (such as a machine learning model or a deep learning model) is trained with historical stacking data (such as past successful and failed stacking cases) to output a comprehensive evaluation parameter comprising a static stability coefficient (such as contact area, contact point distribution, etc.) and dynamic anti-disturbance capability (such as slip probability, anti-overturning capability, etc.).

[0077] Exception handling mechanism

[0078] When the visual verification layer detects an infeasible pose, the dynamic replanning module is triggered to regenerate candidate poses based on the current environmental topology. Specifically, when the visual verification layer determines that a pose is infeasible, the dynamic replanning module recalculates the candidate poses and generates new pose instructions using the current environmental topology and cargo parameters. An abnormal pose backtracking database is established to record the environmental characteristic parameters that lead to pose failures and optimize the constraints of subsequent pose planning. Specifically, the environmental characteristic parameters that lead to pose failures (such as obstacle locations and stacking surface topography) are recorded in the abnormal pose backtracking database. This data is used to optimize the constraints of subsequent pose planning and improve the success rate of pose planning. When the number of consecutive replannings exceeds a preset threshold, the manual intervention protocol is initiated and environmental reconstruction suggestions are output. Specifically, if the number of consecutive replannings exceeds a preset threshold (e.g., 5 times), the manual intervention protocol is initiated, notifying the operator to intervene and outputting environmental reconstruction suggestions (such as clearing obstacles and adjusting the stacking area).

[0079] Example 2

[0080] like Figure 2 As shown, the present invention also provides a multi-camera visual stacking posture planning system for intelligent loading and unloading robots. Based on the above method, the system realizes the automation of the entire process from environmental perception to posture planning and then to robot control. The following is a specific embodiment of the system:

[0081] Multi-camera collaborative perception module

[0082] The module includes multiple industrial cameras deployed in space, a data synchronization unit and a three-dimensional reconstruction engine. Specifically, multiple industrial cameras are installed above, on the side and at the bottom of the stacking area according to the deployment method in the above embodiment 1. The data synchronization unit realizes frame-level data synchronization of multiple cameras through a time synchronization protocol (such as NTP or hardware clock synchronization module). The three-dimensional reconstruction engine uses a fusion algorithm of depth data and RGB data (such as a depth and color image alignment algorithm) to generate a three-dimensional environment topology map including space occupancy status, obstacle distribution and stacking surface morphology. This all-round perception module provides accurate environmental information for posture planning, ensuring the real-time and adaptability of the system.

[0083] Hierarchical pose planning module

[0084] This module includes a subject control unit, a visual verification unit, and a precise positioning unit. The subject control unit is used to generate initial pose instructions. Specifically, by receiving the cargo stacking sequence instructions from the upstream scheduling system, it parses the size, weight, and stacking constraints of the target cargo, generates a preliminary stacking layout plan using the close-packing algorithm, and generates an initial pose instruction set based on the robot arm kinematic model. The visual verification unit is used to perform feasibility verification based on the environmental topology map. Specifically, it determines the feasibility of the pose by detecting the spatial occupancy status of the target area, calculating the contact area ratio, and evaluating the accessibility of the robot arm. The precise positioning unit is used to execute the virtual box matching algorithm. Specifically, it generates the final pose instructions by constructing a virtual box model, performing three-dimensional matching, and optimizing the contact surface parameters. This hierarchical planning mechanism ensures the accuracy and reliability of pose planning.

[0085] Robot control interface module

[0086] This module is used to convert optimized posture commands into motion trajectories that can be executed by the robot and monitor posture deviations during execution in real time. Specifically, through the robot control interface module (such as a ROS node or robot control software), the posture commands are converted into motion trajectory commands for the robot arm and sent to the robot control system. At the same time, sensors (such as force sensors or visual sensors) are used to monitor the movement of the robot arm in real time, detect posture deviations, and adjust the robot arm's movements in a timely manner to ensure the precise execution of the stacking operation.

[0087] Precision positioning unit

[0088] The unit includes a virtual modeling component, a spatial matching engine, and an optimization calculation component. The virtual modeling component is used to generate a deformable virtual box model based on the target cargo parameters. Specifically, a virtual box model with elastic boundaries is generated in the computer based on the size, shape, and weight parameters of the cargo. The spatial matching engine is used to align the virtual box model with real-time environmental data in multiple dimensions. Specifically, through collision detection algorithms and spatial transformation algorithms, the virtual box model is aligned with the candidate placement areas in the three-dimensional environmental topology map. The optimization calculation component is used to iteratively optimize the contact surface parameters using a physics-based simulation algorithm and output the final pose coordinates that meet the mechanical constraints. Specifically, it uses optimization algorithms such as gradient descent or genetic algorithms to adjust the position and posture of the virtual box to generate final pose coordinates that meet preset stability evaluation indicators. This optimization process ensures the stability and safety of cargo placement.

[0089] Visual Verification Unit

[0090] This unit includes an obstacle detection submodule, a stability prediction submodule and a reachability analysis submodule. The obstacle detection submodule is used to identify dynamic obstacles in the environment topology map based on a deep learning model. Specifically, a deep learning algorithm (such as a convolutional neural network) is used to train the obstacle detection model to identify dynamic obstacles in the environment topology map. The stability prediction submodule is used to calculate the static stability coefficient of the candidate posture through finite element analysis. Specifically, the static stability coefficient of the candidate posture is calculated using a finite element analysis model (such as ANSYS or ABAQUS). The reachability analysis submodule is used to verify the reachability space range of the target posture in combination with the robot DH parameter model. Specifically, the robot DH parameter model is used to determine whether the target posture is within the working range of the robotic arm and whether it meets the reachability requirements. This multi-dimensional verification mechanism ensures the feasibility and reliability of posture planning.

[0091] It is understood from common technical knowledge that the present invention may be implemented by other embodiments that do not depart from its spirit or essential features. Therefore, the embodiments disclosed above are, in all respects, merely illustrative and not exclusive. All modifications within the scope of the present invention or equivalent to the scope of the present invention are intended to be encompassed by the present invention.

Claims

1. A multi-camera visual stacking pose planning method for intelligent loading and unloading robots, characterized by: The following steps are involved: A collaborative perception system is built by deploying multiple industrial cameras in the target space to collect dynamic environmental data of the stacking area in real time; Generate adaptive stacking poses based on a hierarchical pose planning mechanism, including: parsing initial pose parameters and generating preliminary placement instructions through a subject control layer, verifying the feasibility of the preliminary placement instructions based on environmental data of the collaborative perception system through a visual verification layer, and optimizing the final placement coordinates through a precise positioning layer combined with a virtual box matching algorithm; Executing the virtual box matching algorithm includes: constructing a virtual box model that matches the target cargo size, performing three-dimensional matching between the virtual box model and the actual spatial data collected by the collaborative sensing system, optimizing the contact surface parameters between the virtual box and the actual stacking area through the gradient descent method, and generating a final pose instruction that meets a preset safety margin.

2. The multi-camera visual stacking pose planning method for an intelligent loading and unloading robot according to claim 1 is characterized in that: The construction of the collaborative perception system includes: Deploy at least three industrial cameras above, to the sides, and at the bottom of the stacking area. Each camera uses a time synchronization protocol to synchronize frame-level data. Establish a spatial coordinate system conversion model between cameras and eliminate blind spots in multi-view data collection through feature point matching algorithms; The depth data and RGB data collected by multiple cameras are fused in real time to generate a 3D environment topology map that includes space occupancy status, obstacle distribution, and stacking surface morphology.

3. The multi-camera visual stacking pose planning method for an intelligent loading and unloading robot according to claim 1 is characterized in that: The feasibility verification of the visual verification layer includes: Detecting a spatial occupancy state of a target area corresponding to the preliminary placement instruction based on the three-dimensional environment topology map; Calculate the contact area ratio between the virtual box model and the stacking surface topography. If the ratio is lower than the preset threshold, mark it as an infeasible pose. Evaluate the reachability of the robotic arm at the target position and determine whether there is a risk of joint overruns based on the robot workspace model.

4. The multi-camera visual stacking pose planning method for an intelligent loading and unloading robot according to claim 1 is characterized in that: The optimization process of the virtual box matching algorithm includes: Construct a virtual box model with elastic boundaries according to the physical size parameters of the target goods; Projecting the virtual box model to a candidate placement area of ​​the three-dimensional environment topology map, and screening a set of candidate poses that meet a minimum safety distance through a collision detection algorithm; The posture with the smallest angle between the normal vector of the contact surface and the direction of gravity is selected from the candidate posture set as the optimization benchmark, and the rotation angle and translation amount of the virtual box are iteratively adjusted until the preset stability evaluation index is met.

5. The multi-camera visual stacking pose planning method for an intelligent loading and unloading robot according to claim 4 is characterized in that: The calculation of the stability evaluation index includes: Obtain the normal vector distribution of the contact points on the stacking surface and calculate the offset between the normal vector distribution and the projection of the center of gravity of the virtual box; Based on the contact point pressure distribution model, the slip probability of the virtual box under dynamic load is predicted; The stability prediction model is trained based on historical stacking data, and the output is a comprehensive evaluation parameter including the static stability coefficient and dynamic anti-disturbance capability.

6. The multi-camera visual stacking pose planning method for an intelligent loading and unloading robot according to claim 1 is characterized in that: The initial pose generation of the subject control layer includes: Receive cargo stacking sequence instructions from the upstream scheduling system and analyze the size, weight, and stacking constraints of the target cargo; Generate a preliminary stacking layout plan based on the close-packing algorithm and calculate the theoretical placement coordinates of each item in the stacking coordinate system; The theoretical placement coordinates are subjected to inverse kinematics solution according to the kinematic model of the robotic arm to generate an initial posture instruction set including a joint angle sequence.

7. The multi-camera visual stacking pose planning method for an intelligent loading and unloading robot according to claim 1 is characterized in that: The method also includes an exception handling mechanism: When the visual verification layer detects that the pose is infeasible, the dynamic replanning module is triggered to regenerate candidate poses based on the current environment topology map; Establish an abnormal posture backtracking database to record the environmental characteristic parameters that lead to posture failure and optimize the constraints of subsequent posture planning; When the number of consecutive replanning exceeds a preset threshold, the manual intervention protocol is initiated and environmental reconstruction suggestions are output.

8. A multi-camera visual stacking pose planning system for an intelligent loading and unloading robot, based on the multi-camera visual stacking pose planning method for an intelligent loading and unloading robot according to any one of claims 1 to 7, characterized in that: include: A multi-camera collaborative perception module, consisting of multiple spatially deployed industrial cameras, a data synchronization unit, and a 3D reconstruction engine, is used to generate dynamic environmental topology maps in real time. A hierarchical pose planning module includes a subject control unit, a visual verification unit, and a precise positioning unit. The subject control unit is used to generate initial pose instructions, the visual verification unit is used to perform feasibility verification based on the environment topology map, and the precise positioning unit is used to execute the virtual box matching algorithm. The robot control interface module is used to convert the optimized posture instructions into a motion trajectory that can be executed by the robot and monitor the posture deviation during the execution process in real time.

9. The multi-camera visual stacking posture planning system for intelligent loading and unloading robots according to claim 8 is characterized in that: The precise positioning unit comprises: A virtual modeling component, used to generate a deformable virtual box model according to target cargo parameters; A spatial matching engine for aligning the virtual box model with real-time environmental data in multiple dimensions; The optimization calculation component is used to iteratively optimize the contact surface parameters using a physics-based simulation algorithm and output the final pose coordinates that meet the mechanical constraints.

10. The multi-camera visual stacking posture planning system for intelligent loading and unloading robots according to claim 8, characterized in that: The visual verification unit includes: The obstacle detection submodule is used to identify dynamic obstacles in the environment topology map based on a deep learning model; The stability prediction submodule is used to calculate the static stability coefficient of the candidate pose through finite element analysis; The reachability analysis submodule is used to verify the reachability space range of the target posture in combination with the robot DH parameter model.

Images