Multi-camera vision stack pose planning method and system for intelligent loading and unloading robot

Abstract

The present application relates to the field of industrial automation, and in particular to a multi-camera vision stack pose planning method and system for intelligent loading and unloading robots. The method comprises the following steps: constructing a collaborative perception system through a plurality of industrial cameras deployed in a target space, and collecting dynamic environment data of a stacking area in real time; generating an adaptive stacking pose based on a hierarchical pose planning mechanism, including: analyzing initial pose parameters and generating preliminary placement instructions through a theme control layer, verifying the feasibility of the preliminary placement instructions based on the environment data of the collaborative perception system through a vision verification layer, and optimizing the final placement coordinates through a precise positioning layer combined with a virtual box matching algorithm. The present application solves the problems of lack of real-time environment perception capability, disconnection between pose planning and actual environment, and inability to handle dynamic changing scenarios in the prior art.

Description

Technical Field

[0001] This invention relates to the field of industrial automation, specifically to a multi-camera vision stacking pose planning method and system for intelligent loading and unloading robots. Background Technology

[0002] In the field of industrial automation, packing robot systems have been widely used, especially in logistics, warehousing, and manufacturing. However, existing packing robot systems still have many problems in practical applications. First, these systems generally lack real-time environmental perception capabilities and cannot accurately perceive dynamic changes in the stacking area. For example, during multi-layer stacking, due to the inability to monitor minute deformations or obstacles on the stacking surface in real time, the robot may place goods in unstable or unsafe positions, leading to tipping or damage. Second, existing pose planning systems are disconnected from the actual environment and cannot adjust the planning scheme based on real-time environmental data. This makes it difficult for the robot to generate reasonable placement poses when facing complex stacking scenarios, thus affecting packing efficiency and quality. In addition, existing systems cannot handle dynamically changing stacking scenarios; once the environment changes, the robot cannot adjust its actions in time. Finally, existing systems lack a mechanism to verify the rationality of the planned pose, failing to ensure that the generated pose is safe and feasible in actual operation. Therefore, there is an urgent need for intelligent loading and unloading robot multi-camera vision stacking pose planning methods and systems. Summary of the Invention

[0003] The purpose of this invention is to provide a multi-camera vision stacking pose planning method and system for intelligent loading and unloading robots, so as to solve the problems of existing technologies such as inability to perceive environmental changes in real time, disconnect between pose planning and actual environment, and inability to handle dynamic scenes.

[0004] To achieve the above objectives, the following technical solution is adopted.

[0005] A multi-camera vision-based stacking pose planning method for intelligent loading and unloading robots, characterized by the following steps:

[0006] A collaborative sensing system is built by deploying multiple industrial cameras in the target space to collect dynamic environmental data of the stacking area in real time.

[0007] The adaptive stacking pose is generated based on a hierarchical pose planning mechanism, including: parsing the initial pose parameters and generating preliminary placement instructions through the theme control layer; verifying the feasibility of the preliminary placement instructions based on the environmental data of the collaborative perception system through the visual verification layer; and optimizing the final placement coordinates through the precise positioning layer combined with the virtual box matching algorithm.

[0008] The virtual container matching algorithm includes: constructing a virtual container model that matches the size of the target cargo; performing three-dimensional matching between the virtual container model and the actual spatial data collected by the collaborative sensing system; optimizing the contact surface parameters between the virtual container and the actual stacking area using the gradient descent method; and generating a final pose command that meets the preset safety margin.

[0009] Optionally, the construction of the collaborative sensing system includes:

[0010] At least three industrial cameras are deployed above, to the sides and bottom of the stacking area, and each camera achieves frame-level data synchronization through a time synchronization protocol;

[0011] Establish a spatial coordinate system transformation model between cameras, and eliminate blind spots in multi-view data acquisition through feature point matching algorithms;

[0012] Real-time fusion of depth data and RGB data collected by multiple cameras generates a 3D environmental topology map that includes space occupancy status, obstacle distribution, and stack surface morphology.

[0013] Optionally, the feasibility verification of the visual verification layer includes:

[0014] Based on the three-dimensional environment topology map, the space occupancy status of the target area corresponding to the initial placement command is detected;

[0015] Calculate the contact area ratio between the virtual box model and the surface topography of the stack, and mark the pose as infeasible when the ratio is lower than a preset threshold;

[0016] Assess the reachability of the robotic arm to the target pose and determine whether there is a risk of joint over-limit based on the robot's workspace model.

[0017] Optionally, the optimization process of the virtual box matching algorithm includes:

[0018] Construct a virtual container model with flexible boundaries based on the physical dimensions of the target cargo;

[0019] The virtual box model is projected onto the candidate placement area of ​​the three-dimensional environment topology map, and a collision detection algorithm is used to filter the candidate pose set that meets the minimum safe distance.

[0020] The pose with the smallest angle between the contact surface normal vector and the gravity direction is selected from the candidate pose set as the optimization benchmark. The rotation angle and translation of the virtual box are iteratively adjusted until the preset stability evaluation index is met.

[0021] Optionally, the calculation of the stability evaluation index includes:

[0022] Obtain the normal vector distribution of the contact points on the stacked surface and calculate its offset from the projection of the virtual box's center of gravity.

[0023] Based on the contact point pressure distribution model, predict the slip probability of the virtual box under dynamic load;

[0024] A stability prediction model is trained by combining historical stacking data, and the output is a comprehensive evaluation parameter that includes static stability coefficient and dynamic disturbance resistance capability.

[0025] Optionally, the initial pose generation of the topic control layer includes:

[0026] Receive cargo stacking order instructions from the upstream scheduling system and parse the size, weight, and stacking constraints of the target cargo;

[0027] A preliminary stacking layout scheme is generated based on the close packing algorithm, and the theoretical placement coordinates of each item in the stacking coordinate system are calculated.

[0028] Based on the kinematic model of the robotic arm, inverse kinematics calculations are performed on the theoretical placement coordinates to generate an initial pose command set containing joint angle sequences.

[0029] Optionally, the method further includes an exception handling mechanism:

[0030] When the visual verification layer detects that the pose is not feasible, it triggers the dynamic replanning module to regenerate candidate poses based on the current environment topology graph.

[0031] Establish an abnormal pose backtracking database to record environmental feature parameters that cause pose failure and optimize the constraints of subsequent pose planning.

[0032] When the number of consecutive replanning attempts exceeds a preset threshold, a manual intervention protocol is initiated and environmental reconstruction suggestions are output.

[0033] The intelligent loading and unloading robot multi-camera vision stacking pose planning system includes:

[0034] The multi-camera collaborative sensing module includes multiple spatially deployed industrial cameras, a data synchronization unit, and a 3D reconstruction engine, used to generate dynamic environmental topology maps in real time.

[0035] The hierarchical pose planning module includes a theme control unit, a visual verification unit, and a precise positioning unit. The theme control unit is used to generate initial pose commands, the visual verification unit is used to perform feasibility verification based on the environmental topology map, and the precise positioning unit is used to execute a virtual box matching algorithm.

[0036] The robot control interface module is used to convert optimized pose commands into executable motion trajectories for the robot and to monitor pose deviations in real time during execution.

[0037] Optionally, the precise positioning unit includes:

[0038] Virtual modeling component, used to generate deformable virtual container models based on target cargo parameters;

[0039] A spatial matching engine is used to align the virtual container model with real-time environmental data in multiple dimensions.

[0040] The computational component is optimized to iteratively optimize the contact surface parameters using a physics-based simulation algorithm, and output the final pose coordinates that satisfy the mechanical constraints.

[0041] Optionally, the visual verification unit includes:

[0042] The obstacle detection submodule is used to identify dynamic obstacles in the environmental topology map based on a deep learning model.

[0043] The stability prediction submodule is used to calculate the static stability coefficient of candidate poses through finite element analysis.

[0044] The reachability analysis submodule is used to verify the reachability space range of the target pose in conjunction with the robot's DH parameter model.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] This invention addresses the shortcomings of existing technologies, such as lack of real-time environmental perception, disconnect between pose planning and the actual environment, and inability to handle dynamically changing scenarios, through a multi-camera collaborative perception system and a hierarchical pose planning mechanism. The multi-camera collaborative perception system enables omnidirectional monitoring, ensuring the robot can perceive dynamic changes in the stacking area in real time, thereby improving the accuracy of packing and positioning. The hierarchical pose planning mechanism, through visual verification and precise positioning optimization, ensures that pose planning can adapt to dynamically changing stacking scenarios, enhancing the system's environmental adaptability. Visual verification and optimization algorithms ensure that the generated poses are safe and feasible in actual operation, thus improving the overall reliability of the system.

[0047] This invention achieves omnidirectional monitoring of the stacking area by deploying at least three industrial cameras above, to the sides, and at the bottom, eliminating blind spots. Through a time synchronization protocol and a spatial coordinate system transformation model, frame-level synchronization and fusion of multi-camera data are achieved, generating a 3D environmental topology map that includes spatial occupancy status, obstacle distribution, and stack surface morphology, providing more accurate environmental information for pose planning. By fusing depth data and RGB data in real time, a detailed 3D environmental topology map is generated, providing richer environmental information for pose planning and thus improving its accuracy.

[0048] This invention detects the spatial occupancy status of a target area based on a 3D environmental topology map, calculates the contact area ratio, and evaluates the reachability of the robotic arm, ensuring that the generated pose is safe and feasible in actual operation. By evaluating the robotic arm's reachability to the target pose and combining it with the robot's workspace model to determine whether there is a risk of joint over-limit, the risk of robot malfunction during task execution is reduced. By detecting the spatial occupancy status and contact area ratio, the placement of goods in unstable or unsafe locations is avoided, improving the safety of the stacking process.

[0049] A virtual box matching algorithm, combined with gradient descent, optimizes contact surface parameters to generate a final pose command that meets a preset safety margin, ensuring the stability and safety of cargo placement. A collision detection algorithm filters candidate poses that meet the minimum safety distance, and the pose with the smallest angle between the contact surface normal and the gravity direction is selected as the optimization benchmark, further improving stacking stability. By iteratively adjusting the rotation angle and translation of the virtual box until the preset stability evaluation index is met, the system can adapt to complex stacking scenarios.

[0050] By calculating the offset between the contact point normal vector distribution and the projection of the virtual box's center of gravity, the slip probability of the virtual box under dynamic loads is predicted, providing a scientific basis for pose optimization. A stability prediction model is trained using historical stacking data, outputting comprehensive evaluation parameters including static stability coefficients and dynamic disturbance resistance capabilities, further optimizing pose planning. Through contact point pressure distribution models and finite element analysis, the accuracy of pose stability prediction is improved, ensuring the stability and reliability of the stacking process.

[0051] This invention receives cargo stacking order instructions from the upstream scheduling system, analyzes the size, weight, and stacking constraints of the target cargo, and generates precise initial pose instructions. Based on a close-packing algorithm, a preliminary stacking layout scheme is generated, and the theoretical placement coordinates of each cargo in the stacking coordinate system are calculated, improving the efficiency of pose planning. An initial pose instruction set containing joint angle sequences is generated through inverse kinematics calculation, ensuring the robot can perform complex stacking tasks.

[0052] When the visual verification layer detects that a pose is infeasible, it triggers the dynamic replanning module to regenerate candidate poses based on the current environment topology, improving the system's flexibility and adaptability. By establishing an abnormal pose backtracking database, environmental feature parameters that cause pose failures are recorded, and the constraints of subsequent pose planning are optimized, further improving the system's intelligence level. When the number of consecutive replanning attempts exceeds a preset threshold, a manual intervention protocol is initiated and environmental reconstruction suggestions are output to ensure that the system can be handled promptly in extreme cases.

[0053] This invention integrates a multi-camera collaborative sensing module, a hierarchical pose planning module, and a robot control interface module into a single unit, forming a complete adaptive stacking pose planning system and improving the overall system performance. The robot control interface module converts optimized pose commands into executable robot trajectories and monitors pose deviations in real time to ensure precise stacking. Through modular design, it achieves full automation from environmental perception to pose planning and robot control, improving the automation level of the packing robot system.

[0054] This invention utilizes a virtual modeling component and a spatial matching engine to align the virtual box model with real-time environmental data in multiple dimensions, ensuring precise matching between the virtual box and the actual stacking area. By optimizing the computational component using a physics-based simulation algorithm, iterative optimization of contact surface parameters is achieved, outputting final pose coordinates that satisfy mechanical constraints, further improving the stability and reliability of stacking. Efficient optimization algorithms ensure rapid generation of optimized pose commands in complex scenarios, improving system computational efficiency. An obstacle detection submodule, based on a deep learning model, identifies dynamic obstacles in the environmental topology map, enhancing the system's adaptability to complex environments. A stability prediction submodule uses finite element analysis to calculate the static stability coefficients of candidate poses, further optimizing pose planning and ensuring stacking stability. An accessibility analysis submodule, combined with the robot's DH parameter model, verifies the reachability space of the target pose, ensuring the robot can safely execute pose commands. Attached Figure Description

[0055] Figure 1 This is a schematic diagram illustrating the steps of an embodiment of the multi-camera vision stacking pose planning method for intelligent loading and unloading robots according to the present invention.

[0056] Figure 2 This is a schematic diagram of a module in an embodiment of the multi-camera vision stacking pose planning system for intelligent loading and unloading robots of the present invention. Detailed Implementation

[0057] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0058] The following detailed description is exemplary and intended to provide further detailed explanation of the invention. Unless otherwise specified, all technical terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention.

[0059] Example 1

[0060] like Figure 1 As shown, this invention provides a multi-camera vision-based stacking pose planning method for intelligent loading and unloading robots. This method achieves dynamic environmental perception and adaptive pose planning of the stacking area through a multi-camera collaborative perception system and a hierarchical pose planning mechanism. The specific implementation steps of this method are as follows:

[0061] Construction of Multi-Camera Collaborative Sensing System

[0062] At least three industrial cameras are deployed above, to the sides, and at the bottom of the stacking area, specifically including:

[0063] A top-down camera is deployed above the stacking area to obtain an overall top-down image of the stacking area, which can clearly show the outline and overall layout of the stacking area.

[0064] A side-view camera is deployed on each side of the stacking area to acquire side images of the stacking area, which can observe the height changes of the stacking area and the situation of obstacles on the side.

[0065] A bottom-view camera is deployed at the bottom of the stacking area to obtain bottom images of the stacking area, allowing observation of the bottom support structure and the occupancy of the bottom space.

[0066] Each camera achieves frame-level data synchronization through a time synchronization protocol, ensuring that the data acquired by multiple cameras is consistent in time. Specifically, the frame acquisition times of all cameras are aligned through the Network Time Protocol (NTP) or a hardware clock synchronization module, so that the data acquired by each camera at the same point in time can be matched with each other.

[0067] A spatial coordinate system transformation model between cameras is established, and blind spots in multi-view data acquisition are eliminated through feature point matching algorithms. Specifically, corner detection algorithms (such as Harris corner detection) are used to extract feature points from images from each camera, and then spatial coordinate transformation algorithms (such as 3D spatial transformation matrices) are used to map feature points from different viewpoints to the same coordinate system, thereby achieving the fusion of multi-view data.

[0068] This method fuses depth and RGB data acquired from multiple cameras in real time to generate a 3D environment topology map that includes spatial occupancy status, obstacle distribution, and stack surface morphology. Specifically, depth data is acquired using a depth sensor (such as a Kinect or ToF camera), while color image data is acquired using an RGB camera. A fusion algorithm for depth and RGB data (such as a depth and color image alignment algorithm) is used to generate a 3D point cloud map containing both depth and color information. Then, point cloud processing algorithms (such as voxelization and surface reconstruction) are used to generate the 3D environment topology map, which clearly represents the spatial occupancy status, obstacle distribution, and surface morphology of the stacked area.

[0069] Layered pose planning mechanism

[0070] The main control layer receives cargo stacking order instructions from the upstream scheduling system and parses the size, weight, and stacking constraints of the target cargo. Specifically, it receives the cargo stacking order and related parameters through an interface with a warehouse management system (WMS) or production scheduling system (MES). It generates a preliminary stacking layout scheme based on a close-packing algorithm and calculates the theoretical placement coordinates of each cargo in the stacking coordinate system. Specifically, it uses a two-dimensional or three-dimensional packing algorithm (such as a genetic algorithm or heuristic algorithm) to generate the optimal stacking layout scheme based on the cargo size and weight, and calculates the theoretical placement coordinates of each cargo in the stacking coordinate system. It performs inverse kinematics calculation on the theoretical placement coordinates based on the robotic arm kinematic model, generating an initial pose instruction set containing joint angle sequences. Specifically, it uses the robotic arm's kinematic model (such as a DH parameter model) to convert the theoretical placement coordinates into a sequence of robotic arm joint angles, generating the initial pose instruction set.

[0071] Visual Verification Layer: Based on the generated 3D environment topology map, it detects the spatial occupancy status of the target area corresponding to the initial placement command. Specifically, it determines whether the area is occupied by searching the point cloud data of the target area in the 3D environment topology map. It calculates the contact area ratio between the virtual box model and the stack surface topology; if the ratio is lower than a preset threshold, it is marked as an infeasible pose. Specifically, it calculates the overlap area between the virtual box model and the stack surface and compares it with the bottom area of ​​the virtual box. If the overlap area ratio is lower than a preset threshold (e.g., 50%), the pose is considered infeasible. It evaluates the reachability of the target pose to the robotic arm and determines whether there is a risk of joint over-limit using the robot's workspace model. Specifically, it uses the robotic arm's workspace model (e.g., reachability map or inverse kinematics solution) to determine whether the target pose is within the robotic arm's working range and whether there is a risk of joint over-limit.

[0072] Precise Positioning Layer: This layer executes a virtual container matching algorithm to construct a virtual container model that matches the dimensions of the target cargo. Specifically, a virtual container model is generated in the computer based on the size parameters of the target cargo (e.g., length, width, and height). The virtual container model is then 3D matched with the actual spatial data collected by the collaborative sensing system. The contact surface parameters between the virtual container and the actual stacking area are optimized using gradient descent to generate a final pose command that meets a preset safety margin. Specifically, the position and orientation of the virtual container are adjusted using gradient descent (e.g., least squares method or gradient descent algorithm from an optimization algorithm library) to optimize the contact surface parameters (e.g., contact area, contact point distribution) between the virtual container and the actual stacking area, generating a final pose command that meets a preset safety margin (e.g., contact area greater than a certain threshold, uniform contact point distribution).

[0073] Optimization process of virtual box matching algorithm

[0074] A virtual container model with flexible boundaries is constructed based on the physical dimensions of the target cargo. Specifically, the boundaries of the virtual container model can be adjusted according to the actual dimensions of the cargo to accommodate cargo of different shapes and sizes. The virtual container model is projected onto the candidate placement area of ​​the 3D environment topology map, and a collision detection algorithm is used to filter candidate poses that meet the minimum safe distance. Specifically, a collision detection algorithm (such as the separating axis theorem or spatial partitioning algorithm) is used to detect collisions between the virtual container and the surrounding environment, and a candidate pose set that meets the minimum safe distance (e.g., 10 mm) is selected. The pose with the smallest angle between the contact surface normal vector and the gravity direction is selected as the optimization benchmark, and the rotation angle and translation of the virtual container are iteratively adjusted until the preset stability evaluation index is met. Specifically, by calculating the angle between the contact surface normal vector and the gravity direction, the pose with the smallest angle is selected as the optimization benchmark, and then an iterative algorithm (such as gradient descent or genetic algorithm) is used to adjust the rotation angle and translation of the virtual container until the preset stability evaluation index (e.g., contact area greater than a certain threshold, uniform distribution of contact points, etc.) is met.

[0075] Calculation of stability evaluation index

[0076] Obtain the normal vector distribution of the contact points on the stacking surface and calculate their offset from the projection of the virtual box's center of gravity. Specifically, extract the normal vectors of the contact points from the 3D environment topology map, calculate the offset of these normal vectors from the projection of the virtual box's center of gravity, and evaluate the stability of cargo placement. Based on the contact point pressure distribution model, predict the slip probability of the virtual box under dynamic loads. Specifically, use the contact point pressure distribution model (such as a finite element analysis model or a mechanical model) to predict the slip probability of the virtual box under dynamic loads (such as cargo weight, robotic arm movement, etc.). Train a stability prediction model using historical stacking data, and output comprehensive evaluation parameters including static stability coefficients and dynamic disturbance resistance capabilities. Specifically, use historical stacking data (such as past successful and failed stacking cases) to train a stability prediction model (such as a machine learning model or a deep learning model), and output comprehensive evaluation parameters including static stability coefficients (such as contact area, contact point distribution, etc.) and dynamic disturbance resistance capabilities (such as slip probability, anti-overturning capability, etc.).

[0077] Exception handling mechanism

[0078] When the visual verification layer detects that a pose is infeasible, the dynamic replanning module is triggered to regenerate candidate poses based on the current environmental topology. Specifically, when the visual verification layer determines that a certain pose is infeasible, the dynamic replanning module recalculates the candidate poses, using the current environmental topology and cargo parameters to generate new pose instructions. An abnormal pose backtracking database is established to record environmental feature parameters that cause pose failures, optimizing the constraints of subsequent pose planning. Specifically, environmental feature parameters that cause pose failures (such as obstacle positions, stack surface topography, etc.) are recorded in the abnormal pose backtracking database, and this data is used to optimize the constraints of subsequent pose planning, improving the success rate of pose planning. When the number of consecutive replanning attempts exceeds a preset threshold, a manual intervention protocol is initiated and environmental reconstruction suggestions are output. Specifically, if the number of consecutive replanning attempts exceeds a preset threshold (e.g., 5 times), a manual intervention protocol is initiated, notifying operators to intervene and outputting environmental reconstruction suggestions (such as clearing obstacles, adjusting stacking areas, etc.).

[0079] Example 2

[0080] like Figure 2 As shown, this invention also provides a multi-camera vision stacking pose planning system for intelligent loading and unloading robots. Based on the above method, this system achieves full automation from environmental perception to pose planning and then to robot control. The following is a detailed implementation of this system:

[0081] Multi-camera collaborative sensing module

[0082] This module comprises multiple spatially deployed industrial cameras, a data synchronization unit, and a 3D reconstruction engine. Specifically, the multiple industrial cameras are installed above, to the sides, and at the bottom of the stacking area according to the deployment method described in Embodiment 1 above. The data synchronization unit achieves frame-level data synchronization among the multiple cameras through a time synchronization protocol (such as NTP or a hardware clock synchronization module). The 3D reconstruction engine utilizes a fusion algorithm of depth data and RGB data (such as a depth and color image alignment algorithm) to generate a 3D environmental topology map that includes spatial occupancy status, obstacle distribution, and the surface topography of the stack. This comprehensive perception module provides accurate environmental information for pose planning, ensuring the real-time performance and adaptability of the system.

[0083] Layered Pose Planning Module

[0084] This module comprises a subject control unit, a vision verification unit, and a precise positioning unit. The subject control unit generates initial pose commands. Specifically, it receives cargo stacking order commands from the upstream scheduling system, analyzes the target cargo's dimensions, weight, and stacking constraints, generates a preliminary stacking layout scheme using a close-packing algorithm, and generates an initial pose command set based on the robotic arm's kinematic model. The vision verification unit performs feasibility verification based on the environmental topology map. Specifically, it determines the feasibility of the pose by detecting the spatial occupancy status of the target area, calculating the contact area ratio, and evaluating the robotic arm's accessibility. The precise positioning unit executes a virtual box matching algorithm. Specifically, it generates the final pose commands by constructing a virtual box model, performing 3D matching, and optimizing contact surface parameters. This hierarchical planning mechanism ensures the accuracy and reliability of pose planning.

[0085] Robot control interface module

[0086] This module converts optimized pose commands into executable motion trajectories for the robot and monitors pose deviations in real time. Specifically, through a robot control interface module (such as a ROS node or robot control software), pose commands are converted into motion trajectory commands for the robotic arm and sent to the robot control system. Simultaneously, sensors (such as force sensors or vision sensors) monitor the robotic arm's movement in real time, detect pose deviations, and adjust the arm's actions promptly to ensure precise execution of the stacking operation.

[0087] Precision positioning unit

[0088] This unit comprises a virtual modeling component, a spatial matching engine, and an optimization calculation component. The virtual modeling component generates a deformable virtual container model based on the target cargo parameters. Specifically, it generates a virtual container model with elastic boundaries in the computer based on the cargo's size, shape, and weight parameters. The spatial matching engine aligns the virtual container model with real-time environmental data in multiple dimensions. Specifically, it aligns the virtual container model with candidate placement areas in the 3D environment topology map using collision detection and spatial transformation algorithms. The optimization calculation component iteratively optimizes the contact surface parameters using physics-based simulation algorithms, outputting the final pose coordinates that satisfy mechanical constraints. Specifically, it adjusts the position and orientation of the virtual container using optimization algorithms such as gradient descent or genetic algorithms to generate final pose coordinates that meet preset stability evaluation indicators. This optimization process ensures the stability and safety of cargo placement.

[0089] Visual verification unit

[0090] This unit comprises an obstacle detection submodule, a stability prediction submodule, and a reachability analysis submodule. The obstacle detection submodule identifies dynamic obstacles in the environmental topology map based on a deep learning model. Specifically, it trains the obstacle detection model using deep learning algorithms (such as convolutional neural networks) to identify dynamic obstacles in the environmental topology map. The stability prediction submodule calculates the static stability coefficients of candidate poses using finite element analysis. Specifically, it calculates the static stability coefficients of candidate poses using a finite element analysis model (such as ANSYS or ABAQUS). The reachability analysis submodule verifies the reachability space range of the target pose using the robot's DH parameter model. Specifically, it uses the robot's DH parameter model to determine whether the target pose is within the working range of the robotic arm and whether it meets the reachability requirements. This multi-dimensional verification mechanism ensures the feasibility and reliability of pose planning.

[0091] As is known from common technical knowledge, this invention can be implemented through other embodiments that do not depart from its spirit or essential characteristics. Therefore, the disclosed embodiments described above are merely illustrative in all respects and are not the only ones. All modifications within the scope of this invention or its equivalents are included in this invention.

Claims

1. A multi-camera vision-based stacking pose planning method for intelligent loading and unloading robots, characterized in that, Includes the following steps: A collaborative sensing system is built by deploying multiple industrial cameras in the target space to collect dynamic environmental data of the stacking area in real time. The construction of the collaborative perception system includes: real-time fusion of depth data and RGB data collected by multiple cameras to generate a three-dimensional environmental topology map that includes space occupancy status, obstacle distribution and stack surface morphology; The adaptive stacking pose is generated based on a hierarchical pose planning mechanism, including: parsing initial pose parameters and generating preliminary placement instructions through a topic control layer; verifying the feasibility of the preliminary placement instructions based on environmental data from the collaborative perception system through a visual verification layer; and optimizing the final placement coordinates through a precise positioning layer combined with a virtual box matching algorithm. The optimization process of the virtual box matching algorithm includes: constructing a virtual box model with elastic boundaries based on the physical size parameters of the target goods. The virtual container matching algorithm includes: constructing a virtual container model that matches the size of the target cargo; performing three-dimensional matching between the virtual container model and the actual spatial data collected by the collaborative sensing system; optimizing the contact surface parameters between the virtual container and the actual stacking area using the gradient descent method; and generating a final pose command that meets the preset safety margin to ensure the stability and safety of cargo placement.

2. The multi-camera vision stacking pose planning method for intelligent loading and unloading robots according to claim 1, characterized in that, The construction of the collaborative sensing system includes: At least three industrial cameras are deployed above, to the sides and bottom of the stacking area, and each camera achieves frame-level data synchronization through a time synchronization protocol; Establish a spatial coordinate system transformation model between cameras, and eliminate blind spots in multi-view data acquisition through feature point matching algorithms; Real-time fusion of depth data and RGB data collected by multiple cameras generates a 3D environmental topology map that includes space occupancy status, obstacle distribution, and stack surface morphology.

3. The multi-camera vision stacking pose planning method for intelligent loading and unloading robots according to claim 1, characterized in that, The feasibility verification of the visual verification layer includes: Based on the three-dimensional environment topology map, the space occupancy status of the target area corresponding to the initial placement command is detected; Calculate the contact area ratio between the virtual box model and the surface topography of the stack, and mark the pose as infeasible when the ratio is lower than a preset threshold; Assess the reachability of the robotic arm to the target pose and determine whether there is a risk of joint over-limit based on the robot's workspace model.

4. The multi-camera vision stacking pose planning method for intelligent loading and unloading robots according to claim 1, characterized in that, The optimization process of the virtual box matching algorithm includes: Construct a virtual container model with flexible boundaries based on the physical dimensions of the target cargo; The virtual box model is projected onto the candidate placement area of ​​the three-dimensional environment topology map, and a collision detection algorithm is used to filter the candidate pose set that meets the minimum safe distance. The pose with the smallest angle between the contact surface normal vector and the gravity direction is selected from the candidate pose set as the optimization benchmark. The rotation angle and translation of the virtual box are iteratively adjusted until the preset stability evaluation index is met.

5. The multi-camera vision stacking pose planning method for intelligent loading and unloading robots according to claim 4, characterized in that, The calculation of the stability evaluation index includes: Obtain the normal vector distribution of the contact points on the stacked surface and calculate its offset from the projection of the virtual box's center of gravity. Based on the contact point pressure distribution model, predict the slip probability of the virtual box under dynamic load; A stability prediction model is trained by combining historical stacking data, and the output is a comprehensive evaluation parameter that includes static stability coefficient and dynamic disturbance resistance capability.

6. The multi-camera vision stacking pose planning method for intelligent loading and unloading robots according to claim 1, characterized in that, The initial pose generation of the topic control layer includes: Receive cargo stacking order instructions from the upstream scheduling system and parse the size, weight, and stacking constraints of the target cargo; A preliminary stacking layout scheme is generated based on the close packing algorithm, and the theoretical placement coordinates of each item in the stacking coordinate system are calculated. Based on the kinematic model of the robotic arm, inverse kinematics calculations are performed on the theoretical placement coordinates to generate an initial pose command set containing joint angle sequences.

7. The multi-camera vision stacking pose planning method for intelligent loading and unloading robots according to claim 1, characterized in that, The method also includes an exception handling mechanism: When the visual verification layer detects that the pose is not feasible, it triggers the dynamic replanning module to regenerate candidate poses based on the current environment topology graph. Establish an abnormal pose backtracking database to record environmental feature parameters that cause pose failure and optimize the constraints of subsequent pose planning. When the number of consecutive replanning attempts exceeds a preset threshold, a manual intervention protocol is initiated and environmental reconstruction suggestions are output.

8. A multi-camera vision stacking pose planning system for intelligent loading and unloading robots, based on the multi-camera vision stacking pose planning method for intelligent loading and unloading robots according to any one of claims 1-7, characterized in that, include: The multi-camera collaborative sensing module includes multiple spatially deployed industrial cameras, a data synchronization unit, and a 3D reconstruction engine, used to generate dynamic environmental topology maps in real time. The hierarchical pose planning module includes a theme control unit, a visual verification unit, and a precise positioning unit. The theme control unit is used to generate initial pose commands, the visual verification unit is used to perform feasibility verification based on the environmental topology map, and the precise positioning unit is used to execute a virtual box matching algorithm. The robot control interface module is used to convert optimized pose commands into executable motion trajectories for the robot and to monitor pose deviations in real time during execution.

9. The intelligent loading and unloading robot multi-camera vision stacking pose planning system according to claim 8, characterized in that, The precise positioning unit includes: Virtual modeling component, used to generate deformable virtual container models based on target cargo parameters; A spatial matching engine is used to align the virtual container model with real-time environmental data in multiple dimensions. The computational component is optimized to iteratively optimize the contact surface parameters using a physics-based simulation algorithm, and output the final pose coordinates that satisfy the mechanical constraints.

10. The intelligent loading and unloading robot multi-camera vision stacking pose planning system according to claim 8, characterized in that, The visual verification unit includes: The obstacle detection submodule is used to identify dynamic obstacles in the environmental topology map based on a deep learning model. The stability prediction submodule is used to calculate the static stability coefficient of candidate poses through finite element analysis. The reachability analysis submodule is used to verify the reachability space range of the target pose by combining the robot's DH parameter model.

Images