Automatic unloading and stacking method based on computer vision and robot control

Abstract

The invention discloses an automatic unloading and stacking method based on computer vision and robot control, and belongs to the technical field of intelligent logistics automation. According to the method, a depth camera is innovatively installed on a first rotating joint of a mechanical arm of the unloading and stacking robot, and a hard trigger collecting mechanism is combined, so that non-stop visual guidance is achieved. In the unloading link, the pose and size of an unknown carton are directly calculated by adopting a template-free instance segmentation and point cloud geometric fitting method, and global safety planning is carried out by utilizing DBSCAN spatial clustering. In the stacking link, seamless monitoring is achieved through back-to-back double cameras, and a stacking scheme is dynamically generated in combination with an adaptive algorithm. According to the method, the problems of dependence on templates and markers, insufficient operation safety, slow system rhythm and the like in the prior art are effectively solved, and efficient, safe and universal full-process automatic operation is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of industrial automation and control technology, in particular to an automatic unloading and stacking method based on computer vision and robot control. BACKGROUND

[0002] With the rapid development of intelligent logistics and manufacturing industry, automated loading and unloading and stacking systems have become a key link to improve the efficiency of warehousing and logistics. Industrial robots combined with machine vision are the core technical means to achieve automation. Currently, the technical solutions in this field mainly rely on two methods: visual guidance based on pre-stored templates and visual positioning based on artificial markers.

[0003] The visual guidance method based on pre-stored templates (such as prior art CN202211287436.4) requires pre-scanning and storing two-dimensional images and three-dimensional point cloud templates of each face of the paper box to be processed. In actual operation, the system needs to real-time register and match the point cloud collected on site with all pre-stored templates to identify the type of paper box and calculate the pose. This method has obvious limitations: first, it has poor universality and cannot handle "non-standard" paper boxes whose size is unknown and have not been pre-recorded into the system, greatly limiting its application in diversified logistics scenarios; second, the calculation complexity of three-dimensional point cloud registration is high, which seriously affects the real-time performance of the system when the number of paper boxes in the stack is large, resulting in a decrease in operation cycle; finally, it uses a simple local comparison strategy for grasping sequence planning, which lacks global analysis of the overall stack structure, and is prone to collision risks during grasping, lacking safety and intelligence.

[0004] Another type of visual positioning method based on artificial markers (such as prior art CN202510213785.9) requires sticking barcodes or other markers on each cargo or pallet. The system indirectly determines the position and attitude of the cargo by recognizing and positioning these pre-set markers. The defects of this method are: first, it introduces additional material costs (printing, sticking and maintenance of markers) and operational complexity, which is not ideal in terms of economy and efficiency; second, the technical path itself lacks universality and cannot be applied to a large number of ordinary paper boxes without markers, making it difficult to become a universal solution.

[0005] In summary, existing technologies all have inherent defects in terms of universality, real-time performance, safety or economy. Therefore, there is an urgent need in the field for an automated method that can break free from the dependence on prior templates and artificial markers, and can intelligently, safely and efficiently handle paper boxes of any size, and achieve seamless connection of the entire unloading and stacking process. SUMMARY

[0006] Therefore, it is necessary to provide an automated unloading and stacking method based on computer vision and robot control to at least solve the inherent defects of generality, real-time performance, safety or economy in the related art.

[0007] In a first aspect, the embodiments of the present application provide an automated unloading and stacking method based on computer vision and robot control, comprising the following steps: S1, controlling an unloading robot to move its mechanical arm so that a first depth camera fixed on a first rotary joint of the mechanical arm reaches a preset global observation point; S2, at the moment of reaching the global observation point, controlling the first depth camera to collect an RGB image and a depth image of a carton stack in the cargo compartment through a hard trigger signal; S3, based on the RGB image and the depth image, calculating the grasping pose and the top surface size of each carton in the cargo compartment in the base coordinate system of the unloading robot through a template-free pose estimation method; S4, based on the grasping pose of all cartons, planning a globally safe grasping sequence through spatial clustering analysis; S5, controlling the unloading robot to grasp the cartons one by one and transfer them to a conveyor belt according to the grasping sequence; S6, controlling a stacking robot to move, and in the moving process, when the first rotary joint of the mechanical arm moves to a first trigger point corresponding to a suction station, controlling a second depth camera fixed on the joint and facing the tray area to collect an RGB-D image of the tray area through a hard trigger signal; when moving to a second trigger point corresponding to a stacking station, controlling a third depth camera fixed on the joint and facing the conveyor belt to collect an RGB-D image of the conveyor belt area through a hard trigger signal; S7, based on the RGB-D image of the conveyor belt area, identifying the pose and complete size of the carton to be grasped; S8, based on the RGB-D image of the tray area, judging the current stacking state of the tray and adaptively calculating the optimal stacking position of the current carton; S9, controlling the stacking robot to grasp the carton to be grasped and stack it according to the optimal stacking position.

[0008] In one of the embodiments, the template-free pose estimation method in step S3 comprises: S31, using an instance segmentation neural network model to process the RGB image to obtain a pixel-level mask of each carton; S32, using the pixel-level mask of each carton to extract corresponding instance point cloud data from the depth image, the instance point cloud being located in the camera coordinate system of the first depth camera; S33, converting each of the instance point clouds from the camera coordinate system to the robot base coordinate system using the pre-calibrated hand-eye matrix and the real-time feedback of the forward kinematics pose of the robot; S34, for each of the converted instance point clouds of the cartons, robustly fitting a plane as the top surface of the carton using the RANSAC algorithm; S35, projecting the instance point cloud of the carton onto the fitted top surface plane; S36, calculating the minimum enclosing rotated rectangle of the projected point cloud in the top surface plane; S37, taking the geometric center point of the minimum enclosing rotated rectangle as the grasping position, taking the normal vector of the top surface plane as the normal direction of the grasping pose, and taking the length and width of the minimum enclosing rotated rectangle as the length and width of the top surface of the carton.

[0009] In one of the embodiments, the step S4 of planning the globally safe grasping sequence through spatial clustering analysis includes: S41, extracting the coordinate components of all carton grasping positions in the first horizontal axis pointing to the depth direction of the cargo compartment; S42, performing clustering analysis on the set of coordinate components in the first horizontal axis using the DBSCAN clustering algorithm; S43, from the clustering results, selecting a cluster with the smallest number of coordinate components, determining the carton corresponding to this cluster as the outermost carton, and only including the outermost carton in the current cycle of the grasping set.

[0010] In one of the embodiments, after determining the grasping set, the step S4 further includes: S44, extracting the coordinate components of all carton grasping positions in the vertical direction in the grasping set; S45, performing clustering analysis on the set of coordinate components in the vertical direction using the DBSCAN clustering algorithm to distinguish different stacking layers; S46, arranging the layers obtained by clustering in descending order of their average height values to determine the order of grasping from high to low; S47, for cartons belonging to the same layer, extracting the coordinate components of their grasping positions in the second horizontal axis perpendicular to the first horizontal axis; S48, for cartons in the same layer, using different sorting rules to sort their coordinate components in the second horizontal axis according to the parity of the layer number of the layer they belong to: if the layer number is odd, sorting in ascending order; if the layer number is even, sorting in descending order, thereby forming an S-shaped traversal path.

[0011] In one of the embodiments, the step S7 of identifying the complete size of the paper box to be grabbed comprises: S71, based on the RGB-D image of the conveyor belt area, the pose of the top surface of the paper box in the base coordinate system of the stacking robot and the length and width of the top surface are identified by the template-free pose estimation method. S72, based on the depth image of the conveyor belt area, a three-dimensional plane equation of the conveyor belt plane in the base coordinate system of the stacking robot is fitted. S73, the vertical distance between the three-dimensional coordinates of the center point of the top surface of the paper box and the conveyor belt plane is calculated, and the distance is taken as the complete height value of the paper box.

[0012] In one of the embodiments, the step S8 of adaptive stacking position calculation comprises: S81, instance segmentation and point cloud processing are performed on the RGB-D image of the tray area to determine whether there is a paper box on the tray. S82, if it is determined that the tray is empty, the following sub-steps are performed: S821, the boundary of the tray is identified to determine the available stacking area. S822, according to the size of the paper box to be stacked and the size of the available stacking area, the maximum number of paper boxes that can be accommodated in the length, width and height directions of the tray is dynamically calculated to generate an initial stacking scheme. S823, according to the stacking scheme, the position coordinates and orientation of the first paper box are determined. S83, if it is determined that there is a paper box on the tray, the following sub-steps are performed: S831, the pose and height of the stacked paper box on the tray are identified and located. S832, according to the layout of the stacked paper box and the size of the paper box to be stacked, the remaining space of the current stack is calculated. S833, based on the stability and space utilization principles, the optimal placement position of the current paper box in the remaining space is determined.

[0013] In one of the embodiments, the hard trigger signal in steps S2 and S6 is generated and sent to the external trigger pin of the corresponding depth camera by the digital I / O interface of the robot controller when the real-time angle or spatial pose of the first rotary joint of the robot is detected to be completely consistent with the preset trigger condition, so as to drive the camera to complete image acquisition at the precise moment of the robot motion.

[0014] In a second aspect, the embodiments of the present application also provide an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to the first aspect when executing the program.

[0015] In a third aspect, the embodiments of the present application further provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method in the first aspect. Advantages

[0016] Compared with the prior art, the present application has the following advantages: 1. Achieving extremely high versatility and adaptability: The "template-free pose estimation method" (as claimed in claim 2) pioneered by the present application directly fits the instance segmentation, point cloud geometric features (RANSAC plane fitting and minimum bounding rectangle calculation) to analyze the carton pose and size in real time, completely abandoning the dependence on pre-stored template library or any artificial markers. This enables the present application to directly process complex and diverse "non-standard" cartons with unknown sizes in the logistics scene, greatly widening the application range of the system, while significantly reducing the deployment and maintenance costs of the system.

[0017] 2. Improving the overall operation efficiency and beat of the system: The present application realizes the maximum parallelism of visual perception and robot motion through the unique "eye-on-one-axis" hardware layout and "hard trigger" acquisition mechanism (as claimed in claims 1 and 7). The robot does not need to pause for waiting for the photograph, realizing "non-stop visual guidance" of "acquisition in motion, operation upon arrival". In addition, efficient geometric algorithms replace time-consuming point cloud registration, and the S-shaped path planning (as claimed in claim 4) optimizes the mechanical arm motion trajectory, which together ensures the efficiency and smoothness of the whole process from unloading to stacking, and greatly improves the overall beat of the system.

[0018] 3. Ensuring high safety and intelligent planning in the operation process: The present application introduces a global grasping sequence planning algorithm based on DBSCAN spatial clustering (as claimed in claims 3 and 4). This method can intelligently analyze the stack from a global perspective, clearly distinguish different carton rows and layers through clustering, and forcibly plan the system to grasp only the outermost cartons, thereby fundamentally eliminating the collision risk caused by the mechanical arm penetrating into the cargo stack for grasping. This planning strategy based on global spatial relationship is much safer and more intelligent than the simple local comparison logic in the prior art.

[0019] 4. Achieving full adaptability and intelligentization in the stacking process: In the stacking process, the "seamless monitoring" of the conveyor belt and the pallet state is realized through the back-to-back dual-camera layout (as claimed in claims 1, 5 and 6), and combined with the adaptive stacking planning algorithm, the optimal stacking scheme can be dynamically calculated and updated according to the real-time perceived carton size and the remaining space of the pallet. This method realizes fully automatic, high space utilization stacking operation without any artificial preset and intervention, with high intelligentization.

[0020] 5. Formed a complete automatic closed loop, optimized the system integration: the invention integrates the two originally independent links of unloading and stacking into a coherent, adaptive automatic closed loop system through innovative visual guidance method and control process. This method covers the whole process from identifying and grabbing in the cargo compartment to accurately stacking on the pallet. The links are closely connected, the overall system integration is high, and the automation level is significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The work flow chart of the unloading and stacking of the present invention; Figure 2 The installation schematic diagram of the camera of the unloading mechanical arm of the present invention; Figure 3 The installation schematic diagram of the camera of the stacking mechanical arm of the present invention; Figure 4 The unloading visual processing work flow chart of the present invention; Figure 5 The stacking visual processing work flow chart of the present invention. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be clearly and completely described in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0023] It should be noted that when a component is referred to as "mounted on" another component, it can be directly mounted on the other component or there can be a middle component. When a component is referred to as "disposed on" another component, it can be directly disposed on the other component or there can be a middle component. When a component is referred to as "fixed on" another component, it can be directly fixed on the other component or there can be a middle component.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0025] As shown in Figures 1 to 5 An automatic unloading and stacking method based on computer vision and robot control, comprising the following steps: S1, control the unloading robot to move its mechanical arm, so that the first depth camera fixed on the first rotary joint of the mechanical arm reaches a preset global observation point; S2, at the moment of reaching the global observation point, control the first depth camera to collect the RGB image and the depth image of the carton stack in the cargo compartment through a hard trigger signal; S3, based on the RGB image and the depth image, calculate the grasping pose and the top surface size of each carton in the cargo compartment in the unloading robot base coordinate system through a template-free pose estimation method; S4, based on the grasping pose of all cartons, plan a globally safe grasping sequence through spatial clustering analysis; S5, control the unloading robot to grasp the cartons in sequence and move them to the conveyor belt according to the grasping sequence; S6, control the stacking robot to move, and in the moving process, when the first rotary joint of the mechanical arm moves to the first trigger point corresponding to the suction station, control the second depth camera fixed on the joint and facing the tray area to collect the RGB-D image of the tray area through a hard trigger signal; when moving to the second trigger point corresponding to the stacking station, control the third depth camera fixed on the joint and facing the conveyor belt to collect the RGB-D image of the conveyor belt area through a hard trigger signal; S7, based on the RGB-D image of the conveyor belt area, identify the pose and complete size of the carton to be grasped; S8, based on the RGB-D image of the tray area, judge the current stacking state of the tray, and adaptively calculate the optimal stacking position of the current carton; S9, control the stacking robot to grasp the carton to be grasped and stack according to the optimal stacking position.

[0026] In some optional embodiments, the template-free pose estimation method in step S3 includes: S31, use an instance segmentation neural network model to process the RGB image to obtain a pixel-level mask of each carton; S32, use the pixel-level mask of each carton to extract corresponding instance point cloud data from the depth image, the instance point cloud being in the camera coordinate system of the first depth camera; S33, use the pre-calibrated hand-eye matrix and the real-time feedback of the robot forward kinematics pose to convert each instance point cloud from the camera coordinate system to the unloading robot base coordinate system; S34, for each converted carton instance point cloud, robustly fit a plane as the top surface of the carton using the RANSAC algorithm; S35, project the instance point cloud of the carton onto the fitted top surface plane; S36, calculate the minimum enclosing rotated rectangle of the projected point cloud in the top plane; S37, take the geometric center point of the minimum enclosing rotated rectangle as the grasping position, take the normal vector of the top plane as the normal direction of the grasping pose, and take the length and width of the minimum enclosing rotated rectangle as the length and width of the top surface of the carton.

[0027] In some optional embodiments, the planning of the globally safe grasping sequence in step S4 through spatial clustering analysis includes: S41, extract the coordinate component of all carton grasping positions in the first horizontal axis pointing to the depth direction of the cargo compartment; S42, perform clustering analysis on the set of coordinate components in the first horizontal axis using the DBSCAN clustering algorithm; S43, from the clustering results, select a cluster with the smallest coordinate component value, determine the carton corresponding to the cluster as the outermost carton, and only include the outermost carton in the current cycle of the to-be-grasped set.

[0028] In one embodiment, after determining the to-be-grasped set, step S4 further includes: S44, extract the coordinate component of all carton grasping positions in the to-be-grasped set in the vertical direction; S45, perform clustering analysis on the set of coordinate components in the vertical direction using the DBSCAN clustering algorithm to distinguish different stacked layers; S46, arrange the layers obtained by clustering in descending order of their average height values to determine the order of grasping from high to low; S47, for cartons belonging to the same layer, extract the coordinate component of their grasping positions in the second horizontal axis perpendicular to the first horizontal axis; S48, for cartons in the same layer, use different sorting rules to sort their coordinate components in the second horizontal axis according to the parity of the layer number of the layer they belong to: if the layer number is odd, arrange in ascending order; if the layer number is even, arrange in descending order, thereby forming an S-shaped traversal path.

[0029] In some optional embodiments, the identification of the complete size of the to-be-grasped carton in step S7 includes: S71, based on the RGB-D image of the conveyor belt area, identify the pose of the carton top surface in the base coordinate system of the stacking robot and the length and width of the top surface through a template-free pose estimation method; S72, based on the depth image of the conveyor belt area, fit a three-dimensional plane equation of the conveyor belt plane in the base coordinate system of the stacking robot; S73, calculate the vertical distance between the three-dimensional coordinates of the center point of the top surface of the carton and the plane of the conveying belt, and take the distance as the complete height value of the carton.

[0030] In some optional embodiments, the adaptive stacking position calculation in step S8 includes: S81, instance segmentation and point cloud processing are performed on the RGB-D image of the tray area to determine whether a carton already exists on the tray; S82, if it is determined that the tray is empty, the following sub-steps are performed: S821, the boundary of the tray is recognized to determine the available stacking area; S822, according to the size of the carton to be stacked and the size of the available stacking area, the maximum number of cartons that can be accommodated in the length, width and height directions of the tray is dynamically calculated, and an initial stacking scheme is generated; S823, according to the stacking scheme, the position coordinates and orientation of the first carton to be placed are determined; S83, if it is determined that a carton already exists on the tray, the following sub-steps are performed: S831, the pose and height of the stacked carton on the tray are recognized and located; S832, according to the layout of the stacked carton and the size of the carton to be stacked, the remaining space of the current stack is calculated; S833, based on the stability and space utilization principles, the optimal placement position of the current carton in the remaining space is determined.

[0031] In some optional embodiments, the hard trigger signal in steps S2 and S6 is generated and sent to the external trigger pin of the corresponding depth camera by the digital I / O interface of the robot controller when the real-time angle or spatial pose of the first rotating joint of the robot arm is detected to be completely consistent with the preset trigger condition, so as to drive the camera to complete image acquisition at the precise moment of robot movement.

[0032] In a second aspect, the embodiments of the present application also provide an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method of the first aspect when executing the program.

[0033] In a third aspect, the embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the steps of the method of the first aspect.

[0034] It will be obvious to a person skilled in the art that the application is not limited to the details of the foregoing exemplary embodiments and can be implemented in other concrete forms without departing from the spirit or essential characteristics of the application. The embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. No reference signs in the claims should be considered as limiting the scope of the claims to the identity of the reference signs therein.

[0035] Furthermore, it should be understood that although the description is made on the basis of the embodiments, not every embodiment contains only one independent technical solution, and the description of the specification is only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that those skilled in the art can understand.

Claims

1. An automated unloading and destacking method based on computer vision and robot control, characterized in that, The method comprises the following steps: S1, controlling the unloading robot to move its mechanical arm so that a first depth camera fixed on a first rotary joint of the mechanical arm reaches a preset global observation point; S2, at the moment of reaching the global observation point, controlling the first depth camera to collect an RGB image and a depth image of the carton stack in the cargo compartment through a hard trigger signal; S3, based on the RGB image and the depth image, calculating the grasping pose and the top surface size of each carton in the cargo compartment in the unloading robot base coordinate system through a template-free pose estimation method; S4, based on the grasping pose of all cartons, planning a globally safe grasping sequence through spatial clustering analysis; S5, controlling the unloading robot to grasp the cartons one by one according to the grasping sequence and move them to the conveyor belt; S6, controlling the stacking robot to move, and in the moving process, when the first rotary joint of the mechanical arm moves to a first trigger point corresponding to the suction station, controlling a second depth camera fixed on the joint and facing the tray area to collect an RGB-D image of the tray area through a hard trigger signal; when moving to a second trigger point corresponding to the stacking station, controlling a third depth camera fixed on the joint and facing the conveyor belt to collect an RGB-D image of the conveyor belt area through a hard trigger signal; S7, based on the RGB-D image of the conveyor belt area, identifying the pose and complete size of the carton to be grasped; S8, based on the RGB-D image of the tray area, judging the current stacking state of the tray, and adaptively calculating the optimal stacking position of the current carton; S9, controlling the stacking robot to grasp the carton to be grasped according to the optimal stacking position and stack it.

2. The automated unloading and depiling method according to claim 1, wherein, The template-free pose estimation method in step S3 comprises: S31, using an instance segmentation neural network model to process the RGB image to obtain a pixel-level mask of each carton; S32, using the pixel-level mask of each carton to extract corresponding instance point cloud data from the depth image, the instance point cloud being located in the camera coordinate system of the first depth camera; S33, using a pre-calibrated hand-eye matrix and a real-time feedback forward kinematics pose of the robot to convert each instance point cloud from the camera coordinate system to the unloading robot base coordinate system; S34, for each converted carton instance point cloud, robustly fitting a plane as the top surface of the carton using the RANSAC algorithm; S35, projecting the instance point cloud of the carton onto the fitted top surface plane; S36, calculating the minimum enclosing rectangle of the projected point cloud in the top surface plane; S37, taking the geometric center point of the minimum enclosing rectangle as the grasping position, taking the normal vector of the top surface plane as the normal direction of the grasping pose, and taking the length and width of the minimum enclosing rectangle as the length and width of the top surface of the carton.

3. The automated unloading and depiling method according to claim 1, wherein, The planning of the globally safe grasping sequence through spatial clustering analysis in step S4 comprises: S41, extracting the coordinate component of all carton grasping positions in the first horizontal axis pointing to the depth direction of the cargo compartment; S42, the DBSCAN clustering algorithm is used to cluster the coordinate component set in the first horizontal axis; S43, from the clustering results, the cluster with the smallest coordinate component value is selected, and the carton corresponding to the cluster is determined as the outermost carton, and only the outermost carton is included in the current cycle of the to-be-grabbed set.

4. The automated unloading and depiling method according to claim 3, wherein, After determining the to-be-grabbed set, the step S4 further includes: S44, extracting the coordinate component of all carton grabbing positions in the to-be-grabbed set in the vertical direction; S45, using the DBSCAN clustering algorithm to cluster the coordinate component set in the vertical direction to distinguish different stacking layers; S46, arranging the layers obtained by clustering in descending order according to their average height values to determine the order of grabbing from high to low; S47, for the cartons belonging to the same layer, the coordinate component of the grabbing position in the second horizontal axis perpendicular to the first horizontal axis is extracted; S48, for the cartons in the same layer, different sorting rules are used to sort the coordinate component in the second horizontal axis according to the layer number parity of the layer where the carton is located: if the layer number is odd, it is arranged in ascending order; if the layer number is even, it is arranged in descending order, thereby forming an S-shaped traversal path.

5. The automated unloading and depiling method according to claim 2, wherein, The complete size of the to-be-grabbed carton in the step S7 includes: S71, based on the RGB-D image of the conveyor belt region, the pose of the carton top surface in the base coordinate system of the stacking robot and the length and width of the top surface are identified by the pose estimation method without template; S72, based on the depth image of the conveyor belt region, the three-dimensional plane equation of the conveyor belt plane in the base coordinate system of the stacking robot is fitted; S73, the vertical distance between the three-dimensional coordinates of the center point of the carton top surface and the conveyor belt plane is calculated, and the distance is taken as the complete height value of the carton.

6. The automated unloading and depiling method according to claim 1, wherein, The adaptive stacking position calculation in the step S8 includes: S81, performing instance segmentation and point cloud processing on the RGB-D image of the tray region to determine whether there is a carton on the tray; S82, if it is determined that the tray is empty, the following sub-steps are performed: S821, identifying the boundary of the tray to determine the available stacking area; S822, dynamically calculating the maximum number of cartons that can be accommodated in the length, width and height directions of the tray according to the size of the to-be-stacked carton and the size of the available stacking area, and generating an initial stacking scheme; S823, determining the position coordinates and orientation of the first carton according to the stacking scheme; S83, if it is determined that there is a carton on the tray, the following sub-steps are performed: S831, identifying and positioning the pose and height of the stacked carton on the tray; S832, calculating the remaining space of the current stack according to the layout of the stacked carton and the size of the to-be-stacked carton; S833, determining the best placement position of the current carton in the remaining space based on the stability and space utilization principles.

7. The automated unloading and depiling method according to claim 1, wherein, The hard trigger signal in the steps S2 and S6 is generated and sent to the external trigger pin of the corresponding depth camera by the digital I / O interface of the robot controller when the real-time angle or spatial pose of the first rotary joint of the mechanical arm is detected to be completely consistent with the preset trigger condition, so as to drive the camera to complete image acquisition at the accurate time during the movement of the robot.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the method of any one of claims 1 to 7 when executing the program.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images