Stacked material recognition method and system for a de-palletizing robot

Abstract

The application relates to the technical field of robots, and provides a de-stacking robot material stack identification method and system, which comprises the following steps: acquiring target depth information of each sampling point in a grabbing surface of a material stack; determining a flatness index of each sampling point in the grabbing surface based on the target depth information of the plurality of sampling points in the grabbing surface; the flatness index is used for indicating the flatness degree of the sampling point; determining a target grabbing area in the grabbing surface according to the flatness indexes of the plurality of sampling points in the grabbing surface; generating and sending grabbing information to a material grabbing device; the grabbing information is used for instructing the material grabbing device to grab the target grabbing area through a grabbing part. The efficiency of grabbing the material can be improved when flexible packaging materials are processed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a method and system for identifying stacked materials by a de-palletizing robot. BACKGROUND

[0002] In automated warehousing and logistics sorting operations, de-palletizing robots identify stacked materials through a vision system, and then perform picking and placing tasks. Traditional identification methods perform very high efficiency and accuracy when dealing with single categories, regular sizes, and neatly stacked rigid goods. However, as the logistics industry continues to demand higher utilization of warehouse space and transportation efficiency, and as pressure to control packaging material costs increases, more and more goods are beginning to use flexible packaging materials instead of traditional rigid cartons. These flexible packages include, but are not limited to, plastic bags, soft packaging bags, bunched items wrapped in shrink film, textile bags, and various irregularly shaped soft containers.

[0003] Due to the inherent plasticity of flexible packaging, when these materials are stacked on a pallet, they are difficult to form a flat, regular stack. Instead, they will squeeze each other, fill the gaps, and cause the entire stack surface to exhibit a complex topography of bumps and depressions. The weight of the upper materials will further exert pressure on the lower materials, causing the deformation of the lower materials to be more significant. For example, a stack of soft packaging bags may no longer have a flat top surface, but rather a top surface composed of multiple raised and depressed areas. Individual materials themselves may also exhibit local bulging, collapsing, or twisting, and their shapes become highly uncertain in the stack. This irregular stacking and continuous deformation make the external contours and dimensions of the materials no longer fixed, but rather dynamically changing with the stacking method and stress conditions.

[0004] Therefore, it becomes difficult for the robot to plan an effective picking path and picking strategy, greatly increasing the risk of picking failure, resulting in a significant increase in identification failure rate for de-palletizing robots when handling flexible packaging materials, severely affecting the efficiency and continuity of de-palletizing. SUMMARY

[0005] The present application provides a method and system for identifying stacked materials by a de-palletizing robot, which can improve the efficiency of picking materials when handling flexible packaging materials.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] In a first aspect, a method for identifying a stack of materials by a de-stacking robot is provided. The method comprises: obtaining target depth information of each sampling point in a grabbing surface of the stack of materials; determining a flatness index of each sampling point in the grabbing surface based on the target depth information of the plurality of sampling points in the grabbing surface; the flatness index is used to indicate a flatness of the sampling point; determining a target grabbing area in the grabbing surface according to the flatness index of the plurality of sampling points in the grabbing surface; generating and sending grabbing information to a material grabbing device; the grabbing information is used to instruct the material grabbing device to grab the target grabbing area by a grabbing part.

[0008] Optionally, the determining of the flatness index of each sampling point in the grabbing surface based on the target depth information of the plurality of sampling points in the grabbing surface comprises: for each sampling point in the plurality of sampling points in the grabbing surface, determining a fitting plane corresponding to the plurality of sampling points in a first area according to the target depth information of the plurality of sampling points in the first area, the sampling point being located at the center of the first area; and determining the flatness index of the sampling point according to the fitting plane and the target depth information of the sampling point in the first area.

[0009] Optionally, the determining of the flatness index of the sampling point according to the fitting plane and the target depth information of the sampling point in the first area comprises: determining a distance between each sampling point in the first area and the fitting plane; and determining the reciprocal of the distance as the flatness index of the sampling point.

[0010] Optionally, the determining of the target grabbing area in the grabbing surface according to the flatness index of the plurality of sampling points in the grabbing surface comprises: determining a first grabbing area in the grabbing surface according to the flatness index of the plurality of sampling points in the grabbing surface; the flatness index of each sampling point in the first grabbing area being greater than a first flatness index threshold; and determining a grabbing area with an area greater than a preset area threshold in the first grabbing area as the target grabbing area.

[0011] Optionally, the determining of the target grabbing area in the grabbing surface according to the flatness index of the plurality of sampling points in the grabbing surface comprises: determining a second grabbing area in the first grabbing area with an area greater than the preset area threshold; and determining a grabbing area with an angle between a normal vector of the grabbing area and a vertical direction being less than a preset angle threshold in the second grabbing area as the target grabbing area.

[0012] Optionally, the grabbing part is a suction cup, and the determining of the target grabbing area in the grabbing surface according to the flatness index of the plurality of sampling points in the grabbing surface comprises: determining a third grabbing area in the second grabbing area with the angle between the normal vector of the grabbing area and the vertical direction being less than the preset angle threshold; obtaining a gas permeability index of the third grabbing area; the gas permeability index is used to indicate a gas permeability of the third grabbing area; and determining a grabbing area with the gas permeability index being less than a preset gas permeability index threshold in the third grabbing area as the target grabbing area.

[0013] Optionally, the air permeability index of the third grabbing area is obtained by: sending first information to the material grabbing device, the first information being used to instruct the material grabbing device to adsorb the suction cup to the second grabbing area with a preset pressure, and instruct the material grabbing device to detect a variation of the pressure in the suction cup within a preset time period by the pressure detection device; receiving second information of the material grabbing device including the variation, and taking the variation as the air permeability index of the third grabbing area.

[0014] Optionally, the target depth information of each sampling point in the grabbing surface of the material stack is obtained by: obtaining initial depth information of each sampling point in the grabbing surface of the material stack; for each sampling point in the plurality of sampling points of the grabbing surface, determining a ratio of a number of first sampling points to a number of second sampling points in a first area, the sampling point being located at the center of the first area, the initial depth information of the first sampling point being within a preset depth range, and the initial depth information of the second sampling point being outside the preset depth range; determining a variance of the initial depth information of the first sampling point in the first area; determining a quality index of the initial depth information of the sampling point according to the ratio and the variance; and in a case where the quality index is greater than a preset quality index threshold, determining the initial depth information of the sampling point as the target depth information of the sampling point.

[0015] Optionally, the quality index of the initial depth information of the sampling point is determined according to the ratio and the variance by: determining a first product of the ratio and a first preset weight; determining a second product of the variance and a second preset weight; the sum of the first preset weight and the second preset weight being 1; and taking the sum of the first product and the second product as the quality index of the initial depth information of the sampling point, or taking the product of the first product and the second product as the quality index of the initial depth information of the sampling point.

[0016] In a second aspect, a material stack recognition system for a depalletizing robot is provided, and the material stack recognition system comprises: an acquisition device configured to acquire target depth information of each sampling point in a grabbing surface of a material stack; a processing device configured to determine a flatness index of each sampling point in the grabbing surface based on the target depth information of the plurality of sampling points of the grabbing surface; the flatness index being used to indicate a flatness degree of the sampling point; the processing device is further configured to determine a target grabbing area in the grabbing surface according to the flatness index of the plurality of sampling points of the grabbing surface; and the processing device is further configured to generate and send grabbing information to the material grabbing device; the grabbing information being used to instruct the material grabbing device to grab the target grabbing area by the grabbing part.

[0017] The stacking material recognition method of the de-stacking robot disclosed in the application determines the flatness index of each sampling point based on the target depth information of each sampling point in the material stacking grabbing surface, and then determines the target grabbing area according to the flatness index, and finally generates and sends grabbing information to guide the material grabbing device to grab. This method effectively solves the problem of irregular stacking of flexible packaging materials in the prior art. By introducing the concept of flatness index, the application can accurately quantify the flatness of the stacking surface, overcoming the dependence of traditional methods on the identification of rigid object edges and planes, and effectively identifying the uneven stacking surface formed by flexible materials. In addition, by determining the target grabbing area, the application provides a clear grabbing point for the de-stacking robot, avoiding the failure of grabbing caused by inaccurate pose estimation, significantly improving the recognition success rate and grabbing efficiency of the de-stacking robot when handling flexible packaging materials, and improving the continuity and reliability of de-stacking. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A flowchart of a de-stacking robot stacking material recognition method provided by the application is shown in the figure;

[0019] Figure 2 A flowchart of another de-stacking robot stacking material recognition method provided by the application is shown in the figure;

[0020] Figure 3 A flowchart of another de-stacking robot stacking material recognition method provided by the application is shown in the figure;

[0021] Figure 4 An architecture diagram of a de-stacking robot stacking material recognition system provided by the application is shown in the figure. DETAILED DESCRIPTION

[0022] In the description of the application, "multiple" means two or more than two, unless otherwise specified. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0023] In addition, in order to clearly describe the technical solutions of the embodiments of the application, the same items or similar items with basically the same function and role are distinguished by using "first", "second", etc. in the embodiments of the application. Those skilled in the art can understand that "first", "second", etc. do not limit the quantity and execution order, and "first", "second", etc. also do not necessarily mean different.

[0024] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being better or more advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0025] It is understood that the term "embodiment" used throughout the specification means that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, various embodiments throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0026] It is understood that in this application, “when…”, “if” and “if” all refer to the corresponding processing that will be carried out under certain objective circumstances, and are not limited to a time, nor do they require that there must be a judgment action when implemented, nor do they mean that there are other limitations.

[0027] It is understood that some optional features in the embodiments of this application can be implemented independently in certain scenarios without relying on other features, such as the current solution on which they are based, to solve the corresponding technical problems and achieve the corresponding effects. Alternatively, they can be combined with other features as needed in certain scenarios. Correspondingly, the apparatus given in the embodiments of this application can also implement these features or functions, which will not be elaborated here.

[0028] In this application, unless otherwise specified, the same or similar parts between the various embodiments can be referred to each other. In the various embodiments and implementation methods of the various embodiments in this application, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments and between the implementation methods of the various embodiments are consistent and can be mutually referenced. The technical features in different embodiments and between the implementation methods of the various embodiments can be combined according to their inherent logical relationships to form new embodiments, implementation methods, implementation methods, or implementation approaches. The following embodiments of this application do not constitute a limitation on the scope of protection of this application.

[0029] For ease of understanding, the title of this application will be explained below:

[0030] A "sampling point" refers to a discrete data point obtained by a vision sensor (e.g., a depth camera) on the picking face of a material stack. Each sampling point contains its position information in three-dimensional space, especially depth information.

[0031] "Target depth information" refers to information that has been processed and filtered to accurately reflect the true depth of a sampling point. In practical applications, raw depth data may be affected by factors such as noise, occlusion, or reflection, so it needs to be processed to obtain reliable target depth information.

[0032] The "flatness index" is a quantitative indicator that describes the flatness of the local area where the sampling point is located. The higher the flatness index, the flatter the area; otherwise, the area is less flat. The determination of this index is crucial for identifying suitable areas for picking.

[0033] The "picking face" refers to the surface of the material stack that can be picked by the robot. This surface may be irregular, consisting of the tops or sides of multiple materials.

[0034] The "target picking area" refers to a specific area identified on the picking face that is suitable for the material picking device to perform picking. This area usually has sufficient flatness, stable structure, and appropriate size to ensure the success of the picking operation.

[0035] The "material picking device" can also be called a depalletizing robot, which can depalletize or palletize material stacks.

[0036] The "picking part" is a specific execution component of the material picking device, such as the suction surface of a suction cup or the clamping end of a gripper.

[0037] "Picking information" is the instruction generated and sent to the material picking device, containing parameters such as the position, pose, and picking force of the target picking area, guiding the material picking device to perform the picking operation.

[0038] In automated warehousing and logistics sorting operations, depalletizing robots identify stacked materials through a vision system and then perform picking and placing tasks. Traditional identification methods perform extremely efficiently and accurately when dealing with single categories, regular sizes, and neatly stacked rigid goods. However, as the logistics industry continues to demand higher utilization of warehouse space and transportation efficiency, and pressure to control packaging material costs, more and more goods begin to use flexible packaging materials instead of traditional rigid cartons. These flexible packaging include but are not limited to plastic bags, soft packaging bags, shrink film wrapped bundles, textile bags, and various irregularly shaped soft containers.

[0039] Due to the inherent plasticity of flexible packaging, when these materials are stacked on a pallet, they are difficult to form a flat and regular stack. Instead, they will squeeze each other, fill the gaps, and the entire stack surface will present a complex form with unevenness. The weight of the upper materials will further exert pressure on the lower materials, making the deformation of the lower materials more significant. For example, the top surface of a stack of soft packaging bags may no longer be a flat surface, but rather composed of multiple raised and recessed areas. Individual materials may also exhibit local bulging, collapse or twisting, and their shape becomes highly uncertain in the stack. This irregular stacking and continuous deformation makes the external contour and size of the materials no longer fixed, but dynamically changes with the stacking method and stress condition.

[0040] Therefore, it is difficult for the robot to plan an effective grasping path and grasping strategy, greatly increasing the risk of grasping failure, resulting in a significant increase in recognition failure rate for the de-stacking robot when handling flexible packaging materials, seriously affecting the efficiency and continuity of de-stacking.

[0041] To solve the above problems, the present application provides a de-stacking robot stacked material recognition method, as shown in Figure 1 The de-stacking robot stacked material recognition method includes the following steps:

[0042] S101, the processing device obtains the target depth information of each sampling point in the grasping surface of the material stack.

[0043] As a possible implementation, as shown in Figure 2 The processing device can execute the following steps to obtain the target depth information of each sampling point in the grasping surface of the material stack:

[0044] S201, the processing device obtains the initial depth information of each sampling point in the grasping surface of the material stack.

[0045] Obtaining the initial depth information of each sampling point in the grasping surface of the material stack can be understood as scanning the grasping surface of the material stack by a depth sensor (e.g., a laser radar, a structured light camera or a time-of-flight camera), thereby obtaining the initial depth information corresponding to each spatial position point.

[0046] S202, the processing device determines, for each sampling point in the plurality of sampling points of the grasping surface, the ratio of the number of first sampling points to the number of second sampling points in the first region.

[0047] The sampling point is located at the center of the first region, the initial depth information of the first sampling point is within the preset depth range, and the initial depth information of the second sampling point is outside the preset depth range.

[0048] Specifically, the processing device takes the current sampling point to be evaluated as the center, and defines a local range, i.e., a first region, around it. In the first region, the initial depth information of all sampling points is compared with a preset depth range. If the initial depth information of a certain sampling point is within the preset depth range, it is regarded as a first sampling point; otherwise, it is regarded as a second sampling point.

[0049] S203, the processing device determines the variance of the initial depth information of the first sampling points in the first region.

[0050] Specifically, the processing device calculates the statistical variance of the initial depth information of those points identified as first sampling points in the first region. Variance is an indicator of the degree of dispersion of a set of data, which is used here to indicate the consistency or flatness of the depth information in the first region. A smaller variance indicates that the depth values in the region are very close, i.e., the surface is relatively flat; while a larger variance may mean that the region has a slope, texture changes or residual noise.

[0051] S204, the processing device determines the quality index of the initial depth information of the sampling points according to the ratio and the variance.

[0052] As a specific implementation of S204, first, the processing device determines the product of the ratio and a first preset weight to obtain a first product. Second, the processing device determines the product of the variance and a second preset weight to obtain a second product; the sum of the first preset weight and the second preset weight is 1. Finally, the sum of the first product and the second product is taken as the quality index of the initial depth information of the sampling points, or the product of the first product and the second product is taken as the quality index of the initial depth information of the sampling points.

[0053] Specifically, in determining the quality index of the initial depth information of the sampling points, two main ways can be used to combine the ratio and the variance. The first way is weighted summation, i.e., first multiply the ratio by a first preset weight to obtain a first product; at the same time, multiply the variance by a second preset weight to obtain a second product. The sum of the first preset weight and the second preset weight is set to 1 to ensure the rationality of the weight distribution. Then, the first product and the second product are summed, and the sum is determined as the quality index of the initial depth information of the sampling points. This way allows the importance of the ratio and the variance in different scenarios to be flexibly evaluated by adjusting the weights.

[0054] As another optional implementation, the quality index can also be obtained by weighted summation. Specifically, after obtaining the first product and the second product, the first product is multiplied with the second product, and the product is determined as the quality index of the initial depth information of the sampling point. This multiplication method can be more sensitive to outliers in some cases, or can show better discrimination when more stringent conditions for ratio and variance are required.

[0055] wherein the ratio can be understood as the ratio of the number of first sampling points in the first region within the preset depth range to the number of second sampling points outside the preset depth range, which reflects the concentration of depth information around the sampling point. The variance indicates the dispersion of the initial depth information of the first sampling point in the first region, which reflects the stability of the depth data. The first preset weight and the second preset weight are parameters preset in advance, and their values can be adjusted according to actual application scenarios, material characteristics or experience to optimize the evaluation effect of the quality index.

[0056] The scheme of the present application introduces specific weighted summation or weighted multiplication calculation methods, making the determination process of the quality index more explicit and controllable. By assigning the first preset weight and the second preset weight to the ratio and the variance respectively, and ensuring that the sum of the weights is 1, the relative importance of the two indicators in quality evaluation can be adjusted according to actual needs. For example, when the concentration of depth information (reflected by the ratio) is more concerned, the first preset weight can be appropriately increased; when the stability of the depth information (reflected by the variance) is more concerned, the second preset weight can be increased. The weighted summation method can linearly combine the contributions of the two indicators, while the weighted multiplication method can produce a higher quality index when both indicators perform well, and significantly reduce the quality index when either indicator performs poorly, thereby providing more stringent quality evaluation. Thus, the quality of the initial depth information of the sampling point can be evaluated more finely, avoiding the uncertainty caused by experience or vague judgment.

[0057] Through the above technical solutions, the present application can provide two specific and adjustable quality index calculation methods, thereby overcoming the problems of fuzzy quality index determination method and lack of flexibility in the prior art. By introducing adjustable weights, the calculation of the quality index can better adapt to the depth information characteristics of different materials and different environments, improving the accuracy and robustness of the quality index evaluation. Whether using weighted summation or weighted multiplication, the quality evaluation of the initial depth information of the sampling point is more scientific and reasonable, thereby ensuring the accuracy of the subsequent target depth information, laying a solid foundation for finally determining a reliable target grasping region, and effectively improving the success rate and efficiency of the stacked material recognition of the unstacking robot.

[0058] In some preferred embodiments, the following is illustrated by a specific example. Assume that for a certain sampling point, the ratio in the first region is calculated as 0.8, and the variance of the initial depth information of the first sampling point in the first region is calculated as 0.1.

[0059] As a specific implementation, if the weighted sum is used to determine the quality index, and the first preset weight is set to 0.6 and the second preset weight is set to 0.4 (0.6 + 0.4 = 1). Then the first product is 0.8 * 0.6 = 0.48, and the second product is 0.1 * 0.4 = 0.04. At this time, the quality index of the sampling point will be determined as 0.48 + 0.04 = 0.52.

[0060] As another specific implementation, if the weighted product is used to determine the quality index, and the first preset weight is set to 0.7 and the second preset weight is set to 0.3. Then the first product is 0.8 * 0.7 = 0.56, and the second product is 0.1 * 0.3 = 0.03. At this time, the quality index of the sampling point will be determined as 0.56 * 0.03 = 0.0168.

[0061] Through the two different calculation methods, the quality index of the sampling point can be flexibly obtained according to the actual needs and the importance of the ratio and the variance, so as to provide more accurate basis for subsequent judgment whether to determine the initial depth information of the sampling point as the target depth information.

[0062] S205, the processing device determines the initial depth information of the sampling point as the target depth information of the sampling point when the quality index is greater than the preset quality index threshold.

[0063] Therefore, in the case where the quality index is greater than the preset quality index threshold, the initial depth information of the sampling point is determined as the target depth information of the sampling point. This means that only when the depth information around a sampling point shows good enough quality, its initial depth information will be considered as high quality and reliable, and thus adopted as the target depth information required for subsequent processing. The preset quality index threshold can be adjusted according to experience or experimental data to balance the accuracy and availability of the depth information.

[0064] The scheme of S101 can be realized by S201-S205, and the subsequent processing device can perform S102.

[0065] S102, the processing device determines a flatness index of each sampling point in the grasping surface based on the target depth information of the plurality of sampling points in the grasping surface; the flatness index is used to indicate the flatness of the sampling point.

[0066] As a possible implementation, as shown in Figure 3 The processing device can perform the following steps to determine the flatness index of each sampling point in the gripping surface:

[0067] S301, the processing device determines, for each sampling point in the plurality of sampling points of the gripping surface, a fitting plane corresponding to the plurality of sampling points in the first region according to the target depth information of the plurality of sampling points in the first region, and the sampling point is located at the center of the first region.

[0068] The first region can be understood as a local neighborhood centered on the current sampling point to be processed. The size and shape of the first region can be adjusted according to the actual application scenario and the required flatness detection accuracy. For example, the first region can be a square, circular or any shaped region, which can cover several to tens of sampling points. By obtaining the target depth information of all sampling points in the first region, the surface morphology of the local region can be analyzed.

[0069] The fitting plane refers to a plane model constructed by a mathematical method, such as the least square method, according to the target depth information of the plurality of sampling points in the first region. The fitting plane aims to approach the actual surface morphology of the local region as much as possible, thereby providing a reference for subsequent flatness index calculation. The flatness index is used to indicate the flatness of the sampling point. By comparing the relationship between the sampling point and the fitting plane, the flatness of the local region where the sampling point is located can be quantified. For example, when the deviation between the sampling point and the fitting plane is small, it indicates that the region is relatively flat; on the contrary, when the deviation is large, it indicates that the region has ups and downs or is not flat.

[0070] The scheme of the present application firstly defines a first region around each sampling point and fits a local plane using the target depth information of all sampling points in the first region when determining the flatness index of each sampling point. This approach can effectively capture the surface characteristics of the local region and avoid errors that may be caused by relying only on the depth information of a single sampling point. Subsequently, by comparing the deviation between the sampling point and the fitting plane, the flatness of the local region represented by the sampling point can be accurately quantified. This local fitting plane based method makes the calculation of the flatness index more robust and accurate, and can better reflect the actual flatness of the material pile gripping surface.

[0071] By the technical solution, when determining the flatness index of each sampling point in the grabbing surface, the flatness of the region where each sampling point is located can be more accurately and finely evaluated in a local plane fitting manner. This method overcomes the limitations that may exist in simple depth comparison, improves the reliability of the flatness index, and thus provides a more accurate basis for subsequent determination of the target grabbing region, which helps to improve the success rate and stability of the material grabbing of the unstacking robot.

[0072] S302, the processing device determines the flatness index of the sampling point according to the fitting plane and the target depth information of the sampling point in the first region.

[0073] As a specific implementation of S302, the processing device determines the distance between each sampling point in the first region and the fitting plane, and determines the reciprocal of the distance as the flatness index of the sampling point.

[0074] The distance between each sampling point in the first region and the fitting plane is determined, that is, the vertical distance of each sampling point in the first region to the corresponding fitting plane in the three-dimensional space is calculated. The distance can quantify the degree of deviation of each sampling point from the local fitting plane, thereby reflecting the flatness of the local region where the sampling point is located. Specifically, if the fitting plane is represented by the equation Ax + By + Cz + D = 0, and the coordinates of a sampling point are (x0, y0, z0), the distance d of the sampling point to the fitting plane can be calculated by the formula d = |Ax0 + By0 + Cz0 + D| / sqrt(A^2 + B^2 + C^2). Here, z0 is the target depth information of the sampling point.

[0075] Further, the reciprocal of the distance is determined as the flatness index of the sampling point. This means that the smaller the distance of the sampling point to the fitting plane, the larger the reciprocal, so that the flatness index is larger, which intuitively indicates that the region where the sampling point is located is more flat. Conversely, the larger the distance of the sampling point to the fitting plane, the smaller the reciprocal, and the smaller the flatness index, indicating that the region where the sampling point is located is less flat. This processing method makes the flatness index positively correlated with the actual flatness, which is convenient for subsequent judgment and processing.

[0076] This implementation first determines the distance from each sampling point to its local fitting plane, and then uses the reciprocal of this distance as a flatness index, providing an effective method for quantifying local flatness. The flatter the local area where a sampling point is located, the closer the sampling point and its surrounding points will be to a plane, thus reducing the distance from the sampling point to its fitting plane. By taking the reciprocal of the distance, the flatness index is positively correlated with the flatness; that is, the larger the flatness index, the flatter the area, which aligns with people's intuitive understanding of "flatness." Therefore, this implementation accurately reflects the local geometric characteristics of the material stacking gripping surface, providing a reliable basis for subsequent determination of the target gripping area.

[0077] By employing the reciprocal of the distance as a flatness index, the above technical solution provides a precise and intuitive method for measuring flatness. This method effectively distinguishes the flatness of different regions on the grasping surface, assigning higher flatness indices to flat regions and lower flatness indices to non-flat regions. Compared to methods using only depth information variance, using the reciprocal of the fitted plane distance more accurately captures the geometric features of the local surface, effectively reducing the impact of noise and minor local undulations on flatness judgment. This improves the robustness and accuracy of the flatness index, laying a solid foundation for subsequently identifying more suitable grasping regions.

[0078] S103. The processing device determines the target grasping area in the grasping surface based on the flatness index of multiple sampling points on the grasping surface.

[0079] As one possible implementation, firstly, a first grasping region in the grasping surface is determined based on the flatness index of multiple sampling points on the grasping surface; the flatness index of each sampling point in the first grasping region is greater than a first flatness index threshold. Secondly, the grasping region in the first grasping region whose area is greater than a preset area threshold is determined as the target grasping region.

[0080] Specifically, when determining the target gripping area, the first step is to filter out preliminary candidate areas based on the flatness index of each sampling point in the gripping surface. The first gripping area refers to the set of regions formed by all sampling points in the gripping surface whose flatness index is greater than a preset first flatness index threshold. The first flatness index threshold is a preset value used to define whether a sampling point is flat enough to be considered a potential gripping point. For example, this threshold can be set empirically or determined experimentally based on factors such as the characteristics of the material, the type of gripping device, and the required gripping stability. Its purpose is to initially eliminate areas with insufficient flatness, ensuring that the areas to be processed subsequently have a certain degree of flatness.

[0081] After obtaining the first grasping region, in order to ensure that the material grasping device can stably and effectively grasp, further screening of the first grasping region is needed. Specifically, the grasping region with an area greater than a preset area threshold in the first grasping region is determined as the target grasping region. The preset area threshold is a preset minimum area value, which is used to ensure that the identified grasping region has sufficient size to adapt to the grasping part of the material grasping device. For example, if the grasping part is a suction cup, the preset area threshold should be greater than the effective adsorption area of the suction cup; if the grasping part is a gripper, the preset area threshold should be greater than the minimum area that the gripper can stably hold. The purpose is to exclude those flat regions with too small area that are insufficient to support stable grasping, thereby improving the success rate and stability of grasping.

[0082] The scheme of the present application performs double screening on the potential grasping region determined based on the flatness index by introducing the first flatness index threshold and the preset area threshold. First, by the first flatness index threshold, regions with insufficient flatness can be effectively filtered out, ensuring that the selected region has high flatness in geometry, which is crucial for stable adsorption or clamping of the material grasping device. Second, by the preset area threshold, regions with insufficient area are further excluded, which may not provide sufficient contact area or support force, leading to unstable or failed grasping. It is due to this step-by-step screening mechanism that the finally determined target grasping region is not only flat but also has sufficient size, thereby significantly improving the success rate and stability of the material grasping device.

[0083] Through the above technical scheme, the present application can effectively solve the problem that the traditional method may identify unsuitable grasping regions. By setting the first flatness index threshold, it can be ensured that the selected region has sufficient flatness, avoiding grasping failure due to uneven surface. At the same time, by introducing the preset area threshold, regions with insufficient area that cannot be effectively grasped by the material grasping device are excluded, thereby avoiding ineffective grasping attempts. This double screening mechanism makes the finally determined target grasping region more consistent with the actual operation requirements of the material grasping device, significantly improving the accuracy and grasping efficiency of the stacked material recognition method of the unstacking robot, and reducing the risk of grasping failure and material damage.

[0084] For example, assume that a depalletizing robot needs to grasp a stack of cartons. First, the depth information of the carton stack grasping surface is obtained by a depth camera, and the flatness index of each sampling point is calculated. For example, if the flatness index of a certain sampling point is 0.8, it means that it is very flat; if it is 0.2, it means that it is not flat. Set the first flatness index threshold to 0.7, then all sampling points with a flatness index greater than 0.7 are identified as the first grasping region. Then, connected component analysis is performed on these first grasping regions, and the area of each connected component is calculated. Assuming that the suction cup diameter of the material grasping device is 10 cm, in order to ensure that the suction cup can be completely adsorbed, the preset area threshold can be set to 80% of the area of the suction cup, for example, 60 cm2. Then, all connected components with an area greater than 60 cm2 are selected from the first grasping region, and these regions are determined as the final target grasping region. In this way, it can be ensured that the robot grasps a region that is flat enough and has a large enough area, so that stable and reliable grasping can be achieved.

[0085] By determining the region in the grasping surface with a flatness index greater than the first flatness index threshold and an area greater than the preset area threshold as the target grasping region, the stability of grasping can be ensured to a certain extent. However, in actual application, even if a region meets the flatness and area requirements, if its surface has a certain inclination, that is, the included angle between its plane normal vector and the vertical direction is too large, it may still cause problems such as unstable grasping, material slipping or difficulty in accurate placement of the material grasping device during grasping, especially in scenarios that require accurate stacking or have strict requirements on the posture of the material, such inclination will significantly affect the efficiency and safety of the operation.

[0086] In this regard, in the above implementation manner, the processing device determines the grasping region in the first grasping region with an area greater than the preset area threshold as the target grasping region, which can include the following steps:

[0087] The processing device determines the grasping region in the first grasping region with an area greater than the preset area threshold as the second grasping region. The grasping region in the second grasping region with an included angle between the plane normal vector and the vertical direction less than the preset included angle threshold is determined as the target grasping region.

[0088] Specifically, after determining the first grasping area in the grasping surface according to the flatness indexes of multiple sampling points of the grasping surface, and determining the grasping areas with an area greater than a preset area threshold in the first grasping area as the second grasping area, further screening of the second grasping area is required. The second grasping area refers to the potential grasping area that has met the flatness and area requirements. Further, the plane normal vector of each potential grasping area in the second grasping area needs to be calculated. The plane normal vector refers to a vector perpendicular to the surface of the area, and its direction indicates the orientation of the area. At the same time, the vertical direction, usually parallel to the direction of gravity, needs to be determined. By calculating the included angle between the plane normal vector of each potential grasping area in the second grasping area and the vertical direction, the inclination of the area can be evaluated. As a preferred embodiment, when the included angle is less than a preset included angle threshold, the area is determined as the final target grasping area. The preset included angle threshold is a configurable parameter, and its value can be set according to factors such as the characteristics of the material, the type of the grasping device, and the stacking accuracy requirement. For example, for materials that need to be placed horizontally, the preset included angle threshold can be set smaller to ensure that the grasping area is as close to horizontal as possible; for materials that allow a certain degree of inclination, the preset included angle threshold can be appropriately relaxed.

[0089] The scheme of the present application effectively solves the problem of unstable grasping caused by only considering flatness and area by introducing the judgment of the included angle between the plane normal vector of the grasping area and the vertical direction. Specifically, on the basis of the second grasping area that has been screened for flatness and sufficient area, those areas that are flat but have too large an inclination are further excluded. It is precisely because of the strict restriction on the posture of the grasping area that ensures the surface direction of the selected target grasping area to be highly consistent with the grasping direction of the material grasping device (usually the vertical direction), thereby providing a more stable and reliable grasping basis for the material grasping device.

[0090] Through the above technical scheme, the stability of the material grasping of the unstacking robot can be significantly improved, and problems such as material slipping, damage or uneven stacking caused by the inclination of the grasping surface can be effectively avoided. In addition, due to the optimization of the grasping posture, the material grasping device can grasp and place with a more accurate posture, thereby improving the overall operation accuracy and efficiency, reducing the operation risk, and prolonging the service life of the equipment.

[0091] For example, assume that the depalletizing robot needs to grab a series of rectangular boxes from a stack. First, the flatness index of each sampling point is obtained and calculated by depth information, and a plurality of potential second grabbing regions are screened according to the flatness index and the area. For example, there may be a top surface of a box, most of the area of which is flat and large enough, but due to slight extrusion or improper placement during stacking, the surface of the box is slightly inclined. If only the flatness index and the area are considered, the inclined surface may still be selected as the target grabbing region. However, according to the scheme of the present application, after the second grabbing region is determined, the angle between the normal vector of the region and the vertical direction is further calculated. If the angle between the normal vector of a certain second grabbing region and the vertical direction exceeds a preset angle threshold (for example, 5 degrees), even if it is flat and has a large enough area, the region will be excluded. Finally, only those regions that are not only flat and have a large enough area, but also have an acceptable inclination will be determined as the target grabbing region. For example, if a box surface is inclined by 10 degrees, and the preset angle threshold is 5 degrees, the surface will not be selected as the target grabbing region, thereby avoiding the risk of unstable grabbing. In this way, it is ensured that the material grabbing device always selects the most suitable and correctly positioned region for operation.

[0092] By obtaining the target depth information of each sampling point in the grabbing surface of the material stack, and determining the flatness index of the sampling point based on the target depth information, and then determining the target grabbing region according to the flatness index and the angle between the normal vector of the region and the vertical direction, the grabbing information for the material grabbing device is provided. However, in actual application, when the grabbing part of the material grabbing device is a suction cup, only considering the flatness and directionality of the grabbing region may not be enough to ensure the success rate of grabbing. For example, if there are small pores on the surface of the material or gaps between the stacked materials, even if the surface is flat and the direction is suitable, the suction cup may not form effective adsorption due to air leakage, resulting in failure to grab or the material falling off. If the above problem is not solved, it will directly affect the efficiency and stability of the depalletizing robot.

[0093] To this end, in the implementation mode described above, when the grabbing part is a suction cup, the processing device determines the grabbing region in the second grabbing region whose angle between the normal vector and the vertical direction is less than the preset angle threshold as the target grabbing region, which can specifically include the following steps:

[0094] The processing device determines the grabbing region in the second grabbing region whose angle between the normal vector and the vertical direction is less than the preset angle threshold as the third grabbing region. Next, the processing device obtains the air permeability index of the third grabbing region; the air permeability index is used to indicate the air permeability of the third grabbing region. Next, the processing device determines the grabbing region in the third grabbing region whose air permeability index is less than the preset air permeability index threshold as the target grabbing region.

[0095] Specifically, the above-mentioned grabbing part can be understood as a component of the material grabbing device for directly contacting and grabbing the material. When it is set as a suction cup, its working principle is to use the negative pressure difference to adsorb the material. The third grabbing area refers to further screening out the area whose included angle with the vertical direction is less than the preset included angle threshold based on the second grabbing area. These areas are usually considered as surfaces with good posture that are suitable for suction cup grabbing. The air permeability index can be understood as an index for quantifying the air permeability of the surface of the third grabbing area. The larger the value, the stronger the air permeability, and the more difficult it is for the suction cup to form effective adsorption. For example, the air permeability index can be obtained by measuring the change in internal pressure of the suction cup during the adsorption process. The larger the change, the more serious the air leakage, and the higher the air permeability index. The preset air permeability index threshold is a pre-set reference value for distinguishing the air permeability of the surface suitable for suction cup grabbing and unsuitable for suction cup grabbing. When the air permeability index of the third grabbing area is less than the threshold, it indicates that the air permeability of the area is within an acceptable range, and the suction cup can effectively adsorb.

[0096] The scheme of the present application effectively solves the problem that when the grabbing part is a suction cup, only relying on flatness and posture information cannot guarantee the success rate of grabbing. Specifically, after determining the third grabbing area with good flatness and posture, the air permeability index of the area is further obtained. If the air permeability index of the area is too high, i.e., the air permeability is strong, it indicates that the suction cup is difficult to form a stable negative pressure in the area, thereby unable to effectively grab the material. By excluding the area with an air permeability index greater than the preset air permeability index threshold, only selecting the area with an air permeability index less than the preset air permeability index threshold as the final target grabbing area, it is ensured that the selected area is not only suitable for suction cup grabbing in terms of geometric shape, but also supports stable adsorption of the suction cup in terms of physical properties.

[0097] Through the above technical scheme, the present application can significantly improve the success rate and stability of the suction cup when the suction cup is used to grab the material by the unstacking robot. Since the air permeability of the material surface is fully considered, invalid grabbing of the suction cup in areas with poor air permeability is avoided, thereby reducing the risk of grabbing failure and material falling, and improving the work efficiency. In addition, the scheme enables the unstacking robot to more intelligently adapt to materials of different materials and stacking states, broadens its application range, and reduces material damage or equipment wear caused by grabbing failure.

[0098] For example, assume that the unstacking robot needs to grab a carton from a stack of cartons stacked by multiple layers of corrugated boxes. First, through depth information processing, the grabbing surface of the carton stack is identified, and the flatness index of each sampling point is calculated to screen out the flat areas. Then, in these flat areas, further screening is performed to select areas where the angle between the plane normal vector and the vertical direction is less than a preset angle threshold to form a third grabbing area. At this time, if only the flatness and posture are considered, the surface of the carton may be damaged or at the joint, and these areas may appear flat in geometry, but actually have higher air permeability. To avoid the failure of the suction cup grabbing, the present application further acquires the air permeability index of the third grabbing area. For example, the material grabbing device can temporarily attach the suction cup to a certain point in the third grabbing area at a low pressure, and use the built-in pressure sensor to detect the pressure change in the suction cup within a preset time. If the pressure change is large, it indicates that the area has serious air leakage, and the air permeability index is high, so the area will be excluded. Finally, only those areas with air permeability index less than a preset air permeability index threshold will be determined as the target grabbing area, so as to ensure that the suction cup can stably and reliably grab the carton.

[0099] In the above embodiment, the air permeability index of the third grabbing area can be obtained by the following scheme:

[0100] The processing device sends first information to the material grabbing device, the first information being used to instruct the material grabbing device to attach the suction cup to the second grabbing area at a preset pressure, and to instruct the material grabbing device to detect the pressure change in the suction cup within a preset time through the pressure detection device; receives second information including the change amount from the material grabbing device, and takes the change amount as the air permeability index of the third grabbing area.

[0101] Specifically, the processing device sends first information to the material grabbing device, which aims to remotely control the material grabbing device to perform the air permeability detection operation. The first information can include adsorption instructions, a preset pressure value, a detection duration, etc. The preset pressure refers to the negative pressure applied by the suction cup when adsorbed on the material surface. The pressure value should be set according to the characteristics of the material and the adsorption capacity of the suction cup to ensure that the suction cup can form a preliminary seal with the material surface. The preset duration refers to the length of time for which the pressure detection device detects the pressure change. The duration should be sufficient to reflect the stability and trend of the internal pressure of the suction cup, but not too long to affect the work efficiency. After receiving the first information, the material grabbing device drives the suction cup to adsorb on the third grabbing area with a preset pressure. At the same time, the pressure detection device configured inside or outside the material grabbing device starts to monitor the pressure change inside the suction cup. The pressure detection device can be a high-precision pressure sensor that can measure the internal pressure of the suction cup in real time and continuously. After the preset duration ends, the pressure detection device calculates the change in the internal pressure of the suction cup. This change can be the maximum drop in pressure, the average drop rate, or the total drop in pressure during the preset duration. For example, if the internal pressure of the suction cup drops from P1 to P2 during the preset duration, the change can be calculated as P1-P2. Subsequently, the material grabbing device sends second information containing the change back to the processing device. After receiving the second information, the processing device directly uses the change as the air permeability index of the third grabbing area. The larger the air permeability index, the faster the internal pressure of the suction cup drops, the stronger the air permeability of the material surface, and the worse the adsorption effect. Conversely, the smaller the air permeability index, the weaker the air permeability of the material surface, and the better the adsorption effect.

[0102] The scheme of the present application obtains the air permeability index by actually simulating the adsorption process of the suction cup, thereby solving the problem that the traditional method is difficult to accurately evaluate the air permeability of the material surface. It is because the change in the internal pressure of the suction cup during the adsorption process can directly reflect the sealing degree between the material surface and the suction cup and the air permeability of the material itself that the method can objectively and quantitatively evaluate the air permeability of the grabbing area. When the material surface has pores or is uneven, the internal pressure of the suction cup will drop rapidly after adsorption, resulting in a large pressure change, which indicates that the area has strong air permeability and is not suitable for suction cup grabbing. Conversely, if the material surface is flat and impermeable, the change in the internal pressure of the suction cup will be small, indicating that the area is suitable for suction cup grabbing. This detection method based on actual physical feedback avoids complex inference of the material quality or surface condition, improving the accuracy and reliability of the air permeability index evaluation.

[0103] Through the above technical solution, this application provides an objective and accurate method for obtaining the air permeability index. Compared to relying solely on visual information or preset parameters to determine the air permeability of the gripping area, this application directly measures the pressure change during suction cup adsorption, which more realistically reflects the actual air permeability of the material surface, thereby effectively avoiding gripping failures caused by air permeability issues. This physical feedback-based detection method significantly improves the success rate and stability of the palletizing robot's gripping operations, reduces the risk of material falling and damage, and thus enhances overall operational efficiency and safety.

[0104] For example, suppose a depalletizing robot needs to grasp a stack of cardboard boxes. After screening based on flatness index and plane normal vector, a third grasping area is determined. To obtain the air permeability index of this area, the processing device sends a first message to the material grasping device. This first message instructs the material grasping device to attach the suction cup to the third grasping area at a preset pressure of -50 kPa, and instructs its internal pressure detection device to monitor the pressure change inside the suction cup within a preset duration of 3 seconds. Upon receiving the instruction, the material grasping device drives the suction cup to adhere to the surface of the cardboard box. The pressure detection device begins recording the pressure inside the suction cup. If the pressure inside the suction cup drops from -50 kPa to -40 kPa within 3 seconds, the pressure change is 10 kPa. The material grasping device sends a second message containing this 10 kPa change back to the processing device. The processing device uses 10 kPa as the air permeability index of the third grasping area. If the preset air permeability threshold is 8 kPa, and 10 kPa is greater than 8 kPa, then this third gripping area is judged to have high air permeability and is not suitable for suction cup gripping. The system will then attempt to find other areas with an air permeability index less than 8 kPa as the final target gripping area. Conversely, if the pressure change is 5 kPa, which is less than 8 kPa, then this area is considered suitable for gripping.

[0105] S104. The processing equipment generates and sends grabbing information to the material grabbing device; the grabbing information is used to instruct the material grabbing device to grab the target grabbing area through the grabbing part.

[0106] Based on S101-S104, the stacking material recognition method of the de-stacking robot disclosed in the present application obtains target depth information of each sampling point in the grabbing surface of the material stack, determines the flatness index of each sampling point based on the depth information, and then determines the target grabbing area according to the flatness index, and finally generates and sends grabbing information to guide the material grabbing device to grab. This method effectively solves the problem of irregular stacking of flexible packaging materials in the prior art. By introducing the concept of flatness index, the present application can accurately quantify the flatness of the stacking surface, overcoming the dependence of traditional methods on rigid object edges and flat surface recognition, and can effectively identify the uneven stacking surface formed by flexible materials. In addition, by determining the target grabbing area, the present application provides a clear grabbing point for the de-stacking robot, avoiding grabbing failure caused by inaccurate pose estimation, significantly improving the recognition success rate and grabbing efficiency of the de-stacking robot when handling flexible packaging materials, and improving the continuity and reliability of de-stacking.

[0107] The processing device can be a processor in the de-stacking robot.

[0108] The specific embodiments of the present application also disclose a de-stacking robot stacking material recognition system, which comprises Figure 4 As shown in the figure, the de-stacking robot stacking material recognition system comprises:

[0109] The acquisition device is configured to obtain target depth information of each sampling point in the grabbing surface of the material stack.

[0110] The processing device is configured to determine a flatness index of each sampling point in the grabbing surface based on the target depth information of the plurality of sampling points of the grabbing surface. The flatness index is used to indicate the flatness of the sampling point.

[0111] The processing device is further configured to determine a target grabbing area in the grabbing surface according to the flatness index of the plurality of sampling points of the grabbing surface.

[0112] The processing device is further configured to generate and send grabbing information to the material grabbing device. The grabbing information is used to instruct the material grabbing device to grab the target grabbing area by the grabbing part.

[0113] The acquisition device and the processing device can be modules in the processing device.

[0114] The traditional existing unstacking robot is difficult to segment the irregular stack formed by flexible packaging materials in three dimensions, and the estimation of the grasping point and posture is inaccurate, which seriously affects the efficiency and continuity of unstacking. In view of this, the unstacking robot stack material recognition system proposed in the present application obtains the target depth information of the material stack grasping surface through the acquisition device, and determines the flatness index of each sampling point based on the information by the processing device, and then identifies the target grasping area, and finally generates and sends the grasping information to the material grasping device to guide the material grasping device to grasp. Through the cooperative work of each device, the system can effectively cope with the irregularity of the flexible packaging material stack, and significantly improve the recognition accuracy and grasping success rate of the unstacking robot for the stacked materials.

[0115] In order to better understand the unstacking robot stack material recognition system proposed in the present application, the acquisition device and the processing device involved therein are described in detail below.

[0116] First of all, the acquisition device is used to obtain the target depth information of each sampling point in the grasping surface of the material stack. The specific method of obtaining the target depth information of each sampling point in the grasping surface of the material stack has been described in the above embodiment, and will not be repeated here. It should be emphasized that the acquisition device can be a combination of one or more sensors, for example, it can be a depth camera such as a structured light camera, a time-of-flight (ToF) camera or a stereo vision system, which is used to directly scan the grasping surface of the material stack to obtain the original depth data. As another implementation, the acquisition device can also be a laser radar for obtaining high-precision point cloud data. In some embodiments, the acquisition device can also include an image acquisition module and a preprocessing module, wherein the image acquisition module is used to obtain the original image data, and the preprocessing module is used to filter, complete and other operations on the original image data to generate the target depth information.

[0117] Secondly, the processing device is configured to determine a flatness index of each sampling point in the grasping surface based on the target depth information of the plurality of sampling points in the grasping surface. The flatness index is used to indicate the flatness of the sampling point. The processing device is further configured to determine a target grasping region in the grasping surface according to the flatness indexes of the plurality of sampling points in the grasping surface. The processing device is further configured to generate and send grasping information to the material grasping device. The grasping information is used to instruct the material grasping device to grasp the target grasping region by the grasping part. The specific method for determining the flatness index, determining the target grasping region, and generating and sending the grasping information has been described in detail in the above embodiments, and will not be repeated here. It should be emphasized that the processing device can be a combination of one or more computing units, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC). In some embodiments, the processing device can be an independent computing module, which integrates the necessary algorithms and logic circuits inside, for performing the above-mentioned processing functions. As another implementation, the processing device can also be part of a distributed computing system, in which different processing tasks are assigned to different computing nodes for execution. The processing device can also include a memory for storing program instructions and intermediate data and results generated during processing.

[0118] Compared with the prior art, the core innovation of the de-palletizing robot stack material recognition system proposed in the present application lies in that through the cooperative work of the acquisition device and the processing device, effective recognition of the irregularity of the flexible packaging material stack is realized. Traditional systems often rely on the recognition of the edges and geometric shapes of rigid objects. When facing flexible packaging materials, due to the irregular surface and fuzzy boundary, three-dimensional segmentation is difficult, and the grasping point and pose estimation are inaccurate.

[0119] The system of the present application acquires target depth information of the material stack grasping surface through the acquisition device, and determines the flatness index of each sampling point based on these information by the processing device, which can effectively overcome the challenge of irregular flexible material surface. The flatness index can directly quantify the flatness of the local area, so that the system no longer relies too much on clear geometric edges or preset models, but can extract the region features suitable for grasping from the depth data itself. For example, in the traditional scheme, the surface of a pile of soft packaging bags may be incorrectly segmented into multiple irregular small blocks, or recognized as one large block that cannot be grasped. However, the scheme of the present application can identify the relatively flat regions on the surface of these soft packaging bags that are suitable for suction or clamping through the processing device to calculate the flatness index, even if the boundaries of these regions are not clear.

[0120] In addition, the system of the present application determines the target grabbing area according to the flatness index through the processing device, and generates accurate grabbing information to directly guide the material grabbing device to grab. This system architecture avoids the deviation of subsequent grabbing posture estimation caused by the failure or inaccuracy of three-dimensional segmentation in traditional systems, thereby significantly improving the identification accuracy and grabbing success rate of the de-palletizing robot when processing flexible packaging materials. In this way, the system of the present application provides an efficient and reliable solution for the de-palletizing of flexible packaging materials in automated warehousing and logistics sorting operations, with significant progress and practicality.

[0121] The above merely illustrates the embodiments of the present application and is not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying stacked materials in a depalletizing robot, characterized in that, The method for identifying stacked materials using the depalletizing robot includes: Obtain the target depth information of each sampling point in the gripping surface of the material stack; Based on the target depth information of multiple sampling points on the gripping surface, a flatness index is determined for each sampling point on the gripping surface; the flatness index is used to indicate the flatness of the sampling point. The target grasping area in the grasping surface is determined based on the flatness index of multiple sampling points of the grasping surface; Generate and send grasping information to the material grasping device; the grasping information is used to instruct the material grasping device to grasp the target grasping area through the grasping part; The acquisition of target depth information for each sampling point in the gripping surface of the material stack includes: Obtain the initial depth information of each sampling point in the gripping surface of the material stack; For each of the multiple sampling points on the grasping surface, determine the ratio of the number of first sampling points to the number of second sampling points in the first region. The sampling points are located at the center of the first region, the initial depth information of the first sampling points is within a preset depth range, and the initial depth information of the second sampling points is outside the preset depth range. Determine the variance of the initial depth information of the first sampling point in the first region; The quality index of the initial depth information of the sampling point is determined based on the ratio and the variance. If the quality index is greater than a preset quality index threshold, the initial depth information of the sampling point is determined as the target depth information of the sampling point.

2. The method for identifying stacked materials in a depalletizing robot according to claim 1, characterized in that, Based on the target depth information of multiple sampling points on the gripping surface, the flatness index of each sampling point in the gripping surface is determined, including: For each of the multiple sampling points on the grasping surface, a fitting plane corresponding to the multiple sampling points in the first region is determined based on the target depth information of the multiple sampling points in the first region, wherein the sampling point is located at the center of the first region; The flatness index of the sampling points is determined based on the target depth information of the fitting plane and the sampling points in the first region.

3. The method for identifying stacked materials in a depalletizing robot according to claim 2, characterized in that, Determining the flatness index of the sampling points based on the target depth information of the fitting plane and the sampling points in the first region includes: Determine the distance between each sampling point in the first region and the fitted plane; determine the reciprocal of the distance as the flatness index of the sampling point.

4. The method for identifying stacked materials in a depalletizing robot according to claim 1, characterized in that, The target grasping area in the grasping surface is determined based on the flatness index of multiple sampling points of the grasping surface, including: A first grasping region in the grasping surface is determined based on the flatness index of multiple sampling points of the grasping surface; the flatness index of each sampling point in the first grasping region is greater than a first flatness index threshold. The area of ​​the first grasping region that is larger than a preset area threshold is determined as the target grasping region.

5. The method for identifying stacked materials in a depalletizing robot according to claim 4, characterized in that, Determining the target grasping region as the grasping region whose area is greater than a preset area threshold within the first grasping region includes: The area of ​​the first grasping region that is larger than a preset area threshold is determined as the second grasping region; The target grasping area is defined as the grasping area where the angle between the plane normal vector and the vertical direction in the second grasping area is less than a preset angle threshold.

6. The method for identifying stacked materials in a depalletizing robot according to claim 5, characterized in that, The gripping part is a suction cup. The target gripping area is defined as the gripping area in the second gripping area where the angle between the plane normal vector and the vertical direction is less than a preset angle threshold, including: The grabbing area in the second grabbing area where the angle between the plane normal vector and the vertical direction is less than a preset angle threshold is determined as the third grabbing area; Obtain the air permeability index of the third gripping area; the air permeability index is used to indicate the air permeability of the third gripping area; The target grasping area is defined as the grasping area whose breathability index is less than the preset breathability index threshold in the third grasping area.

7. The method for identifying stacked materials in a depalletizing robot according to claim 6, characterized in that, Obtaining the air permeability index of the third grasping area includes: Send a first message to the material gripping device, the first message being used to instruct the material gripping device to adsorb the suction cup onto the second gripping area with a preset pressure, and to instruct the material gripping device to detect the change in pressure inside the suction cup within a preset time period through a pressure detection device; The material gripping device receives second information including the change amount, and uses the change amount as the air permeability index of the third gripping area.

8. The method for identifying stacked materials in a depalletizing robot according to claim 1, characterized in that, The quality index of the initial depth information of the sampling points is determined based on the ratio and the variance, including: The product of the ratio and the first preset weight is determined to obtain the first product; The product of the variance and the second preset weight is determined to obtain the second product; the sum of the first preset weight and the second preset weight is 1; The sum of the first product and the second product is used as the quality index of the initial depth information of the sampling point, or the product of the first product and the second product is used as the quality index of the initial depth information of the sampling point.

9. A stacking material identification system for a depalletizing / palletizing robot, characterized in that, The stacking material identification system of the depalletizing robot includes: Acquisition device, used to acquire target depth information of each sampling point in the gripping surface of the material stack; It is also used to obtain the initial depth information of each sampling point in the gripping surface of the material stack; For each of the multiple sampling points on the grasping surface, determine the ratio of the number of first sampling points to the number of second sampling points in the first region. The sampling points are located at the center of the first region, the initial depth information of the first sampling points is within a preset depth range, and the initial depth information of the second sampling points is outside the preset depth range. Determine the variance of the initial depth information of the first sampling point in the first region; The quality index of the initial depth information of the sampling point is determined based on the ratio and the variance. If the quality index is greater than a preset quality index threshold, the initial depth information of the sampling point is determined as the target depth information of the sampling point. A processing device is used to determine a flatness index for each sampling point in the gripping surface based on target depth information of multiple sampling points in the gripping surface; the flatness index is used to indicate the flatness of the sampling point. The processing device is also configured to determine a target grasping area in the grasping surface based on the flatness index of multiple sampling points of the grasping surface; The processing device is also used to generate and send grasping information to the material grasping device; the grasping information is used to instruct the material grasping device to grasp the target grasping area through the grasping part.

Images