3D machine vision processing method and system for carton unstacking

Abstract

The invention provides a 3D machine vision processing method and system for carton unstacking. Comprising the steps that an original point cloud image of a carton to be unstacked is collected through a 3D machine vision system, the original point cloud image is preprocessed, and an industrial robot is controlled through a robot control system to drive a suction cup to execute the unstacking action; the method further comprises the steps that material parameters of cartons to be unstacked are obtained through a material feature library of the 3D machine vision system, surface pollutant features of the cartons to be unstacked are extracted through an image segmentation algorithm of the 3D machine vision system, and the stacking deformation quantity of the cartons to be unstacked is calculated through the 3D machine vision system; constructing a point cloud noise correction model based on the material parameters, the surface pollutant features and the stacking deformation quantity, and performing noise filtering on the preprocessed original point cloud image by using the point cloud noise correction model; the operation precision, safety and reliability of the 3D machine vision unstacking system can be improved.

Description

Technical Field

[0001] This invention relates to the field of 3D vision processing technology, and in particular to a 3D machine vision processing method and system for depalletizing cardboard boxes. Background Technology

[0002] In logistics warehousing, food processing, and other fields, cardboard box depalletizing is a crucial link connecting goods storage and transfer. 3D machine vision technology, due to its ability to quickly acquire cardboard box spatial pose information, has become a core perception method for industrial robots in depalletizing operations. In existing technologies, 3D machine vision systems typically only perform conventional preprocessing such as Gaussian filtering and downsampling on the acquired raw point cloud images to remove simple interference factors such as ambient light and equipment vibration. Then, they directly calculate the industrial robot's grasping pose based on the preprocessed point cloud data and control the suction cup to execute the depalletizing action through fixed parameters. However, existing technologies have significant technical shortcomings: they do not consider the material characteristics of the cardboard boxes to be depalletized, such as the elastic modulus, surface friction coefficient, and the coupling effect of surface contaminants, such as oil stain coverage, water stain thickness, and stacking deformation, on the noise of the 3D point cloud. In actual depalletizing scenarios, the deformation caused by the stacking and compression of cardboard boxes leads to distortion of point cloud morphology, surface oil stains disrupt the uniformity of laser reflection, and water stains easily create specular reflection interference. Furthermore, these three factors do not act independently—for example, under high deformation conditions, the interference of oil stains on laser reflection intensifies nonlinearly. This coupling effect significantly increases point cloud noise errors, and conventional preprocessing methods cannot effectively filter out such coupled noise. These problems directly lead to a significant decrease in the accuracy of subsequent grasping pose calculations, making industrial robots prone to grasping position deviations and even causing cardboard boxes to fall off. Simultaneously, residual noise also makes dynamic risk assessment of personnel lack reliable data support, causing inaccurate timing of safety control actions and severely impacting the stability and safety of depalletizing operations.

[0003] Based on the above problems, there is an urgent need for a technical solution that can solve the problem of incomplete point cloud noise filtering caused by the coupling of material, contaminants and stacking deformation, so as to improve the operation accuracy and safety and reliability of 3D machine vision destacking system. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a 3D machine vision processing method for depalletizing cartons, comprising:

[0005] The original point cloud image of the cardboard box to be destacking is acquired by a 3D machine vision system and the original point cloud image is preprocessed. The industrial robot is controlled by the robot control system to drive the suction cup to perform the destacking action.

[0006] The method further includes:

[0007] The material parameters of the cardboard boxes to be destabilized are obtained through the material feature library of the 3D machine vision system, the surface contaminant features of the cardboard boxes to be destabilized are extracted through the image segmentation algorithm of the 3D machine vision system, and the stacking deformation of the cardboard boxes to be destabilized is calculated through the 3D machine vision system.

[0008] A point cloud noise correction model is constructed based on the material parameters, the surface contaminant characteristics, and the stacking deformation. The point cloud noise correction model is then used to filter noise from the preprocessed original point cloud image.

[0009] The ambient temperature and humidity data of the depalletizing work area are collected by temperature and humidity sensors. The positional offset of the industrial robot grasping the carton to be depalletized is calculated by combining the output results of the point cloud noise correction model. The vacuum adjustment value of the suction cup is calculated by combining the positional offset and the material parameters.

[0010] The system collects dynamic data of personnel in the destacking work area by using a personnel dynamic detection camera. It then constructs a safety stop threshold model by combining the pose offset, the vacuum adjustment value, and the output results of the point cloud noise correction model. The system calculates the risk value of the destacking operation in real time. When the risk value exceeds the output threshold of the safety stop threshold model, the industrial robot is triggered to perform an emergency stop action or a safety avoidance action.

[0011] Preferably, the material parameters include the elastic modulus and surface friction coefficient of the cardboard box to be destabilized, and the surface contaminant characteristics include the oil stain coverage and average water stain thickness on the surface of the cardboard box to be destabilized.

[0012] The step of obtaining the material parameters of the cardboard box to be destacking through the material feature library of the 3D machine vision system includes calculating the similarity between the actual laser reflectivity of the cardboard box to be destacking and the laser reflectivity of each standard material in the material feature library. When the similarity is greater than a preset similarity threshold, the corresponding standard material is determined as the material type of the cardboard box to be destacking. The standard elastic modulus and standard surface friction coefficient corresponding to the standard material are called as the initial material parameters. The initial material parameters are corrected based on the correlation model between the stacking deformation and the material parameters to obtain the final material parameters.

[0013] The step of extracting surface contaminant features of the cardboard boxes to be destabilized using the image segmentation algorithm of the 3D machine vision system includes dividing the surface of the cardboard boxes to be destabilized into oily and non-oily areas using the image segmentation algorithm, calculating the proportion of the oily area to the total surface area of ​​the cardboard boxes to be destabilized as the oil coverage rate, and calculating the average thickness of water stains on the surface of the cardboard boxes to be destabilized using the laser reflection intensity difference of the 3D machine vision system.

[0014] More preferably, the acquisition frequency of the temperature and humidity sensor is synchronized with the point cloud image acquisition frame rate of the 3D machine vision system;

[0015] The step of calculating the pose offset of the industrial robot grasping the cardboard boxes to be destabilized based on the output results of the point cloud noise correction model includes: firstly, calculating the preliminary pose offset of the cardboard boxes to be destabilized based on the output results of the point cloud noise correction model; secondly, calculating the deviation value between the environmental temperature and humidity data and the preset standard temperature and humidity data; thirdly, performing a coupling operation between the deviation value and the output results of the point cloud noise correction model to obtain the environmental compensation component of the pose offset; and fourthly, superimposing the preliminary pose offset and the environmental compensation component to obtain the final pose offset.

[0016] More preferably, the output of the point cloud noise correction model is a point cloud noise correction coefficient K, which is calculated using the following formula:

[0017] ;

[0018] in, The stacking deformation of the cartons to be destabilized is calculated by the deviation between the 3D point cloud image and the standard model of the cartons to be destabilized, and the unit is % . The oil stain coverage rate on the surface of the cardboard boxes to be unpacked is expressed as a percentage, ranging from 0 to 100. The average thickness of water stains on the surface of the cardboard boxes to be unpacked is in mm. The critical thickness of the water stain is the experimental calibration value, in mm. The actual elastic modulus of the cardboard boxes to be unpacked is expressed in MPa. The elastic modulus of the standard material is the experimentally calibrated value, and the unit is MPa. , , , All of these are experimental fitting coefficients, obtained through multiple sets of experiments under different pollution-deformation scenarios.

[0019] More preferably, the pose offset is calculated using the following formula:

[0020] ;

[0021] in, This refers to the point cloud noise correction coefficient; The actual contact area between the suction cup and the cardboard box to be unpacked is calculated from the 3D point cloud image, and the unit is cm². The minimum effective contact area of ​​the suction cup is the experimental calibration value, in cm². Standard vacuum level, experimental calibration value, unit is MPa; The current vacuum level of the suction cup is expressed in MPa. This is the real-time ambient temperature, in °C. The standard ambient temperature is the experimental calibration value, and the unit is °C. Real-time ambient humidity, in %RH; The standard ambient humidity is the experimental calibration value, and the unit is %RH; , , All of these are experimental fitting coefficients, obtained through multiple sets of experiments in different temperature, humidity, and contact area scenarios.

[0022] More preferably, the vacuum adjustment value is calculated using the following formula:

[0023] ;

[0024] in, Standard vacuum level, unit: MPa; The actual weight of the cardboard boxes to be unpacked is obtained by multiplying the volume of the cardboard boxes to be unpacked by the material density calculated from the 3D point cloud image, and the unit is kg; The standard grasp weight is the experimental calibration value, and the unit is kg. The actual surface friction coefficient of the cardboard boxes to be unpacked is dimensionless. is the standard surface friction coefficient, and is the experimentally calibrated value, which is dimensionless; This represents the pose offset, in mm. This refers to the point cloud noise correction coefficient; , , All of these are experimental fitting coefficients, obtained through multiple sets of experiments with different weight-friction coefficient scenarios.

[0025] More preferably, the output threshold of the safety stop threshold model is a safety stop trigger threshold Tstop, which is calculated using the following formula:

[0026] ;

[0027] in, The speed at which personnel enter the unpacking work area is calculated using the frame difference method of images captured by the personnel dynamic detection camera, and the unit is m / s; The angle between the direction of personnel intrusion and the direction of movement of the industrial robot, expressed in degrees. The safe entry speed is the experimental calibration value, and the unit is m / s; The actual weight of the cardboard boxes to be unpacked is in kg. Standard gripping weight, in kg; This represents the pose offset, in mm. The vacuum adjustment value is calculated using the formula described in claim 6, and the unit is MPa. This represents the maximum vacuum adjustment amount, which is the experimental calibration value, and the unit is MPa. is the point cloud noise correction coefficient, which is dimensionless; , , All of these are experimental fitting coefficients, obtained through experiments calibrated using multiple groups of different personnel in dynamic-grabbing parameter scenarios.

[0028] Further preferably, the step of triggering the industrial robot to perform an emergency stop or safety avoidance action includes: a preset emergency threshold; when the risk value exceeds the safety stop trigger threshold but is less than the preset emergency threshold, the industrial robot is triggered to perform a safety avoidance action, and the robot motion controller drives the industrial robot to move away from the direction of personnel intrusion. The moving speed is calculated by relating the moving speed to the risk value through a preset proportional coefficient; the higher the risk value, the higher the moving speed. When the risk value exceeds the preset emergency threshold, the industrial robot is triggered to perform an emergency stop action, and the robot motion controller sends a control signal to the pilot valve. The pilot valve cuts off the passage between the vacuum pump and the suction cup, causing the suction cup to release the cardboard box to be unpacked.

[0029] Further preferably, the material feature library of the 3D machine vision system contains standard parameters for at least three types of cardboard materials and two types of soft packaging materials. The standard parameters include standard elastic modulus, standard surface friction coefficient, and standard laser reflectivity. The preset similarity threshold is obtained through multiple sets of material matching experiments and has a value range of 85%-95%. The step of correcting the initial material parameters based on the association model of stacked deformation and material parameters includes establishing a mapping relationship between stacked deformation and elastic modulus and surface friction coefficient through the association model, inputting the calculated stacked deformation into the association model, outputting the elastic modulus correction amount and surface friction coefficient correction amount, and correcting the standard elastic modulus and standard surface friction coefficient in the initial material parameters respectively to obtain the final elastic modulus and surface friction coefficient.

[0030] A 3D machine vision processing system for depalletizing cartons, applied to any of the 3D machine vision processing methods for depalletizing cartons as described above, includes: a multi-DOE stereo camera, a temperature and humidity sensor, a personnel motion detection camera, an industrial robot, a robot motion controller, a vacuum system, a pilot valve, a vacuum pump, an air compressor, and a processor; the multi-DOE stereo camera is electrically connected to the processor and is used to acquire raw point cloud images of the cartons to be depalletized and transmit them to the processor; the temperature and humidity sensor is electrically connected to the processor and is used to acquire environmental temperature and humidity data of the depalletizing work area and transmit them to the processor; the personnel motion detection camera is electrically connected to the processor and is used to acquire personnel motion data within the depalletizing work area and transmit it to the processor; the processor is electrically connected to the robot motion controller and is used to receive raw point cloud images of the cartons to be depalletized. The point cloud image, the environmental temperature and humidity data, and the personnel dynamic data are used to calculate the pose offset, vacuum adjustment value, and risk value using a preset algorithm, and control commands are sent to the robot motion controller. The robot motion controller is electrically connected to the industrial robot and is used to drive the industrial robot to move according to the control commands. The robot motion controller is also electrically connected to the pilot valve and is used to control the opening and closing of the pilot valve according to the control commands. The pilot valve is connected to the vacuum pump, the air compressor, and the suction cup, respectively. The vacuum pump and the air compressor together constitute the vacuum system, which is used to adjust the vacuum degree of the suction cup through the pilot valve. The processor stores a computer program, and when the processor executes the computer program, it implements the steps of the 3D machine vision processing method for depalletizing cartons as described in any one of claims 1 to 9.

[0031] Technical effects:

[0032] A point cloud noise correction model is constructed that integrates the material parameters of the cardboard boxes to be destabilized, surface contaminant characteristics, and stacking deformation. Furthermore, it calculates the grasping pose offset in conjunction with environmental temperature and humidity, and constructs a safe stopping threshold model by combining personnel dynamics and grasping parameters. This technology precisely addresses the core issue in the background technology where the coupling of material, contaminants, and stacking deformation leads to incomplete 3D point cloud noise filtering, effectively improving point cloud data quality and grasping pose accuracy, preventing cardboard boxes from falling off. Simultaneously, it enables graded triggering of safe actions, ensuring the stability and safety of destabilization operations. Attached Figure Description

[0033] Figure 1 This is a flowchart of the 3D machine vision processing method for depalletizing cartons used in this application.

[0034] Figure 2 This is a connection block diagram of the 3D machine vision processing system for depalletizing cartons used in this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0036] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0037] Traditional technical solutions suffer from the following problems: They only perform simple preprocessing on the acquired raw point cloud images, failing to consider the material differences of the cartons to be destacking, the characteristics of surface contaminants such as oil or water stains, and the coupling effect of deformation caused by stacking and compression on 3D point cloud noise. This results in incomplete point cloud noise filtering and large errors in subsequent grasping pose calculations. Furthermore, they do not adjust grasping parameters based on real-time environmental temperature and humidity data of the destacking area. Fixed poses and suction cup vacuum levels cannot adapt to changes in the characteristics of carton materials under different temperatures and humidity levels, easily leading to grasping slippage. In addition, monitoring of personnel movement is limited to simple judgments of whether intruders have entered, without linking key parameters such as grasping pose accuracy and vacuum level to calculate dynamic risks. Safety control lacks tiered criteria, and the triggering timing for emergency stops or avoidance is inaccurate, posing a collision hazard.

[0038] Based on this, please refer to Figure 1 This embodiment provides a 3D machine vision processing method for depalletizing cartons, including:

[0039] S1: Acquire the original point cloud image of the cardboard box to be destacking through a 3D machine vision system and preprocess the original point cloud image. Then, control the industrial robot to drive the suction cup to perform the destacking action through the robot control system.

[0040] S2: Obtain the material parameters of the cardboard boxes to be destabilized through the material feature library of the 3D machine vision system, extract the surface contaminant features of the cardboard boxes to be destabilized through the image segmentation algorithm of the 3D machine vision system, and calculate the stacking deformation of the cardboard boxes to be destabilized through the 3D machine vision system.

[0041] S3: Construct a point cloud noise correction model based on the material parameters, the surface contaminant characteristics, and the stacking deformation, and use the point cloud noise correction model to filter noise from the preprocessed original point cloud image;

[0042] S4: Collect ambient temperature and humidity data of the depalletizing work area through temperature and humidity sensors, calculate the pose offset of the industrial robot grasping the carton to be depalletized by combining the output results of the point cloud noise correction model, and calculate the vacuum adjustment value of the suction cup by combining the pose offset and the material parameters.

[0043] S5: Collect dynamic data of personnel in the destacking work area through a personnel dynamic detection camera, and construct a safety stop threshold model by combining the pose offset, the vacuum adjustment value and the output result of the point cloud noise correction model. Calculate the risk value of the destacking operation in real time. When the risk value exceeds the output threshold of the safety stop threshold model, trigger the industrial robot to perform an emergency stop action or a safety avoidance action.

[0044] This solution constructs a noise correction model through multi-dimensional parameter coupling, solving the problem of incomplete noise filtering caused by neglecting the coupling effects of material, contaminants, and deformation in traditional preprocessing, thus improving the quality of point cloud data. By dynamically adjusting the pose and vacuum level in conjunction with the ambient temperature and humidity and the noise correction results, it avoids the problem of poor adaptability of fixed parameters and ensures grasping stability. By associating grasping parameters with the dynamic calculation of risk values ​​by personnel, it realizes hierarchical triggering of safety control, reduces the collision risk of human-machine collaborative operations, and forms a closed-loop control of noise reduction, grasping, and safety, adapting to the actual needs of complex depalletizing scenarios.

[0045] Traditional methods for obtaining material parameters of cartons to be destabilized do not consider the impact of deformation generated during the stacking process on key parameters such as the elastic modulus and surface friction coefficient of the material. This results in a large deviation between the obtained material parameters and the actual working conditions, failing to provide accurate data support for subsequent noise correction and grasping planning. When extracting surface contaminant features, the calculation of oil stain coverage lacks clear regional division criteria and relies heavily on subjective judgment or simple threshold segmentation. The average thickness of water stains is even more difficult to measure accurately through conventional methods and is often estimated. This leads to low reliability of contaminant feature data, further exacerbating the errors in point cloud noise filtering and grasping parameter calculation, and affecting the overall accuracy of destabilization operations.

[0046] Based on this, in the 3D machine vision processing method for depalletizing cartons, the material parameters include the elastic modulus and surface friction coefficient of the cartons to be depalletized, and the surface contaminant characteristics include the oil coverage and average water stain thickness on the surface of the cartons to be depalletized. The step of obtaining the material parameters of the cartons to be depalletized from the material feature library of the 3D machine vision system includes calculating the similarity between the actual laser reflectivity of the cartons to be depalletized and the laser reflectivity of each standard material in the material feature library. When the similarity is greater than a preset similarity threshold, the corresponding standard material is determined as the material type of the cartons to be depalletized, and the corresponding standard material is called. The standard elastic modulus and standard surface friction coefficient are used as initial material parameters. The initial material parameters are corrected based on the correlation model between the stacking deformation and the material parameters to obtain the final material parameters. The step of extracting the surface contaminant features of the cardboard boxes to be destabilized using the image segmentation algorithm of the 3D machine vision system includes dividing the surface of the cardboard boxes to be destabilized into oily and non-oily areas using the image segmentation algorithm, calculating the proportion of the oily area to the total surface area of ​​the cardboard boxes to be destabilized as the oil coverage rate, and calculating the average thickness of water stains on the surface of the cardboard boxes to be destabilized using the laser reflection intensity difference of the 3D machine vision system.

[0047] This scheme achieves dynamic calibration of material parameters by combining laser reflectivity similarity matching with stacking deformation correction, avoiding the problem of ignoring the influence of deformation in fixed matching, and making the material parameters more consistent with actual working conditions. By using image segmentation algorithms to clearly define the oil stain area and using the difference in laser reflectivity intensity to quantify the water stain thickness, it solves the drawbacks of subjective and estimated traditional pollutant feature extraction, ensuring the objectivity and accuracy of pollutant feature data. Accurate material parameters and pollutant features provide high-quality input for the subsequent construction of point cloud noise correction models, laying a data foundation for precise control of destacking operations.

[0048] Traditional technical solutions suffer from the following problems: the acquisition frequency of the temperature and humidity sensor is inconsistent with the frame rate of the point cloud image acquisition of the 3D machine vision system, resulting in the inability to synchronize the acquired environmental temperature and humidity data with the point cloud data in the time dimension. This leads to spatiotemporal misalignment and poor compensation effect when performing pose compensation based on environmental data. Furthermore, the calculation logic is fragmented. It only mentions using the output results of the point cloud noise correction model and the environmental temperature and humidity data to calculate the pose offset, but does not clarify the order and logical relationship of the calculations. In particular, it does not explain how to obtain the initial pose offset, and directly performs environmental compensation, resulting in an incomplete calculation process for the pose offset.

[0049] Based on this, in the 3D machine vision processing method for depalletizing cartons, the acquisition frequency of the temperature and humidity sensor is synchronized with the point cloud image acquisition frame rate of the 3D machine vision system; the step of calculating the pose offset of the industrial robot grasping the cartons to be depalletized based on the output results of the point cloud noise correction model includes first calculating the preliminary pose offset of the cartons to be depalletized based on the output results of the point cloud noise correction model, calculating the deviation value between the environmental temperature and humidity data and the preset standard temperature and humidity data, coupling the deviation value with the output results of the point cloud noise correction model to obtain the environmental compensation component of the pose offset, and superimposing the preliminary pose offset with the environmental compensation component to obtain the final pose offset.

[0050] This solution ensures a high degree of spatiotemporal matching between environmental and point cloud data by synchronizing the temperature and humidity acquisition frequency with the point cloud acquisition frame rate, avoiding the compensation misalignment problem caused by traditional asynchronous methods and laying the foundation for accurate compensation. It clarifies the complete logical chain from initial pose offset calculation, environmental deviation value calculation, compensation component coupling to final offset superposition, filling the gaps in traditional calculation logic and making the pose offset calculation process clear and reproducible. Through the coupling compensation of point cloud noise correction results and environmental deviation, it fully considers the combined impact of noise and environmental factors, resulting in more accurate pose offset calculations. This effectively reduces positional deviations during industrial robot grasping, improving the accuracy and stability of grasping.

[0051] Traditional 3D machine vision processing methods employ fixed filtering parameters or single-dimensional noise correction strategies when filtering point cloud noise. They fail to consider the coupling effect between the stacking deformation of the cardboard boxes to be destacking, the surface oil and water stain coverage, and the material's elastic modulus. Stacking deformation leads to morphological distortion of point cloud data, while oil and water stains alter laser reflection characteristics. Materials with different elastic moduli exhibit varying sensitivities to these factors. The combined effect of these factors causes point cloud noise to exhibit non-linear changes. Fixed filtering parameters cannot adapt to complex noise characteristics, resulting in incomplete noise filtering. Furthermore, the lack of quantifiable noise correction coefficients to measure noise levels in different scenarios prevents dynamic adjustment of noise reduction efforts, leading to unstable noise reduction results. Consequently, when calculating the grasping pose based on point cloud data with significant residual noise, errors increase considerably.

[0052] Based on this, in the 3D machine vision processing method for depalletizing cartons, the output of the point cloud noise correction model is the point cloud noise correction coefficient K, which is calculated using the following formula:

[0053] ;

[0054] in, The stacking deformation of the cardboard boxes to be destabilized is calculated by the deviation between the 3D point cloud image and the standard model of the cardboard boxes to be destabilized. The unit is %, which represents the percentage of the deformation caused by stacking and compression of the cardboard boxes relative to their standard shape. The larger the value, the more severe the deformation. The oil stain coverage rate on the surface of the cardboard boxes to be unpacked is expressed in % and ranges from 0 to 100. It represents the proportion of the total area covered by oil stains. The higher the value, the stronger the oil stain interference. The average thickness of water stains on the surface of the cardboard box to be unpacked is in mm. It is the average vertical distance from the surface of the cardboard box to the outermost layer of water stains, reflecting the degree of influence of water stains on laser reflection. The critical thickness of water stains is an experimentally calibrated value in mm. It is a threshold for water stain thickness that has been determined through multiple experiments. Exceeding this threshold will cause a significant decrease in the friction between the suction cup and the cardboard surface, leading to slippage. The actual elastic modulus of the cardboard boxes to be unstacked is expressed in MPa. It is a physical quantity that measures the cardboard material's ability to resist deformation. The larger the value, the harder the material and the stronger its resistance to deformation. The elastic modulus of the standard material is denoted as MPa, and the experimental calibration value is used to compare the rigidity difference between the actual material and the standard material. , , , All of these are experimental fitting coefficients, obtained through multiple sets of experiments with different pollution-deformation scenarios. They are dimensionless and are used to balance the influence weights of oil stain coverage, water stain thickness, and material elastic modulus on the K value, ensuring that the K value can accurately reflect the overall noise level.

[0055] The core function of this formula is to quantify the coupled influence of stacking deformation, surface contaminants, and material rigidity of the cardboard boxes to be destabilized on 3D point cloud noise, and output a point cloud noise correction coefficient K. This provides a precise adjustment basis for subsequent point cloud noise filtering, solving the technical problem that traditional single-dimensional noise reduction cannot adapt to complex noise characteristics. Based on this formula, parameters are collected and the K value is calculated using existing equipment, and then the noise filtering step is implemented. The formula as a whole adopts the product form of the deformation influence term, the contaminant influence term, and the material rigidity influence term to achieve the synergistic coupling of the three key factors, rather than a simple superposition. The rationality of this structural design is that point cloud noise is a nonlinear result of the combined effect of the three factors. The change of a single factor will change the degree of influence of other factors on noise. For example, under the condition of high deformation ε, the interference of oil stains σ on laser reflectivity will be more significant. The product form can accurately reflect this synergistic effect. Where (1-ε) is the stacking deformation effect term, and ε is the stacking deformation, which is calculated by the deviation between the 3D point cloud and the standard model. This formula reflects the logic that the greater the deformation, the smaller the noise correction coefficient. When the cardboard box undergoes a large deformation due to stacking and squeezing, the morphological distortion of the point cloud data is aggravated. Even if the interference of pollutants is filtered out, the correction coefficient still needs to be reduced to adapt to the noise caused by the deformation, so as to avoid excessive correction and loss of point cloud information. For the comprehensive impact of pollutants, the effects of oil pollution and water stains are summed. Because oil pollution and water stains interfere with laser reflection differently—oil pollution causes uneven reflectivity, while water stains create specular reflection—they need to be quantified separately and then superimposed. The impact of oil pollution is represented by an exponential function. This is because the impact of oil stain coverage σ on noise increases non-linearly; once σ exceeds 50%, the noise growth rate accelerates. An exponential form can accurately fit this characteristic, with coefficients a and b used to balance the weighting and growth rate of the oil stain impact; the impact of water stains is assessed using... This is because when the thickness δ of the water stain reaches a critical value... At this point, the impact of water stains on noise tends to saturate. This linear fraction visually demonstrates the relationship that the thicker the water stain, the smaller the correction coefficient. The coefficient c is used to balance the proportion of the impact of water stains and oil stains. Finally, E represents the modulus of elasticity of the cardboard box, which is influenced by the material's rigidity. γ is the standard elastic modulus. This formula reflects the logic that the more rigid the material, the larger the correction coefficient. Rigid materials are more resistant to deformation and contaminants, and point cloud noise is less affected by both. Therefore, the correction coefficient needs to be increased to retain more effective point cloud information. γ is the exponential coefficient, which is used to adjust the sensitivity of the material rigidity to the correction coefficient, so as to avoid large fluctuations in the correction coefficient due to small differences in the material and ensure calculation stability.

[0056] This solution quantifies the coupling effects of stacking deformation, oil stains, water stains, and the material's elastic modulus into a noise correction coefficient K using a formula. This solves the problem that traditional single-dimensional noise reduction cannot adapt to complex noise characteristics. The K value changes dynamically with each influencing factor, providing a precise basis for noise filtering and avoiding incomplete or excessive noise reduction caused by fixed filtering parameters. Noise filtering based on the K value can specifically remove coupled noise in different scenarios, outputting high-quality point cloud data, providing reliable input for subsequent pose offset calculations, and reducing grasping errors caused by point cloud noise.

[0057] Traditional industrial robots calculate pose offsets when grasping cardboard boxes without incorporating noise correction results from 3D point clouds. This ignores the impact of noise on determining the actual contact area of ​​the suction cup and assessing the current vacuum level, resulting in pose offset calculations that do not consider point cloud data quality and are disconnected from the actual grasping scenario. Furthermore, relying solely on empirical values ​​to set pose adjustment amounts fails to account for the impact of deviations in ambient temperature and humidity in the depalletizing area on the cardboard material's expansion and contraction, and surface properties. Excessively high or low temperatures cause thermal expansion and contraction, while humidity changes alter the surface friction coefficient. These factors all affect the accuracy of the grasping pose, but traditional methods do not include them in the calculation. Consequently, the pose offset cannot adapt to environmental changes, leading to positional deviations or even grasping failures when the industrial robot grasps according to the calculated results.

[0058] Based on this, in the 3D machine vision processing method for depalletizing cartons, the pose offset is calculated using the following formula:

[0059] ;

[0060] in, K is the point cloud noise correction coefficient, which is calculated by the formula described in claim 4. It is dimensionless and reflects the influence of point cloud data quality on the assessment of contact area and vacuum degree. The larger the K value, the smaller the point cloud noise and the more reliable the assessment result. The actual contact area between the suction cup and the cardboard box to be unpacked is calculated using 3D point cloud images, with the unit being cm². This area is the actual area where the suction cup and the cardboard box surface are in contact, which directly affects the stability of the force during gripping. The minimum effective contact area of ​​the suction cup is the experimentally calibrated value in cm², which is the minimum contact area threshold that has been experimentally verified to ensure gripping without detachment. The standard vacuum level is the experimental calibration value, and the unit is MPa. It is the optimal vacuum pressure value under standard operating conditions, with standard temperature and humidity and no contaminants to ensure stable grasping. The current vacuum level of the suction cup is expressed in MPa. It is the real-time detected vacuum pressure inside the suction cup, reflecting the magnitude of the current gripping force. The real-time ambient temperature is expressed in °C, which is the actual air temperature in the depalletizing work area and affects the degree of thermal expansion and contraction of the carton. The standard ambient temperature is the experimental calibration value, and the unit is °C. It is the standard operating temperature used as the temperature reference. The real-time ambient humidity is expressed in %RH, representing the ratio of the current water vapor content in the air to the saturated water vapor content at the same temperature, which affects the coefficient of friction of the cardboard box surface. The standard ambient humidity is the experimental calibration value, and the unit is %RH. It is the standard operating condition humidity used as the humidity reference. , , All of these are experimental fitting coefficients, obtained through multiple sets of experiments with different temperature and humidity-contact area scenarios. They are dimensionless and are used to adjust the contribution of contact area, vacuum degree, and temperature and humidity deviation to the pose offset, ensuring that the calculation results fit the actual scenario.

[0061] The formula is divided into two parts: point cloud quality correlation term and temperature and humidity compensation term. The summation is used because the two have independent mechanisms of influence on pose offset: point cloud quality affects the evaluation accuracy of the grasping state, and temperature and humidity affect the physical properties of the material. They need to be calculated separately and then superimposed to achieve comprehensive compensation.

[0062] The term "point cloud quality correlation" refers to the multiplicative effect of the K-point cloud noise correction coefficient, which binds point cloud data quality to the pose calculation depth. When the K value is small, the point cloud noise is large. Even if the grasping state parameters are relatively ideal in terms of contact area and vacuum degree, the calculation weight of pose offset needs to be reduced to avoid over-adjustment of pose due to point cloud errors. The term "grasping state influence" in parentheses is a sum of the influence of contact area and vacuum degree, as both directly determine grasping stability and must be included in the pose calculation. The influence of contact area is expressed as a logarithmic function. This is because the relationship between the contact area S and the pose accuracy is non-linear. When S approaches the minimum effective contact area Smin, even a small deviation in pose can cause the gripper to detach. This effect needs to be logarithmically amplified, and the coefficient d is used to balance the contribution of the contact area to the pose offset. The vacuum degree effect is... This is because the ratio of the current vacuum level Pv to the standard vacuum level Pv0 directly reflects the sufficiency of the gripping force—the closer Pv is to Pv0, the more stable the gripping force is, and the lower the requirement for pose accuracy. Therefore, the smaller this ratio is, the smaller the calculated value of the pose offset is. The coefficient e is used to adjust the weight of the influence of vacuum level.

[0063] For the temperature and humidity compensation term, the product of temperature deviation T-T0 and humidity deviation H-H0 is adopted because the effects of temperature and humidity on material expansion and contraction have a synergistic effect. For example, when high temperature T>T0 and high humidity H>H0 are superimposed, the expansion and contraction of the cardboard material is much greater than the effect of a single factor. The product form can accurately fit this synergistic characteristic. The coefficient f is used to balance the proportion of temperature and humidity compensation term and point cloud quality correlation term, so as to avoid the distortion of pose offset calculation due to drastic environmental changes and ensure that even under extreme temperature and humidity, pose adjustment can still conform to the actual material characteristics.

[0064] This solution incorporates the point cloud noise correction coefficient K into the pose offset calculation, thus establishing a correlation between point cloud data quality and pose adjustment, avoiding calculation errors caused by neglecting data quality in traditional methods. By coupling the calculation of temperature and humidity deviations, it quantifies the impact of environmental changes on carton characteristics, filling the gap in traditional methods that do not consider environmental factors. All parameters in the formula are based on measurable or calibrated actual working conditions, with clear calculation logic. It can dynamically output pose offsets that adapt to point cloud quality, contact area, vacuum degree, and ambient temperature and humidity. Industrial robots can adjust their gripping positions accordingly, significantly reducing positional deviations and improving gripping success rate and stability.

[0065] Traditional suction cup vacuum adjustment methods for depalletizing cartons have significant technical flaws: they set the vacuum level solely based on fixed gripping weights or empirical values, failing to consider the differences between the actual and standard weights of the cartons to be depalletized, and the impact of variations in surface friction coefficients on gripping force requirements. This leads to a mismatch between the vacuum level and actual gripping conditions—insufficient vacuum can easily cause detachment when the weight is large or the friction coefficient is low. Furthermore, they neglect the synergistic effect of gripping accuracy (reflected by pose offset) and data quality (reflected by point cloud noise correction coefficients) on vacuum level. Using high vacuum levels when pose accuracy and data quality are high wastes energy, while failing to specifically increase vacuum levels when pose accuracy and data quality are low further exacerbates gripping risks. In addition, the calculation of vacuum level adjustment values ​​lacks quantitative formula support, relying heavily on manual adjustments, resulting in low efficiency and poor consistency, making it difficult to adapt to batch depalletizing scenarios.

[0066] Based on this, in the 3D machine vision processing method for depalletizing cartons, the vacuum adjustment value is calculated using the following formula:

[0067] ;

[0068] in, The standard vacuum degree, measured in MPa, is the reference vacuum pressure value that ensures stable gripping under standard working conditions, such as standard weight, friction coefficient, and positional accuracy. It was obtained through multiple sets of experiments. The actual weight of the cardboard boxes to be unpacked is calculated by multiplying the volume of the cardboard box and the material density using a 3D point cloud image. The unit is kg, which directly determines the basic vacuum force required for gripping. The standard grasp weight, in kg, serves as the experimental calibration value for weight comparison. The actual surface friction coefficient of the cardboard box to be destabilized is dimensionless and reflects the magnitude of the frictional force between the cardboard box surface and the suction cup. The smaller the value, the higher the vacuum compensation required. is the standard surface friction coefficient, dimensionless, and is the reference value of the friction coefficient under standard materials; The position offset is calculated using the formula described in claim 5, and the unit is mm. The smaller the value, the more accurate the grasping pose and the lower the dependence on vacuum. The point cloud noise correction coefficient is calculated using the formula described in claim 4. It is dimensionless, and the larger the value, the better the point cloud data quality and the more reliable the basis for vacuum degree calculation. , , All of these are experimental fitting coefficients, calibrated through multiple sets of experiments with different weight-friction coefficient scenarios. They are dimensionless and are used to balance the influence weight of weight, pose offset, and point cloud noise on the vacuum degree adjustment value, ensuring that the calculation results closely match actual grasping requirements.

[0069] The formula adopts the structure of standard vacuum degree minus deviation compensation term. The core logic is: standard vacuum degree Pv0 is the optimal value under ideal working conditions. However, in actual working conditions, deviations in weight, friction, pose accuracy, and point cloud quality will cause the ideal vacuum degree to no longer be suitable. The actual vacuum degree adjustment amount needs to be calculated through deviation compensation term. The larger the value of deviation compensation term, the greater the deviation between actual working conditions and ideal working conditions. More needs to be deducted from the standard vacuum degree to achieve accurate adaptation of vacuum degree.

[0070] The deviation compensation term is a sum of the effects of weight, friction, pose accuracy, and point cloud quality. Since these three factors have different requirements for vacuum level, they need to be quantified separately and then superimposed. The weight-friction synergistic effect term is expressed as the product of their ratios because the vacuum required for gripping is positively correlated with weight and negatively correlated with the coefficient of friction. When the actual weight m of the carton is greater than the standard weight m0, then... Furthermore, the actual coefficient of friction μ is less than the standard coefficient of friction μ0. When the product of the two factors is used, the demand for vacuum will be amplified, and the value of the compensation term needs to be increased to increase the vacuum. The coefficient g is used to balance the weight of this synergistic effect and avoid excessive adjustment of vacuum due to extreme deviations in weight or friction.

[0071] The pose accuracy factor is represented by ΔP, which is the pose offset. It is used in reciprocal form because pose accuracy is negatively correlated with vacuum requirements. The smaller ΔP is, the more precise the pose, the more stable the contact between the suction cup and the cardboard box, and the lower the dependence on vacuum. The smaller the value, the smaller the compensation term value, and the closer the vacuum degree adjustment is to the standard value; the coefficient h is used to adjust the degree of influence of pose accuracy on vacuum degree, ensuring that even if there is a slight deviation in pose, vacuum degree compensation can ensure stable grasping.

[0072] K is the point cloud quality influence term, which is the point cloud noise correction coefficient. The linear product form is used because the point cloud quality directly affects the evaluation accuracy of parameters such as weight, friction, and pose. The larger the K value, the lower the point cloud noise, the more reliable the evaluation result, and the more accurate the basis for vacuum degree adjustment. Therefore, it is necessary to reduce the value of the compensation term to make the vacuum degree closer to the standard value. The coefficient i is used to balance the influence of point cloud quality on vacuum degree and avoid vacuum degree adjustment inaccuracy due to point cloud error.

[0073] This scheme incorporates the combined effects of weight, friction coefficient, pose offset, and point cloud noise into the calculation of vacuum degree adjustment value through a quantitative formula, thus solving the problem of poor adaptability of traditional fixed value or single-factor adjustment.

[0074] Traditional safety control methods for depalletizing operations have serious technical limitations: they rely solely on simple infrared sensors to detect whether personnel have entered the depalletizing work area, failing to consider the varying impacts of personnel entry speed and direction, as well as the angle between the entry direction and the industrial robot's movement direction, on collision risk—the risk is far higher when personnel enter at high speed in the same direction than when they enter at low speed in the opposite direction, yet traditional methods trigger the same safety action in both cases, leading to distorted risk assessment. Furthermore, they fail to consider key grasping parameters such as the actual weight of the cartons to be depalletized, the grasping posture offset, the suction cup vacuum adjustment value, and the point cloud noise correction coefficient. When the cartons are heavy, the posture accuracy is low, and the vacuum is insufficient, even slow entry poses a high collision risk, yet the safety level is not specifically improved. Conversely, when the cartons are light, the posture is accurate, and the vacuum is sufficient, excessive triggering of safety actions reduces depalletizing efficiency. In addition, there is a lack of quantified safety stop thresholds; the timing of safety action triggers depends entirely on sensor switch signals, lacking a tiered basis and failing to balance safety and efficiency.

[0075] Based on this, in the 3D machine vision processing method for depalletizing cartons, the output threshold of the safety stop threshold model is the safety stop trigger threshold Tstop, which is calculated using the following formula:

[0076] ;

[0077] in, The speed at which personnel enter the destabilization work area is calculated using the frame difference method of images captured by personnel dynamic detection cameras, with the unit being m / s. The greater the speed, the greater the collision kinetic energy and the higher the risk. The angle between the direction of personnel intrusion and the direction of movement of the industrial robot, expressed in degrees. The value increases as the included angle decreases; the risk is highest when entering from the same direction at θ=0°, and lowest when entering from the opposite direction at θ=180°. The safe intrusion speed, measured in m / s, is the experimentally calibrated, permissible threshold for the movement speed of low-risk personnel. The actual weight of the cardboard boxes to be unpacked is in kg. The greater the weight, the greater the impact force during a collision, and the higher the risk. The standard gripping weight, in kg, serves as the benchmark for weight risk assessment. This represents the pose offset in mm. A smaller value indicates a more accurate pose, stronger controllability of the robot's motion trajectory, and lower risk. The vacuum degree adjustment value is calculated using the formula described in claim 6, and the unit is MPa. The larger the value, the closer the vacuum degree is to the optimal value, the better the grasping stability, and the lower the risk. This is the maximum vacuum adjustment amount, in MPa, which is the upper limit of the maximum adjustable vacuum range determined by the experiment. This is the point cloud noise correction coefficient, which is dimensionless. The larger the value, the better the point cloud data quality, the higher the calculation accuracy of each parameter, and the more reliable the risk assessment. , , All are experimental fitting coefficients, obtained through multiple sets of experiments with different personnel dynamics and grasping parameter scenarios. They are dimensionless, and the influence of personnel dynamics, pose accuracy, vacuum degree and data quality on the safety threshold are adjusted to ensure that the threshold can accurately quantify the collision risk.

[0078] The formula as a whole adopts the sum of personnel dynamics, weight coordination risk, pose accuracy risk, and grasping stability risk. Since the three factors determine the safety risks of depalletizing operations from different dimensions, they need to be quantified separately and then superimposed to achieve a comprehensive risk assessment: personnel dynamics determine the potential kinetic energy of the collision, weight determines the impact force of the collision, pose accuracy determines the controllability of the robot's movement, and grasping stability determines whether the goods fall off during the collision. The four factors work together to form a complete risk assessment system.

[0079] For the risk factor related to personnel dynamics and weight, the product of the ratio of relative velocity to weight is used. The core is to quantify the synergistic impact of the kinetic energy collision force of personnel intrusion. Here, v represents the ratio of the relative velocities of the personnel, v is the velocity of the personnel entering the robot, and θ is the angle between the direction of entry and the direction of the robot's movement. The velocity of a person is decomposed into a component in the same direction as the direction of the robot's motion. The smaller θ is, the larger the component in the same direction is, the greater the relative velocity is, and the higher the risk of collision. v0 is the safe intrusion velocity. The larger this ratio is, the higher the potential kinetic energy of the person intruding. The ratio is the weight ratio, where m is the actual weight of the carton and m0 is the standard gripping weight. The larger the ratio, the greater the impact force during the collision. The product of the two can accurately fit the high-risk scenario of high-speed, same-direction intrusion into a heavily loaded vehicle. The coefficient j is used to balance the weight of this collaborative risk item and avoid the distortion of risk assessment caused by small changes in personnel speed or weight.

[0080] The pose accuracy risk term is represented by ΔP, which is the pose offset. It is used in reciprocal form because pose accuracy and safety risk are negatively correlated. The smaller ΔP is, the stronger the controllability of the robot's trajectory. Even if personnel approach, the robot can avoid them by fine-tuning the trajectory, thus reducing the risk. The smaller the value, the smaller the risk item value; the coefficient k is used to adjust the degree of influence of pose accuracy on risk assessment, ensuring that the risk threshold can be significantly improved when the pose deviation is large, and safety actions can be triggered in a timely manner.

[0081] To capture stability risks, a product of vacuum sufficiency and point cloud quality reliability is used. The core is to quantify the indirect impact of the capture state on safety risks. To indicate the degree of vacuum adequacy, ΔPv is the vacuum adjustment value, and ΔPvmax is the maximum vacuum adjustment amount. The larger the value, the closer the vacuum level is to the optimal value, and the more stable the grasping. Therefore, the smaller the value of this formula, the lower the risk. For point cloud quality reliability, K is the point cloud noise correction coefficient. The larger K is, the more reliable the evaluation of the grasping parameters and the more guaranteed the grasping stability. Therefore, the smaller the value of this formula, the lower the risk. The product of the two can accurately reflect the high-risk scenario of insufficient vacuum degree and large point cloud error. At this time, the product value is large and the risk term value is large. The coefficient l is used to balance the weight of the risk term, ensuring that the risk threshold can be increased in time when the grasping is unstable, thus ensuring the safety of operation.

[0082] This solution constructs a safe stopping threshold through a multi-parameter coupled quantitative formula, solving the problem that traditional methods cannot distinguish risk levels by the presence or absence of detection. It deeply correlates personnel dynamics with grasping parameters to achieve accurate risk classification, avoiding efficiency losses or safety hazards caused by excessive safety actions or risk omissions. All parameters in the formula are derived from real-time detection or experimental calibration, resulting in strong real-time performance and high accuracy in risk calculation. This provides an objective basis for triggering safe actions of industrial robots, balancing the safety and efficiency of destacking operations.

[0083] Traditional depalletizing operations suffer from significant technical shortcomings in the safety motion control of industrial robots: Emergency stop and safety avoidance actions are not triggered in a tiered manner; regardless of the risk of personnel intrusion, an emergency stop is uniformly executed, leading to frequent interruptions in low-risk scenarios and a substantial decrease in operational efficiency. Furthermore, when triggering safety avoidance actions, the robot's movement speed is not clearly correlated with the risk level; a fixed speed is used for avoidance. Insufficient avoidance speed at high risk levels can easily cause collisions, while excessive avoidance speed at low risk levels results in energy waste and mechanical wear. Additionally, when executing an emergency stop, only the robot's movement is stopped; the suction cups and vacuum pump are not simultaneously disconnected. If the vacuum level is insufficient at this point, cartons may still fall off after stopping, causing damage to goods or secondary safety hazards. Moreover, the signal interaction logic between the robot's motion controller and the pilot valve is not clearly defined, resulting in poor coordination of action execution.

[0084] Based on this, in the 3D machine vision processing method for depalletizing cartons, the step of triggering the industrial robot to perform an emergency stop or safety avoidance action includes a preset emergency threshold. When the risk value exceeds the safety stop trigger threshold but is less than the preset emergency threshold, the industrial robot is triggered to perform a safety avoidance action. The robot motion controller drives the industrial robot to move away from the direction of personnel intrusion. The moving speed is calculated by relating the risk value to a preset proportional coefficient. The higher the risk value, the higher the moving speed. When the risk value exceeds the preset emergency threshold, the industrial robot is triggered to perform an emergency stop action. The robot motion controller sends a control signal to the pilot valve. The pilot valve cuts off the passage between the vacuum pump and the suction cup, causing the suction cup to release the cartons to be depalletized.

[0085] This solution achieves graded triggering of safety actions by pre-setting emergency thresholds, solving the efficiency loss problem caused by traditional single stop actions. Avoidance actions in low-risk scenarios can ensure the continuity of operations. By linking avoidance speed with risk value through a proportional coefficient, it ensures that there is appropriate avoidance efficiency under different risk levels, balancing safety and energy consumption control. During emergency stop, the pilot valve is simultaneously controlled to cut off the vacuum pump path to avoid the risk of carton falling off. The signal interaction logic between the robot motion controller and the pilot valve is clearly defined, that is, the controller sends control signals to drive the pilot valve to act, realizing the coordination of stop and depressurization, ensuring the feasibility of the technical solution. The overall safety action control logic is clear, and the various hardware and software steps are closely coordinated, which not only improves the safety of depalletizing operations but also ensures operational efficiency, adapting to the needs of industrial mass production.

[0086] Traditional 3D machine vision systems suffer from technical flaws in their material feature library construction and material parameter correction methods. The material feature library contains only standard parameters for a single or a few types of cardboard materials, failing to cover a wide range of common cardboard and soft-pack materials. This results in a low material matching success rate when dealing with diverse depalletized goods. Furthermore, the preset similarity threshold lacks a clear experimental calibration range, relying heavily on experience. An excessively high threshold can lead to no matching materials, while an excessively low threshold can result in false matches, affecting the accuracy of subsequent material parameters. In addition, the correction of initial material parameters lacks specific correlation model support, simply adjusting values ​​without establishing a mapping relationship between stacking deformation and elastic modulus and surface friction coefficient. This fails to quantify the impact of deformation on material parameters, leading to significant deviations between the corrected material parameters and actual working conditions. Consequently, it cannot provide reliable data support for subsequent steps such as point cloud noise correction and vacuum degree adjustment, thus affecting overall depalletizing accuracy.

[0087] Based on this, in the 3D machine vision processing method for depalletizing cardboard boxes, the material feature library of the 3D machine vision system contains standard parameters for at least three cardboard box materials and two flexible packaging materials. The standard parameters include standard elastic modulus, standard surface friction coefficient, and standard laser reflectivity. The preset similarity threshold is obtained through multiple sets of material matching experiments, with a value range of 85%-95%. The step of correcting the initial material parameters based on the association model of stacking deformation and material parameters includes establishing a mapping relationship between stacking deformation and elastic modulus and surface friction coefficient through the association model, inputting the calculated stacking deformation into the association model, outputting the elastic modulus correction amount and surface friction coefficient correction amount, and correcting the standard elastic modulus and standard surface friction coefficient in the initial material parameters respectively to obtain the final elastic modulus and surface friction coefficient.

[0088] This solution expands the material feature library to cover various cardboard and flexible packaging materials, solving the problem of low matching success rate caused by insufficient library capacity in traditional methods, and adapting to diverse depalletizing scenarios. It clarifies the experimental calibration range of the similarity threshold to 85%-95%, avoiding the subjectivity of empirical settings. Those skilled in the art can determine the specific threshold through experiments based on this range, ensuring matching accuracy. By establishing a mapping relationship between stacking deformation and material parameters through a correlation model, the solution quantifies the impact of deformation on material parameters and outputs specific correction amounts, solving the problems of traditional corrections lacking basis and having large deviations. The accuracy of the corrected material parameters is significantly improved, providing high-quality input data for point cloud noise correction models, vacuum degree adjustment formulas, etc., further ensuring the accuracy and stability of depalletizing operations.

[0089] Traditional 3D machine vision processing systems for depalletizing cardboard boxes suffer from severe ambiguity in hardware module interactions: they merely list core hardware components such as multi-DOE stereo cameras, temperature and humidity sensors, and processors, without clearly defining the specific electrical connections and signal interaction methods between these components. For example, how the raw point cloud images acquired by the multi-DOE stereo cameras are transmitted to the processor, and the communication link between the temperature and humidity sensors and the processor, are not explained, resulting in an unclear physical structure of the hardware system. Furthermore, the specific functions of each hardware module are not defined individually. For instance, the output data type of the personnel motion detection camera and the control command interaction logic between the robot motion controller and the industrial robot are unclear, creating a gap in the functional-hardware correspondence. In addition, the composition and control logic of the vacuum system are not explained; only the vacuum pump and suction cup are mentioned, without clarifying the role of the air compressor in the vacuum system, how the pilot valve adjusts the vacuum level of the suction cup, and the connection between the software-level computer program and the hardware collaboration—that is, how the processor calls the hardware to execute the aforementioned steps through the program—resulting in a lack of software-hardware collaborative support for the entire system.

[0090] Based on this, please refer to Figure 2 The 3D machine vision processing system for depalletizing cartons includes a multi-DOE stereo camera, a temperature and humidity sensor, a personnel motion detection camera, an industrial robot, a robot motion controller, a vacuum system, a pilot valve, a vacuum pump, an air compressor, and a processor. The multi-DOE stereo camera is electrically connected to the processor and is used to acquire original point cloud images of the cartons to be depalletized and transmit them to the processor. The temperature and humidity sensor is electrically connected to the processor and is used to acquire environmental temperature and humidity data of the depalletizing work area and transmit them to the processor. The personnel motion detection camera is electrically connected to the processor and is used to acquire personnel motion data of the depalletizing work area and transmit it to the processor. The processor is electrically connected to the robot motion controller and is used to receive the original point cloud images and the environmental temperature and humidity data. Based on the personnel dynamic data, a preset algorithm is executed to calculate the pose offset, vacuum adjustment value, and risk value, and control commands are sent to the robot motion controller. The robot motion controller is electrically connected to the industrial robot and is used to drive the industrial robot to move according to the control commands. The robot motion controller is electrically connected to the pilot valve and is used to control the opening and closing of the pilot valve according to the control commands. The pilot valve is connected to the vacuum pump, the air compressor, and the suction cup, respectively. The vacuum pump and the air compressor together constitute the vacuum system, which is used to adjust the vacuum degree of the suction cup through the pilot valve. The processor stores a computer program, and when the processor executes the computer program, it implements any of the steps of the 3D machine vision processing method for depalletizing cartons.

[0091] This solution addresses the ambiguity of hardware connections in traditional systems by clearly defining the electrical connections and signal flow of each hardware module, thus constructing a clear physical structure framework. It defines the specific functions of each module, such as the personnel dynamic detection camera collecting personnel dynamic data, establishing a clear correspondence between hardware and function to avoid functional gaps. It clarifies the composition of the vacuum system and the control logic of the pilot valve, and connects the collaborative relationship between the processor program and hardware execution, i.e., the program drives the hardware to execute the steps described above, achieving software-hardware synergy. The entire system solution is fully disclosed, and the interaction logic of each module is clear.

[0092] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A 3D machine vision processing method for depalletizing cartons, comprising: The original point cloud image of the cardboard box to be destacking is acquired by a 3D machine vision system and the original point cloud image is preprocessed. The industrial robot is controlled by the robot control system to drive the suction cup to perform the destacking action. The method is characterized in that it further includes: The material parameters of the cardboard boxes to be destabilized are obtained through the material feature library of the 3D machine vision system, the surface contaminant features of the cardboard boxes to be destabilized are extracted through the image segmentation algorithm of the 3D machine vision system, and the stacking deformation of the cardboard boxes to be destabilized is calculated through the 3D machine vision system. A point cloud noise correction model is constructed based on the material parameters, the surface contaminant characteristics, and the stacking deformation. The point cloud noise correction model is then used to filter noise from the preprocessed original point cloud image. The ambient temperature and humidity data of the depalletizing work area are collected by temperature and humidity sensors. The positional offset of the industrial robot grasping the carton to be depalletized is calculated by combining the output results of the point cloud noise correction model. The vacuum adjustment value of the suction cup is calculated by combining the positional offset and the material parameters. The system collects dynamic data of personnel in the destacking work area by using a personnel dynamic detection camera. It then constructs a safety stop threshold model by combining the pose offset, the vacuum adjustment value, and the output results of the point cloud noise correction model. The system calculates the risk value of the destacking operation in real time. When the risk value exceeds the output threshold of the safety stop threshold model, the industrial robot is triggered to perform an emergency stop action or a safety avoidance action.

2. The 3D machine vision processing method for depalletizing cartons according to claim 1, characterized in that, The material parameters include the elastic modulus and surface friction coefficient of the cardboard boxes to be destabilized, and the surface contaminant characteristics include the oil coverage and average thickness of water stains on the surface of the cardboard boxes to be destabilized. The step of obtaining the material parameters of the cardboard box to be destacking through the material feature library of the 3D machine vision system includes calculating the similarity between the actual laser reflectivity of the cardboard box to be destacking and the laser reflectivity of each standard material in the material feature library. When the similarity is greater than a preset similarity threshold, the corresponding standard material is determined as the material type of the cardboard box to be destacking. The standard elastic modulus and standard surface friction coefficient corresponding to the standard material are called as the initial material parameters. The initial material parameters are corrected based on the correlation model between the stacking deformation and the material parameters to obtain the final material parameters. The step of extracting surface contaminant features of the cardboard boxes to be destabilized using the image segmentation algorithm of the 3D machine vision system includes dividing the surface of the cardboard boxes to be destabilized into oily and non-oily areas using the image segmentation algorithm, calculating the proportion of the oily area to the total surface area of ​​the cardboard boxes to be destabilized as the oil coverage rate, and calculating the average thickness of water stains on the surface of the cardboard boxes to be destabilized using the laser reflection intensity difference of the 3D machine vision system.

3. The 3D machine vision processing method for depalletizing cartons according to claim 1, characterized in that, The temperature and humidity sensor's acquisition frequency is synchronized with the point cloud image acquisition frame rate of the 3D machine vision system. The step of calculating the pose offset of the industrial robot grasping the cardboard boxes to be destabilized based on the output results of the point cloud noise correction model includes: firstly, calculating the preliminary pose offset of the cardboard boxes to be destabilized based on the output results of the point cloud noise correction model; secondly, calculating the deviation value between the environmental temperature and humidity data and the preset standard temperature and humidity data; thirdly, performing a coupling operation between the deviation value and the output results of the point cloud noise correction model to obtain the environmental compensation component of the pose offset; and fourthly, superimposing the preliminary pose offset and the environmental compensation component to obtain the final pose offset.

4. The 3D machine vision processing method for depalletizing cartons according to claim 1, characterized in that, The output of the point cloud noise correction model is the point cloud noise correction coefficient K, which is calculated using the following formula: ； in, The stacking deformation of the cartons to be destabilized is calculated by the deviation between the 3D point cloud image and the standard model of the cartons to be destabilized, and the unit is % . The oil stain coverage rate on the surface of the cardboard boxes to be unpacked is expressed as a percentage, ranging from 0 to 100. The average thickness of water stains on the surface of the cardboard boxes to be unpacked is in mm. The critical thickness of the water stain is the experimental calibration value, in mm. The actual elastic modulus of the cardboard boxes to be unpacked is expressed in MPa. The elastic modulus of the standard material is the experimentally calibrated value, and the unit is MPa. , , , All of these are experimental fitting coefficients, obtained through multiple sets of experiments under different pollution-deformation scenarios.

5. The 3D machine vision processing method for depalletizing cartons according to claim 1, characterized in that, The pose offset is calculated using the following formula: ； in, This refers to the point cloud noise correction coefficient; The actual contact area between the suction cup and the cardboard box to be unpacked is calculated from the 3D point cloud image, and the unit is cm². The minimum effective contact area of ​​the suction cup is the experimental calibration value, in cm². Standard vacuum level, experimental calibration value, unit is MPa; The current vacuum level of the suction cup is expressed in MPa. This is the real-time ambient temperature, in °C. The standard ambient temperature is the experimental calibration value, and the unit is °C. Real-time ambient humidity, in %RH; The standard ambient humidity is the experimental calibration value, and the unit is %RH; , , All of these are experimental fitting coefficients, obtained through multiple sets of experiments in different temperature, humidity, and contact area scenarios.

6. The 3D machine vision processing method for depalletizing cartons according to claim 1, characterized in that, The vacuum adjustment value is calculated using the following formula: ； in, Standard vacuum level, unit: MPa; The actual weight of the cardboard boxes to be unpacked is obtained by multiplying the volume of the cardboard boxes to be unpacked by the material density calculated from the 3D point cloud image, and the unit is kg; The standard grasp weight is the experimental calibration value, and the unit is kg. The actual surface friction coefficient of the cardboard boxes to be unpacked is dimensionless. is the standard surface friction coefficient, and is the experimentally calibrated value, which is dimensionless; This represents the pose offset, in mm. This refers to the point cloud noise correction coefficient; , , All of these are experimental fitting coefficients, obtained through multiple sets of experiments with different weight-friction coefficient scenarios.

7. The 3D machine vision processing method for depalletizing cartons according to claim 1, characterized in that, The output threshold of the safe stop threshold model is the safe stop trigger threshold Tstop, which is calculated using the following formula: ； in, The speed at which personnel enter the unpacking work area is calculated using the frame difference method of images captured by the personnel dynamic detection camera, and the unit is m / s; The angle between the direction of personnel intrusion and the direction of movement of the industrial robot, expressed in degrees. The safe entry speed is the experimental calibration value, and the unit is m / s; The actual weight of the cardboard boxes to be unpacked is in kg. Standard gripping weight, in kg; This represents the pose offset, in mm. The vacuum adjustment value is calculated using the formula described in claim 6, and the unit is MPa. This represents the maximum vacuum adjustment amount, which is the experimental calibration value, and the unit is MPa. is the point cloud noise correction coefficient, which is dimensionless; , , All of these are experimental fitting coefficients, obtained through experiments calibrated using multiple groups of different personnel in dynamic-grabbing parameter scenarios.

8. The 3D machine vision processing method for depalletizing cartons according to claim 1, characterized in that, The steps for triggering the industrial robot to perform an emergency stop or safety avoidance action include: 1) Setting a preset emergency threshold; 2) When the risk value exceeds the safety stop trigger threshold but is less than the preset emergency threshold, the industrial robot is triggered to perform a safety avoidance action. The robot motion controller drives the industrial robot to move away from the direction of personnel intrusion. The moving speed is calculated by relating the moving speed to the risk value using a preset proportional coefficient; the higher the risk value, the higher the moving speed. 3) When the risk value exceeds the preset emergency threshold, the industrial robot is triggered to perform an emergency stop action. The robot motion controller sends a control signal to a pilot valve, which cuts off the passage between the vacuum pump and the suction cup, causing the suction cup to release the cardboard boxes to be unpacked.

9. A 3D machine vision processing method for depalletizing cardboard boxes according to claim 2, characterized in that, The material feature library of the 3D machine vision system contains standard parameters for at least three types of cardboard materials and two types of soft packaging materials. The standard parameters include standard elastic modulus, standard surface friction coefficient, and standard laser reflectivity. The preset similarity threshold is obtained through multiple sets of material matching experiments and ranges from 85% to 95%. The step of correcting the initial material parameters based on the association model of stacked deformation and material parameters includes establishing a mapping relationship between stacked deformation and elastic modulus and surface friction coefficient through the association model, inputting the calculated stacked deformation into the association model, outputting the elastic modulus correction amount and surface friction coefficient correction amount, and correcting the standard elastic modulus and standard surface friction coefficient in the initial material parameters respectively to obtain the final elastic modulus and surface friction coefficient.

10. A 3D machine vision processing system for depalletizing cartons, applied to the 3D machine vision processing method for depalletizing cartons as described in any one of claims 1-9, characterized in that, include: The system includes a multi-DOE stereo camera, a temperature and humidity sensor, a personnel motion detection camera, an industrial robot, a robot motion controller, a vacuum system, a pilot valve, a vacuum pump, an air compressor, and a processor. The multi-DOE stereo camera is electrically connected to the processor and is used to acquire and transmit raw point cloud images of the cartons to be destacking to the processor. The temperature and humidity sensor is electrically connected to the processor and is used to acquire and transmit environmental temperature and humidity data of the destacking work area to the processor. The personnel motion detection camera is electrically connected to the processor and is used to acquire and transmit personnel motion data of the personnel within the destacking work area to the processor. The processor is electrically connected to the robot motion controller and is used to receive the raw point cloud images, the environmental temperature and humidity data, and the personnel motion data, and perform pre-processing. The algorithm calculates the pose offset, vacuum adjustment value, and risk value, and sends control commands to the robot motion controller. The robot motion controller is electrically connected to the industrial robot and drives the industrial robot to move according to the control commands. The robot motion controller is also electrically connected to the pilot valve and controls the opening and closing of the pilot valve according to the control commands. The pilot valve is connected to the vacuum pump, the air compressor, and the suction cup, respectively. The vacuum pump and the air compressor together constitute the vacuum system, which is used to adjust the vacuum degree of the suction cup through the pilot valve. The processor stores a computer program, and when the processor executes the computer program, it implements the steps of the 3D machine vision processing method for depalletizing cartons as described in any one of claims 1 to 9.

Images