Machine vision-based intelligent control system and device for robot warehousing and unstacking

Abstract

The application discloses a mechanical hand warehousing intelligent unstacking control system and device based on machine vision and relates to the technical field of image data processing.The mechanical hand warehousing intelligent unstacking control system based on machine vision comprises the following modules: a mechanical hand warehousing clamping monitoring module, a mechanical hand clamping effect quantification module and a mechanical hand clamping force control module.The application monitors the influence of the mechanical hand warehousing on point cloud data when the mechanical hand warehousing intelligently unstacks a specified soft packaging box stack, selects whether to take a soft packaging influence compensation measure, then quantifies the clamping effect of the mechanical hand clamping the specified soft packaging box stack to determine whether to perform clamping point correction adjustment, and finally performs qualification judgment on the successful rate of clamping to determine whether to take a clamping force control adjustment strategy, thereby improving the accuracy of the mechanical hand unstacking control, and solving the problem that the existing technology has low mechanical hand unstacking control accuracy due to the interference of soft packaging boxes on machine vision scanning results.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a machine vision-based intelligent depalletizing and palletizing control system and device for robotic arms entering warehouses. Background Technology

[0002] Existing intelligent palletizing / depalletizing systems based on machine vision are fusion systems of multiple technologies, including 3D (Three-Dimensional) visual perception (point cloud combined with depth map), AI (Artificial Intelligence of Things), and others. The system integrates AI (Artificial Intelligence) for intelligent recognition and attitude estimation, gripping point planning combined with path optimization, palletizing / depalletizing strategies and task management systems, force control combined with fault tolerance mechanisms, and system control integration with industrial communication. It uses 3D visual perception to collect point clouds and images of the stacking area, reconstructs the surface of goods, accurately locates the center, attitude, and gripping surface of objects, and identifies the type of goods (boxes, parcels, bags). AI intelligent recognition and attitude estimation selects the optimal gripping surface from multiple possible gripping surfaces (anti-occlusion, high stability), accurately identifying gripping points even for irregularly shaped, soft-packaged, and reflective objects, and automatically adjusting the attitude of the robot's end effector. Gripping point planning combined with path optimization dynamically plans the robot's path based on gripping points and target points, avoiding obstacles (palletized goods, conveyor belts, shelves, etc.) and controlling the robot's compliant movement to achieve precise gripping and placement. The palletizing / depalletizing strategy and task management system generates the optimal depalletizing order (e.g., from top to bottom, from outside to inside) and manages FIFO (First In, First Out) / LIFO (Last In, First Out) inbound / outbound operations. The system follows a Last-In-First-Out (LIFO) sequence and also tracks the status of goods: location, whether they have been moved, and which shelf they are placed on. Force control combined with a fault-tolerant mechanism not only prevents deformation or falling when grasping soft packaging or irregular objects, but also implements a "stop upon contact" mechanism to avoid collisions, and senses grasping failures / slippages in real time to trigger re-grabbing logic. The system control integration combined with industrial communication is used to interface with conveyor lines, automated warehouses, warehouse management systems (WMS) / warehouse control systems (WCS), and multi-robot collaborative scheduling.

[0003] For example, Chinese invention patent CN110322457B discloses a destacking method combining 2D and 3D vision, including a stereo vision device and an intermediate camera. The destacking method includes: Step 1, using the stereo vision device to acquire 3D point cloud data of the stack, and determining the height distribution map of the stack based on the 3D point cloud data; Step 2, using the intermediate camera to acquire a 2D image of the upper surface of the stack, and using an edge detection algorithm to determine the edge line of the object in the 2D image; Step 3, determining the center coordinates of the object based on the positional relationship of the edge line in the 2D image, and generating grasping information for the robotic arm to grasp the object based on the center coordinates, the vertical height of the center coordinates in the height distribution map, and the 3D point cloud data.

[0004] For example, Chinese invention patent CN111524184B discloses an intelligent depalletizing method and system based on 3D vision, comprising: S1, acquiring image information of the stack of boxes and performing preprocessing; S2, obtaining the top layer area of ​​the stacked boxes based on the preprocessed image information and setting the depalletizing order; S3, obtaining candidate frames containing at least one box according to the set depalletizing order, and determining the placement posture of the box to be grasped using both traditional image processing methods and deep learning methods; S4, determining the placement posture of the box to be grasped in the candidate frames by combining the judgment results of the two methods; S5, calculating the rotation angle of the grasping robot and controlling the robot to grasp the box and depalletize it.

[0005] The above-mentioned technology has at least the following technical problems:

[0006] In existing technologies, 3D vision can provide shape and posture information, but gripper grasping is still prone to failure for soft packaging, easily deformable or uneven surfaces. Because the surface of soft packaging objects may have irregular shapes such as wrinkles, bulges, and deformations, the generated point cloud data is not smooth and discontinuous, making the point cloud "jump" or sparse. The surface normals and gripping positions given are unreliable. Moreover, soft objects will deform due to force after being placed, and the "posture" seen by the vision system may be an instantaneous state. The identified center position, normal vector, and edge information cannot truly reflect its gripability, resulting in the inability to accurately control the gripping force or the inappropriate selection of gripping points. There is a problem that the soft packaging material interferes with the machine vision scanning results, leading to low accuracy of the robot's depalletizing and palletizing control. Summary of the Invention

[0007] To address the technical problem of low accuracy in robotic arm depalletizing and palletizing control caused by interference from flexible packaging materials in existing technologies, this application provides a machine vision-based intelligent depalletizing and palletizing control system and device for robotic arms in warehousing. The technical solution is as follows:

[0008] On one hand, a machine vision-based intelligent depalletizing and palletizing control system for robotic arms entering the warehouse is provided. This system includes: a robotic arm entry gripping monitoring module, a robotic arm gripping effect quantification module, and a robotic arm gripping force control module. The robotic arm entry gripping monitoring module monitors the impact of designated flexible packaging stacks on point cloud data in real time during intelligent depalletizing and palletizing of these stacks to determine whether to implement flexible packaging impact compensation measures. The robotic arm gripping effect quantification module quantifies the gripping effect of the robotic arm on the designated flexible packaging stacks to determine whether corresponding gripping point correction adjustments are needed. This gripping point correction adjustment improves the accuracy of the robotic arm's gripping point selection for the designated flexible packaging stacks. The robotic arm gripping force control module assesses the success rate of gripping within a preset gripping cycle to determine whether to implement a gripping force control adjustment strategy. This strategy improves the accuracy of the gripping force applied by the robotic arm to the designated flexible packaging stacks.

[0009] On the other hand, a machine vision-based intelligent depalletizing and palletizing control device for robotic arms in warehousing is provided. This device, which utilizes a machine vision-based intelligent depalletizing and palletizing control system for robotic arms in warehousing, includes: a visual perception structure, a robotic arm and gripping structure, and a control decision module. The visual perception structure mainly includes an industrial camera mounted on the end effector of the robotic arm, a light source system to assist visual recognition, and a visual computing unit for image processing and point cloud analysis. The robotic arm and gripping structure is a multi-degree-of-freedom industrial robot used to support depalletizing, rotation, and handling actions. It is equipped with gripping detection sensors to determine whether gripping is successful, and various grippers, specifically including vacuum suction cup grippers, electric grippers, and pneumatic grippers. The control decision module runs visual recognition and gripping path planning algorithms, working in conjunction with the robot body to control the robotic arm's movements.

[0010] The beneficial effects of the technical solutions provided in this application include at least the following:

[0011] 1. When the robotic arm intelligently unpalletizes designated flexible packaging pallets during warehousing, the impact of the designated flexible packaging pallets on the point cloud data is monitored in real time to determine whether to take measures to compensate for the impact of flexible packaging, thereby reducing the interference of flexible packaging characteristics on the machine vision. Then, the gripping effect of the robotic arm in grasping the designated flexible packaging pallets is quantified to determine whether to perform corresponding gripping point correction and adjustment, ensuring that the robotic arm can select gripping points more accurately. Finally, the success rate of gripping within the preset gripping cycle is judged to determine whether to take gripping force control adjustment strategy, thereby applying more precise gripping force to the designated flexible packaging pallets, and thus improving the accuracy of the robotic arm's unpalletizing control.

[0012] 2. First, the influence ratio and preset depth value of the flexible packaging point cloud are obtained from the preset database, providing a data foundation for subsequent data analysis. The deviation between the depth value in the Z direction of the point cloud and the preset depth value is quantified to obtain the corresponding depth deviation influence quantity. The deviation degree of the depth value in the Z direction of the point cloud is initially quantified. Then, the depth deviation influence quantity, point cloud curvature, point cloud jump frequency, and point cloud overlap rate are coupled with the corresponding influence ratio of the flexible packaging point cloud after weighted operation. Finally, the point cloud data-pinch influence quantity is obtained, which quantifies the degree of interference of flexible packaging characteristics on machine vision recognition. This helps to take corresponding measures in a timely manner to improve the accuracy of machine vision recognition.

[0013] 3. First, the gripping effect analysis data is acquired in real time and normalized to ensure consistent data dimensions. Then, the gripping effect analysis weights are extracted from the preset database to accurately analyze the influence of each gripping effect analysis data on the quantified value of the gripping effect. Finally, the gripping effect analysis data and the corresponding gripping effect analysis weights are weighted and coupled to obtain the quantified value of the gripping effect. This accurately quantifies the accuracy of the gripping point of the specified soft packaging box stack selected by the robot arm, and allows for timely implementation of corresponding measures to ensure the accuracy of the selected gripping point. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the intelligent depalletizing and palletizing control system for robotic arms in warehousing provided in this application embodiment;

[0016] Figure 2 This is a flowchart illustrating the process of selecting whether to take measures to compensate for the impact of flexible packaging, provided in an embodiment of this application.

[0017] Figure 3 This is a flowchart illustrating the process of determining whether to perform corresponding gripping point correction adjustment, provided in an embodiment of this application.

[0018] Figure 4 This is a schematic diagram of the interface for controlling the gripping parameters of the robotic arm in the intelligent depalletizing and palletizing platform provided in this application embodiment;

[0019] Figure 5 This is a schematic diagram of the interface for adjusting and limiting the clamping force of the robotic arm in the intelligent depalletizing and palletizing platform provided in this application embodiment. Detailed Implementation

[0020] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0021] In the embodiments of this application, words such as "exemplarily" and "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design scheme described as "exemplary" in this invention should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in the embodiments of this application, the meaning expressed by "and / or" can be both, or it can be either one or the other.

[0022] In the embodiments of this application, sometimes the subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0023] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0024] This application provides a machine vision-based intelligent depalletizing and palletizing control system for robotic arms used in warehousing. For example... Figure 1 The schematic diagram shown is of the structure of the intelligent depalletizing and palletizing control system for robotic arms in warehousing provided in the embodiment of this application. The processing flow of the system includes the following modules: robotic arm warehousing gripping monitoring module, robotic arm gripping effect quantification module, and robotic arm gripping force control module.

[0025] Among them, the robotic arm inbound gripping monitoring module is used to monitor the impact of the specified soft packaging box stacks on the point cloud data in real time when the robotic arm performs intelligent unpacking and palletizing of the specified soft packaging box stacks in the warehouse, so as to select whether to take soft packaging impact compensation measures. The soft packaging impact compensation measures are used to improve the accuracy of machine vision in recognizing the specified soft packaging box stacks.

[0026] The robotic arm gripping effect quantification module is used to quantify the gripping effect of the robotic arm in grasping a specified soft packaging stack to determine whether corresponding gripping point correction adjustment is needed. This improves the accuracy of the gripping action from the source to reduce the gripping failure rate. The gripping point correction adjustment is used to improve the accuracy of the robotic arm in selecting the gripping point of the specified soft packaging stack.

[0027] The robotic arm gripping force control module is used to determine the success rate of gripping within a preset gripping cycle to determine whether to adopt a gripping force control adjustment strategy. It realizes intelligent matching of the optimal gripping force to prevent gripping failure or damage to items caused by insufficient or excessive gripping force. The gripping force control adjustment strategy is used to improve the accuracy of the gripping force applied by the robotic arm to the specified soft packaging box stack.

[0028] In this embodiment, from gripping monitoring to effect evaluation and gripping force adjustment, the system establishes a closed-loop control process for the robotic arm's gripping decision-making and execution, effectively improving the efficiency and automation level of depalletizing and palletizing operations. Furthermore, addressing the characteristics of flexible packaging pallets being prone to deformation and displacement, a point cloud data monitoring and impact compensation mechanism enables real-time response to environmental disturbances and physical changes, ensuring the stability of the robotic arm's gripping operation. Through the collaborative work of various modules, continuous optimization of operational behavior in the two key dimensions of gripping point and gripping force helps improve the success rate of the robotic arm's gripping within the preset gripping cycle, enhances operational continuity and system stability, and ensures the accuracy of the robotic arm's depalletizing and palletizing control.

[0029] It's important to note that before designing a machine vision-based robotic arm intelligent palletizing and depalletizing control system for warehousing, technical professionals typically prioritize building a pre-set database of parameters to support the execution of various control strategies. This database compiles several key control parameters, including: point cloud-grip judgment value, the influence ratio of flexible packaging point cloud, preset depth value, camera exposure time adjustment, gripping effect judgment value, gripping effect analysis weight, gripping point correction, preset success rate, preset gripping force adjustment, and gripping force adjustment influence rate. These parameters are all pre-set by engineers with specialized technical backgrounds based on the adopted algorithm model and actual hardware configuration. This database provides crucial data support for subsequent automated processes such as data uploading, storage optimization, filtering, and decision-making.

[0030] like Figure 2 The diagram shown is a flowchart illustrating the process of selecting whether to take flexible packaging impact compensation measures according to an embodiment of this application. The specific logic is as follows: First, real-time acquisition of the impact data of a specified flexible packaging stack on the point cloud data; then, acquisition of the point cloud data-clamping impact quantity based on the flexible packaging point cloud impact data, and comparison of the point cloud data-clamping impact quantity with the point cloud-clamping judgment quantity. If the point cloud data-clamping impact quantity is less than the point cloud-clamping judgment quantity, the specified flexible packaging stack continues to be captured; otherwise, the difference between the point cloud data-clamping impact quantity and the point cloud-clamping judgment quantity is recorded as the clamping impact difference quantity. The clamping impact difference quantity is mapped in a preset database to obtain the camera exposure. The machine vision system adjusts the camera exposure time by multiplying the initial camera exposure time with the adjusted camera exposure time to obtain an optimized camera exposure time. This optimized exposure time is then used to rescan the specified flexible packaging stack until the point cloud data-grip influence is less than the point cloud-grip judgment value. Finally, the number of rescans is obtained in real-time and compared with a preset number. If the number of rescans is less than the preset number, the robot continues to grasp the specified flexible packaging stack; otherwise, the robot stops grasping the stack and issues a gripping anomaly warning. This process not only improves the accuracy of gripping point recognition but also increases the efficiency of the robot's operation.

[0031] Furthermore, the impact of designated flexible packaging pallets on point cloud data is monitored in real time to determine whether to implement flexible packaging impact compensation measures. The specific steps are as follows:

[0032] Step 1: Real-time acquisition of the impact data of the specified flexible packaging pallet on the point cloud data. The impact data of the flexible packaging point cloud includes the point cloud curvature, the point cloud depth value in the Z direction, the point cloud jump frequency, and the point cloud overlap rate.

[0033] Specifically, the curvature of the point cloud is obtained by estimating the point cloud normals, such as using NormalEstimation from the PCL (Programmable Logic Controller System) library, combined with local curvature calculations (such as the eigenvalue method); the 3D camera directly outputs the point cloud coordinates (x, y, z), where z is the depth value of the point cloud in the Z direction; the point cloud jump frequency is the ratio of the number of outliers to the total number of points; the point cloud overlap rate represents the degree of overlap between objects in space (especially in projection), which is obtained through PCL voxel grid (3D).

[0034] Step 2: Obtain the point cloud data-grip influence based on the point cloud influence data of flexible packaging, and compare the point cloud data-grip influence with the point cloud-grip judgment value obtained from the preset database. If the point cloud data-grip influence is less than the point cloud-grip judgment value, continue to grab the specified flexible packaging box stack; otherwise, take flexible packaging influence compensation measures. The point cloud data-grip influence is used to quantify the degree of interference of flexible packaging characteristics on machine vision recognition.

[0035] Specifically, the point cloud-grip judgment quantity is obtained from the preset database. The preset staff inputs the point cloud influence data of soft packaging with qualified gripping effect from the historical data into the specific constraint expression of the point cloud data-grip influence quantity to obtain the dataset of point cloud data-grip influence quantity. The average value of the dataset is recorded as the point cloud-grip judgment quantity.

[0036] In this embodiment, based on multi-dimensional data such as point cloud curvature, point cloud Z-direction depth value, point cloud jump frequency, and point cloud overlap rate, the impact of flexible packaging characteristics on point cloud data quality is evaluated more comprehensively, which helps to improve the stability and accuracy of gripping point recognition. Furthermore, by calculating the point cloud data-grip influence in real time and comparing it with the corresponding point cloud-grip judgment quantity, intelligent response to different flexible packaging scenarios is realized, enhancing the system's adaptability in complex material environments. Moreover, when the influence of flexible packaging characteristics on machine vision exceeds the point cloud-grip judgment quantity, an automatic compensation mechanism is triggered, which helps to avoid misgrabbing or gripping failure caused by point cloud interference, thereby improving the overall efficiency and reliability of the robot's operation.

[0037] As a further embodiment, the specific method for obtaining the point cloud data-grip influence is as follows:

[0038] First, the influence ratio and preset depth value of the soft packaging point cloud are obtained from the preset database. The influence ratio of the soft packaging point cloud includes the influence ratio of point cloud curvature, the influence ratio of point cloud depth value in the Z direction, the influence ratio of point cloud jump frequency, and the influence ratio of point cloud overlap rate.

[0039] It should be added that the system inputs the collected soft packaging point cloud impact data into a pre-trained soft packaging point cloud impact mapping model to output the corresponding soft packaging point cloud impact ratio, i.e., the degree of influence of this type of data on the point cloud data-grip impact. This mapping model is built based on the logistic regression algorithm and relies on the least squares criterion, completing the training process through the statsmodels framework. The training data used by the model comes from the soft packaging point cloud impact characteristics collected within historical time periods, and the reference soft packaging point cloud impact ratio set by professional technicians based on empirical rules, used to fit the mapping relationship between the two, thereby accurately reflecting the degree of influence of soft packaging point cloud characteristics on actual gripping tasks. Furthermore, the preset depth value is preset by preset staff based on the changing patterns of historical data and operational standards and stored in a preset database.

[0040] Next, the deviation between the depth value in the Z direction of the point cloud and the preset depth value is quantified to obtain the corresponding depth deviation influence. Here, the deviation quantization means that the ratio of the depth value in the Z direction of the point cloud to the preset depth value is calculated.

[0041] Finally, the depth deviation influence, point cloud curvature, point cloud jump frequency, and point cloud overlap rate are weighted and coupled with the corresponding soft packaging point cloud influence ratio. That is, the corresponding data is multiplied by the corresponding influence ratio and then summed to obtain the point cloud data-pinch influence.

[0042] In this embodiment, the algorithm combines the influence of depth deviation, point cloud curvature, point cloud jump frequency, and point cloud overlap rate with the corresponding influence ratio of flexible packaging point cloud to obtain the point cloud data-grip influence. In this formula, as the point cloud curvature, point cloud jump frequency, and point cloud overlap rate increase, the influence of flexible packaging characteristics on the point cloud data becomes higher, leading to a decrease in the accuracy of the robot arm's gripping of the specified flexible packaging stack. A larger depth deviation influence indicates a higher degree of deviation of the point cloud's Z-direction depth value from the preset depth value. As the depth deviation influence increases, the corresponding point cloud data-grip influence also increases, indicating that the higher the influence of flexible packaging characteristics on the point cloud data, the higher its gripping influence on the robot arm. Accurately quantifying the influence of flexible packaging characteristics on the point cloud data through the point cloud data-grip influence helps to make timely judgments based on the point cloud data-grip influence, thereby taking corresponding optimization measures and improving the accuracy of the robot arm's control over the gripping of the specified flexible packaging stack.

[0043] For example, assuming the influence ratio of point cloud curvature is 0.4, the influence ratio of point cloud depth in the Z direction is 0.3, the influence ratio of point cloud jump frequency is 0.2, and the influence ratio of point cloud overlap rate is 0.1, with a preset depth value of 1.2m, when the point cloud curvature is 0.01, the point cloud depth in the Z direction is 1.2m, the point cloud jump frequency is 0.5Hz, and the point cloud overlap rate decreases from 0.85 to 0.3, it indicates that the overlap of point clouds between two frames or two viewpoints has decreased. This may be caused by scanning angle offset, increased occlusion, or sensor movement, resulting in target "deformation" or incomplete point clouds, thus affecting the robot's grasping ability. When the point cloud curvature is 0.02, the point cloud depth in the Z direction increases from 1.2m to 3m, the point cloud jump frequency is 1Hz, and the point cloud overlap rate is 0.9, it indicates that the target is moving away from the sensor, the point cloud density decreases, the spatial resolution deteriorates, and edge artifacts are more likely to occur. Unclear information or loss of details leads to inaccurate point cloud data analysis, which in turn affects the gripping control of the robotic arm. When the point cloud curvature increases from 0.01 to 0.12, while the point cloud depth in the Z direction is 1.5m, the point cloud jump frequency is 0.3Hz, and the point cloud overlap rate remains constant at 0.85, it indicates that the target surface has become more curved or sharper, and features such as edges, gaps, and holes may be identified, which helps in geometric segmentation or edge extraction, thus affecting the gripping effect of the robotic arm. When the point cloud jump frequency increases from 0.2Hz to 3Hz, while the point cloud curvature is 0.02, the point cloud depth in the Z direction is 1.2m, and the point cloud overlap rate remains constant at 0.9, it indicates drastic changes between point cloud frames, which may be due to sensor jitter, rapid target movement, or software registration errors, affecting the dynamic tracking stability of the target, and thus affecting the gripping control of the robotic arm.

[0044] It should be noted that the specific process for implementing compensation measures for the impact of flexible packaging is as follows:

[0045] A1, the difference between point cloud data - gripping influence quantity and point cloud - gripping judgment quantity is recorded as gripping influence difference quantity. The gripping influence difference quantity is mapped in the preset database to obtain the camera exposure time adjustment quantity.

[0046] Specifically, the pinch effect difference is input into the pre-built camera exposure time mapping table in the preset database, and the corresponding camera exposure time adjustment is output. The camera exposure time mapping table is used to fit the mapping relationship between the pinch effect difference and the camera exposure time adjustment, and the value of the camera exposure time adjustment ranges from 0 to 1.

[0047] A2, the optimized camera exposure time is obtained by multiplying the initial camera exposure time of the machine vision based on the camera exposure time adjustment.

[0048] A3, rescans the specified soft packaging stacks using the optimized camera exposure time until the point cloud data-grip influence is less than the point cloud-grip judgment amount.

[0049] A4. The system acquires the number of rescans in real time and compares it with the preset number to prevent the system from getting stuck in an invalid identification loop and to ensure the overall operation rhythm. If the number of rescans is lower than the preset number, the system continues to grab the specified soft packaging box stack; otherwise, the robot stops grabbing the specified soft packaging box stack and issues an abnormal robot grabbing warning. This helps prevent system misoperation or equipment damage and enhances the safety and stability of system operation. The preset number is set by the preset staff based on historical data, work experience, and operation standards, and is stored in a preset database in advance for direct retrieval when needed.

[0050] In this embodiment, by calculating the difference in gripping influence and mapping it to the camera exposure time adjustment, the camera exposure parameters are precisely controlled, enabling the acquisition of high-quality point cloud data even under lighting interference or the influence of flexible packaging materials, thus improving the accuracy of visual recognition. Furthermore, by constructing a closed-loop process of "recognition → judgment → exposure adjustment → re-recognition," the system ensures that it can automatically optimize perception conditions based on real-time feedback, improving its adaptability to different flexible packaging scenarios. Moreover, by optimizing visual perception, the gripping decision is ensured to be based on reliable data, which helps reduce abnormal situations such as misjudgment, misgrabbing, and missed grabbing caused by flexible packaging, thereby improving the success rate of the robot's operation.

[0051] like Figure 3The diagram shown is a flowchart illustrating the process for determining whether to perform corresponding gripping point correction adjustment according to an embodiment of this application. The specific logic is as follows: First, the gripping effect analysis data of the specified soft packaging stack acquired in real time is analyzed to obtain a quantified value of the gripping effect. Then, the quantified value of the gripping effect is compared with the judgment value of the gripping effect. If the quantified value of the gripping effect is greater than the judgment value of the gripping effect, the specified soft packaging stack is continued to be gripped. Otherwise, the difference between the quantified value of the gripping effect and the judgment value of the gripping effect is calculated to obtain a gripping effect difference. Then, the gripping effect difference is mapped in a preset database to obtain a gripping point correction amount. Finally, the gripping point correction amount is used to compensate for the deviation data of the robot arm to improve the accuracy of the robot arm in selecting the gripping point of the specified soft packaging stack. Through the above process, not only is the effectiveness of the robot arm's selection of gripping points more accurately determined, but the adaptability of the robot arm in the soft packaging application scenario is also improved.

[0052] like Figure 4 The diagram shows the interface for controlling the gripping parameters of the robotic arm in the intelligent depalletizing and palletizing platform provided in this application embodiment. The left side of the interface is the platform's navigation bar, which includes the main control panel, robotic arm monitoring, robotic arm monitoring, pallet monitoring, task configuration, system configuration, and alarms and logs. The top of the interface displays motion control, including gripping test, placement test, single-step execution, loop execution, and emergency stop. The central part of the interface contains the display interface for various dimensions of gripping parameter control, specifically gripper settings, gripping point settings, gripping point correction, and candidate gripping point options. This interface not only allows the robotic arm user to understand the robotic arm's operation in real time, but also helps the user adjust the various control parameters of the robotic arm to improve the accuracy of the robotic arm's depalletizing and palletizing control.

[0053] Furthermore, the gripping effect of the robotic arm on a specified stack of soft packaging boxes is quantified to determine whether corresponding gripping point correction adjustments are needed. The specific steps are as follows:

[0054] S1, analyze the real-time acquired gripping effect data of the specified soft packaging box stack to obtain a quantitative value of the gripping effect, thereby quantifying the accuracy of the gripping point of the specified soft packaging box stack selected by the robot arm.

[0055] S2, compare the quantized value of the gripping effect with the judgment value of the gripping effect obtained from the preset database. If the quantized value of the gripping effect is greater than the judgment value of the gripping effect, continue to grab the specified soft packaging box stack; otherwise, perform the corresponding gripping point correction adjustment.

[0056] Specifically, the clamping effect analysis data corresponding to the qualified clamping effect in the historical data is input into the specific expression of the clamping effect quantification value to obtain a dataset of clamping effect quantification values. The mean of the dataset is then calculated to obtain the clamping effect judgment value, which is then stored in a preset database in advance.

[0057] In this embodiment, by conducting real-time quantitative evaluation of the gripping effect, the effectiveness of the robotic arm's selected gripping points is more accurately determined, and deviations caused by deformation of flexible packaging materials or visual errors are avoided. Simultaneously, the system automatically determines whether gripping point correction is needed based on the gripping results, forming a self-circulating optimization mechanism of "gripping → evaluation → correction → re-gripping," effectively improving the overall gripping reliability and intelligence level. Furthermore, when the gripping effect is substandard, the system can quickly respond and proactively adjust the gripping strategy to adapt to the uncertainty of flexible packaging materials, improving the robotic arm's fault tolerance and stability in complex environments. It also avoids repeated gripping or misoperation caused by improper gripping point selection, ensuring a high gripping success rate while reducing ineffective operations and improving the overall line operating efficiency.

[0058] As a further solution, the specific method for obtaining the quantification value of the pinch effect is as follows:

[0059] The first step is to acquire gripping effect analysis data in real time and perform data normalization processing. The gripping effect analysis data includes gripping point-point cloud density, number of point cloud noise points, and point cloud hole ratio. The data normalization processing is specifically the minimum-maximum normalization method.

[0060] It should be added that the grip point-point cloud density is the ratio of the number of point clouds in a fixed-size neighborhood centered on the grip point to the corresponding object volume; the number of noise points in the point cloud is obtained by statistical filtering; the point cloud of the target region is projected onto a two-dimensional grid plane (such as the XY or XZ plane), and the space is divided into grids of fixed size and marked with points and no points. The ratio of the number of grids without points to the total number of grids is the point cloud hole ratio.

[0061] The second step is to extract the weighting of the gripping effect analysis from the preset database. The weighting of the gripping effect analysis includes the weighting of the gripping point-point cloud density analysis, the weighting of the number of noise points in the point cloud analysis, and the weighting of the proportion of holes in the point cloud analysis.

[0062] It should be added that the system inputs the gripping effect analysis data into a pre-trained gripping effect analysis mapping model to output the corresponding gripping effect analysis weights, i.e., quantifying the degree of influence of each gripping effect analysis data point on the quantified gripping effect value. The gripping effect analysis mapping model is built based on the logistic regression algorithm, uses the cross-entropy loss function as the optimization criterion, and is trained using the scikit-learn framework. The training data comes from gripping effect analysis data collected in historical periods, as well as gripping effect analysis weights set by professionals based on experience. The aim is to fit the correspondence between the gripping effect analysis data and the quantified gripping effect value, thereby achieving accurate modeling and evaluation of the degree of influence.

[0063] The third step involves coupling the clamping effect analysis data with the corresponding clamping effect analysis weights after weighted calculations. Specifically, this involves obtaining the product of the clamping point-point cloud density and the clamping point-point cloud density analysis weight, the product of the reciprocal of the number of noise points in the point cloud and the number of noise points in the point cloud, and the product of the reciprocal of the point cloud hole ratio and the point cloud hole ratio analysis weight. These three results are then added together to obtain the quantified clamping effect value. To avoid the number of noise points and the point cloud hole ratio being zero, 1 is added to the denominator during the reciprocal processing. That is, when the corresponding data approaches 0, the entire reciprocal approaches 1, ensuring the validity of the fraction.

[0064] In this embodiment, the algorithm combines the gripping effect analysis data with the corresponding gripping effect analysis weight to obtain a quantified value of the gripping effect. In the formula, when the gripping point-point cloud density increases, it indicates a lower deviation of the gripping point selected by the robot arm, resulting in a more accurate gripping point selection and a better gripping effect; the quantified value of the gripping effect then increases accordingly. Conversely, as the number of noise points and the proportion of holes in the point cloud increase, it indicates a greater deviation in the selection of the gripping point, resulting in a worse gripping effect; the quantified value of the gripping effect then decreases accordingly. By analyzing the quantified value of the gripping effect, the accuracy of the gripping point selected by the robot arm for the specified soft packaging stack is not only more accurately quantified, but the gripping effect of the robot arm is also reflected indirectly. This helps to take corresponding optimization measures for the gripping point in a timely manner, thereby improving the accuracy of the gripping point selected by the robot arm for the specified soft packaging stack, and thus improving the accuracy of the robot arm's grasping control of the specified soft packaging stack, ensuring the gripping effect of the robot arm.

[0065] For example, when the gripping point-to-point cloud density decreases from 850 points / cm³ to 200 points / cm³, while the number of noise points in the point cloud remains at 200, and the point cloud hole ratio remains at 5%, it indicates that the gripping point cloud is sparse, the gripping points may be incomplete, the normals inaccurate, the gripping position unstable or failed, and the corresponding gripping effect may be worse. When the gripping point-to-point cloud density is fixed at 800 points / cm³, and the number of noise points in the point cloud increases from 100 to 1000, the point cloud hole ratio... If the percentage remains at 3%, it indicates that the gripping point may be misjudged as noise, the gripper may be misaligned, or the gripping may be ineffective. The algorithm may misdetect "false targets" or misjudge surface undulations, resulting in poor gripping performance. When the gripping point-point cloud density is stable at 900 points / cubic centimeter, the number of noise points in the point cloud is stable at 150 points, but the percentage of voids in the point cloud increases from 3% to 18%, it indicates that the usable area has decreased, the gripping point may be in a missing area, the gripping may fail or be unstable, and the system may not be able to identify the gripping surface, thus affecting the gripping performance.

[0066] It should be added that the specific content of the corresponding gripping point correction adjustment is as follows: the difference between the quantified value of the gripping effect and the judgment value of the gripping effect is calculated to obtain the gripping effect difference value. The gripping effect difference value is mapped in the preset database to obtain the gripping point correction amount. The gripping point correction amount is used to compensate for the robot's deviation data to improve the accuracy of the robot's selection of the gripping point of the specified soft packaging box stack. The robot's deviation data includes displacement error and torque error.

[0067] The system inputs the gripping effect difference into a pre-built gripping point correction mapping model in a preset database to output the corresponding gripping point correction amount. This value quantifies the magnitude of adjustment to the current gripping point. The gripping point correction mapping model is built based on a linear regression algorithm, using the least squares method as the optimization criterion, and is trained using the scikit-learn framework. The model training data comes from gripping effect differences recorded in historical periods and gripping point correction amounts set by professionals based on experience. The aim is to fit the mapping relationship between the gripping effect difference and the gripping point correction amount, thereby achieving accurate modeling and evaluation of the degree of gripping point adjustment. In addition, the compensation for the robot's deviation data through the gripping point correction amount is represented by multiplying the gripping point correction amount by the robot's deviation data.

[0068] In this embodiment, by calculating the gripping effect difference and mapping it to the gripping point correction amount, more accurate compensation for displacement and torque errors of the robotic arm in actual operation is achieved, fundamentally improving the accuracy of gripping point selection. Furthermore, by comparing the quantified gripping effect value with the preset judgment value and calculating the difference, error compensation forms a closed-loop quantification mechanism based on the actual effect, enhancing the system's adaptive adjustment capability. At the same time, by considering displacement and torque errors, comprehensive optimization of the gripping path and gripping posture is achieved, which helps to cope with the complex gripping challenges of soft packaging items caused by center of gravity shift, surface irregularities, etc. Moreover, by compensating for deviation data to improve the end effector action, slippage, skewness, or gripping failure during the gripping process is reduced, improving the stability and success rate of the robotic arm's gripping.

[0069] like Figure 5 The diagram shows the interface for adjusting and limiting the clamping force of the robotic arm in the intelligent depalletizing platform provided in this application embodiment. The left side of the interface is the platform's navigation bar, which includes the main control panel, robotic arm monitoring, robotic arm monitoring, pallet monitoring, task configuration, system configuration, and alarms and logs. The central part of the interface includes clamping force settings, stage force threshold settings, and adaptive settings for flexible packaging clamping force. Specifically, the clamping force settings include maximum and minimum clamping force limits; the stage force threshold settings include contact stage, lifting stage, moving stage, and placement stage; and the adaptive settings for flexible packaging clamping force include flexible packaging-clamping force adjustment, gripping point-clamping force adjustment, and gripping success rate-clamping force adjustment. This interface provides a more intuitive view of the clamping force control, allowing for adjustments to corresponding parameters based on the control status, thereby improving the accuracy of clamping force control.

[0070] Furthermore, the success rate of gripping within the preset gripping cycle is assessed to determine whether a gripping force control adjustment strategy should be adopted. The specific process is as follows: After obtaining the gripping point correction and adjustment within the preset gripping cycle, the robot arm achieves the gripping success rate of the specified soft packaging stack. The gripping success rate is compared with the preset success rate set in the preset database. If the gripping success rate is less than the preset success rate, a gripping force control adjustment strategy is adopted through the preset gripping force adjustment amount. Otherwise, the specified soft packaging stack continues to be gripped. This realizes the intelligent triggering of the gripping force control strategy, effectively ensuring the stability and reliability of the robot arm's gripping action. The preset success rate is set by the preset staff based on the gripping requirements and stored in the preset database.

[0071] The preset clamping force adjustment includes a first clamping force adjustment, a second clamping force adjustment, and a third clamping force adjustment. The first clamping force adjustment is the result obtained by mapping the point cloud data-clamping influence quantity in the preset database. The second clamping force adjustment is the result obtained by mapping the clamping effect quantification value in the preset database. The third clamping force adjustment is the result obtained by mapping the grasping success rate difference in the preset database, where the grasping success rate difference is the difference between the preset success rate and the grasping success rate.

[0072] Specifically, the preset clamping force adjustment amount is obtained based on the clamping force adjustment mapping model in the preset database. This model includes a first clamping force adjustment mapping model, a second clamping force adjustment mapping model, and a third clamping force adjustment mapping model. All clamping force adjustment mapping models are constructed based on a linear regression algorithm, using the least squares method as the optimization criterion, and trained using the scikit-learn framework. The model training data comes from point cloud data obtained from historical data, specifically the difference between clamping influence, clamping effect quantification, and grasping success rate, as well as the first, second, and third clamping force adjustment amounts set by professionals based on experience. The aim is to fit the mapping relationship between point cloud data, clamping influence, the difference between clamping effect quantification and grasping success rate, and the corresponding preset clamping force adjustment amounts, thereby achieving accurate modeling and evaluation of the degree of clamping force adjustment.

[0073] It should be explained that the specific details of the clamping force control adjustment strategy are as follows:

[0074] B1. Obtain the clamping force adjustment influence rate corresponding to the preset clamping force adjustment amount from the preset database. The clamping force adjustment influence rate includes the first clamping force influence rate, the second clamping force influence rate, and the third clamping force influence rate.

[0075] It should be noted that the preset clamping force adjustment amount is input into the clamping force adjustment mapping set set in the preset database, and the corresponding clamping force adjustment influence rate is output. The clamping force adjustment mapping set is a set used to fit the mapping relationship between the preset clamping force adjustment amount and the clamping force adjustment influence rate.

[0076] B2, after weighting the preset clamping force adjustment amount and the corresponding clamping force adjustment influence rate, the clamping force adjustment ratio is obtained by coupling. This avoids a single adjustment path and improves the system's adaptability to different physical characteristics and clamping states. The clamping force adjustment ratio is the data for controlling the clamping force of the robot arm on a specified soft packaging box stack. Through the weighting and fusion of the above three factors, a more scientific and reasonable clamping force adjustment ratio is formed, which helps to improve the targeting and accuracy of the adjustment effect.

[0077] B3. After the preset gripping cycle, the clamping force adjustment ratio is transmitted to the preset operator terminal, and the preset operator is prompted to adjust the preset clamping force according to the clamping force adjustment ratio. That is, the preset clamping force is multiplied by the clamping force adjustment ratio to obtain the optimized clamping force. The specified soft packaging box stack is clamped by the optimized clamping force. If the gripping success rate of the specified soft packaging box stack in the next preset gripping cycle is still lower than the preset success rate after adjustment, the gripping is stopped and a fault inspection warning is issued.

[0078] In this embodiment, more precise control of the gripping force helps prevent detachment due to insufficient gripping force or deformation or even damage to soft packaging due to excessive gripping force, thus improving the overall grasping success rate and operational safety of the robotic arm. Simultaneously, the system provides feedback on the gripping force adjustment ratio to the operator via terminal prompts, creating a collaborative mechanism between system adjustment and manual intervention. This retains human controllability while reducing the error rate. If multiple adjustments still fail to meet the grasping success rate requirements, the system automatically terminates the operation and issues a fault repair warning, preventing equipment damage or operational interruption caused by repeated failures. This method achieves quantifiability, traceability, and controllability in the gripping force adjustment process, meeting the high reliability and adaptability requirements of robotic arms in various industrial scenarios and enhancing the system's application value in complex operating environments such as flexible manufacturing and logistics automation.

[0079] Meanwhile, this application also provides a machine vision-based intelligent depalletizing and palletizing control device for robotic arms entering the warehouse. This device is used to implement the aforementioned machine vision-based intelligent depalletizing and palletizing control system for robotic arms entering the warehouse. The device specifically includes: a visual perception structure, a robotic arm and gripping structure, and a control decision module. The visual perception structure mainly includes an industrial camera installed at the end of the robotic arm, a light source system to assist visual recognition, and a visual computing unit for image processing and point cloud analysis. The robotic arm and gripping structure is a multi-degree-of-freedom industrial robot used to support depalletizing, rotation, and handling actions. It is equipped with a grasping detection sensor to determine whether the grasping is successful, and various grippers, specifically including vacuum suction cup grippers, electric grippers, and pneumatic grippers. The control decision module is used to run visual recognition and grasping path planning algorithms, and works in conjunction with the robot body to control the robotic arm's movements. The control cabinet includes an industrial PLC, relays, IO (Input / Output) modules, and power management, etc.

[0080] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0081] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0082] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0083] It should be understood that, in the various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0084] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0085] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0086] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0087] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0089] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0090] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A machine vision-based intelligent depalletizing and palletizing control system for robotic arms in warehousing, characterized in that, include: The robotic arm's inbound and gripping monitoring module, the robotic arm's gripping effect quantification module, and the robotic arm's gripping force control module; The robotic arm entry and gripping monitoring module is used to monitor the impact of the designated soft packaging box stack on the point cloud data in real time when the robotic arm enters the warehouse to intelligently unpack and stack the designated soft packaging box stack, so as to select whether to take soft packaging impact compensation measures. The real-time monitoring of the impact of designated flexible packaging pallets on point cloud data to determine whether to implement flexible packaging impact compensation measures involves the following steps: Step 1: Real-time acquisition of the impact data of the soft packaging box stack on the point cloud data of the soft packaging point cloud data. The soft packaging point cloud impact data includes point cloud curvature, point cloud depth value in the Z direction, point cloud jump frequency, and point cloud overlap rate. Step 2: Obtain the point cloud data-grip impact based on the point cloud impact data of flexible packaging, and compare the point cloud data-grip impact with the point cloud-grip judgment value obtained from the preset database. If the point cloud data-grip impact is less than the point cloud-grip judgment value, continue to grab the specified flexible packaging box stack; otherwise, take flexible packaging impact compensation measures. The point cloud data-grip impact is used to quantify the degree of interference of flexible packaging characteristics on machine vision recognition, and the point cloud jump frequency is the ratio of the number of abnormal points to the total number of points. The specific method for obtaining the point cloud data-squeezing influence is as follows: The influence ratio and preset depth value of the soft packaging point cloud are obtained from the preset database. The influence ratio of the soft packaging point cloud includes the influence ratio of point cloud curvature, the influence ratio of point cloud depth value in the Z direction, the influence ratio of point cloud jump frequency, and the influence ratio of point cloud overlap rate. The degree of deviation between the depth value in the Z direction of the point cloud and the preset depth value is quantified to obtain the corresponding depth deviation influence. After weighting the depth deviation influence, point cloud curvature, point cloud jump frequency, and point cloud overlap rate with the corresponding soft packaging point cloud influence ratio, the point cloud data-pinch influence is obtained by coupling the point cloud data-pinch influence. The robotic arm gripping effect quantification module is used to quantify the gripping effect of the robotic arm in grasping a specified soft packaging box stack in order to determine whether to perform corresponding gripping point correction adjustment. The gripping point correction adjustment is used to improve the accuracy of the robotic arm in selecting the gripping point of the specified soft packaging box stack. The robotic arm gripping force control module is used to determine the success rate of gripping within a preset gripping cycle to determine whether to adopt a gripping force control adjustment strategy. The gripping force control adjustment strategy is used to improve the accuracy of the gripping force applied by the robotic arm to a specified soft packaging box stack.

2. The intelligent depalletizing and palletizing control system for robotic arms in warehousing based on machine vision according to claim 1, characterized in that, The specific process for implementing compensation measures for the impact of flexible packaging is as follows: The difference between point cloud data - gripping influence and point cloud - gripping determination is called gripping influence difference. The gripping influence difference is mapped in the preset database to obtain the camera exposure time adjustment amount. The optimized camera exposure time is obtained by multiplying the initial camera exposure time of the machine vision based on the camera exposure time adjustment. The specified soft packaging box stacks are rescanned by optimizing the camera exposure time until the point cloud data-grip influence is less than the point cloud-grip judgment amount. The number of rescans is obtained in real time and compared with the preset number. If the number of rescans is lower than the preset number, the robot continues to grab the specified soft packaging box stack. Otherwise, the robot stops grabbing the specified soft packaging box stack and issues an abnormal grabbing warning.

3. The intelligent depalletizing and palletizing control system for robotic arms in warehousing based on machine vision according to claim 1, characterized in that, The specific steps for quantifying the gripping effect of the robotic arm in grasping a specified stack of soft packaging boxes to determine whether corresponding gripping point correction adjustments are necessary are as follows: S1, Analyze the real-time acquired gripping effect data of the specified soft packaging box stack to obtain a quantitative value of the gripping effect in order to quantify the accuracy of the gripping point of the specified soft packaging box stack selected by the robot arm. S2, compare the quantized value of the gripping effect with the judgment value of the gripping effect obtained from the preset database. If the quantized value of the gripping effect is greater than the judgment value of the gripping effect, continue to grab the specified soft packaging box stack; otherwise, perform the corresponding gripping point correction adjustment.

4. The intelligent depalletizing and palletizing control system for robotic arms in warehousing based on machine vision according to claim 3, characterized in that, The specific method for obtaining the quantification value of the clamping effect is as follows: Real-time acquisition of gripping effect analysis data and data normalization processing are performed. The gripping effect analysis data includes gripping point-point cloud density, number of point cloud noise points, and point cloud hole ratio. Extract the gripping effect analysis weight from the preset database. The gripping effect analysis weight includes the gripping point-point cloud density analysis weight, the point cloud noise point number analysis weight, and the point cloud hole ratio analysis weight. The clamping effect analysis data is coupled with the corresponding clamping effect analysis weights after weighting operations to obtain the clamping effect quantification value.

5. The intelligent depalletizing and palletizing control system for robotic arms in warehousing based on machine vision according to claim 3, characterized in that, The specific details of performing the corresponding grip point correction adjustment are as follows: The difference between the quantified value of the gripping effect and the judgment value of the gripping effect is calculated to obtain the gripping effect difference. The gripping effect difference is mapped in a preset database to obtain the gripping point correction amount. The gripping point correction amount is used to compensate for the deviation data of the robot arm to improve the accuracy of the robot arm in selecting the gripping point of the specified soft packaging box stack. The deviation data of the robot arm includes displacement error and torque error.

6. The intelligent depalletizing and palletizing control system for robotic arms in warehousing based on machine vision according to claim 3, characterized in that, The process of determining whether to adopt a gripping force control adjustment strategy by judging the success rate of gripping within a preset gripping cycle is as follows: After obtaining the gripping point correction adjustment within the preset gripping cycle, the success rate of the robot arm in gripping the specified soft packaging box stack; The grabbing success rate is compared with the preset success rate set in the preset database. If the grabbing success rate is less than the preset success rate, the clamping force control adjustment strategy is adopted through the preset clamping force adjustment amount. Otherwise, the specified soft packaging box stack is continued to be grabbed. The preset clamping force adjustment amount includes a first clamping force adjustment amount, a second clamping force adjustment amount, and a third clamping force adjustment amount; The first clamping force adjustment amount is obtained by mapping point cloud data - clamping influence amount in a preset database; The second clamping force adjustment amount is obtained by mapping the clamping effect quantification value into a preset database; The third clamping force adjustment amount is obtained by mapping the grasping success rate difference amount in a preset database, where the grasping success rate difference amount is the difference between the preset success rate and the grasping success rate.

7. The intelligent depalletizing and palletizing control system for robotic arms in warehousing based on machine vision according to claim 6, characterized in that, The specific details of the clamping force control adjustment strategy are as follows: Obtain the clamping force adjustment influence rate corresponding to the preset clamping force adjustment amount from the preset database. The clamping force adjustment influence rate includes a first clamping force influence rate, a second clamping force influence rate, and a third clamping force influence rate. After weighting the preset clamping force adjustment amount and the corresponding clamping force adjustment influence rate, the clamping force adjustment ratio is obtained by coupling. The clamping force adjustment ratio is the data for controlling the clamping force of the robot arm on the specified soft packaging box stack. After the preset gripping cycle, the clamping force adjustment ratio is transmitted to the preset operator terminal, and the preset operator is prompted to adjust the preset clamping force according to the clamping force adjustment ratio. If, after adjustment, the gripping success rate of the specified soft packaging box stack in the next preset gripping cycle is still lower than the preset success rate, gripping is stopped and a fault repair warning is issued.

8. A machine vision-based intelligent depalletizing and palletizing control device for robotic arm warehousing, used to implement the machine vision-based intelligent depalletizing and palletizing control system for robotic arm warehousing as described in any one of claims 1-7, characterized in that, include: Visual perception structure, robotic arm and gripping structure, and control decision module; The visual perception structure includes an industrial camera mounted on the end effector of a robotic arm, a light source system to assist visual recognition, and a visual computing unit for image processing and point cloud analysis. The robotic arm and gripping structure is a multi-degree-of-freedom industrial robot used to support destacking, rotation and handling actions. It is equipped with gripping detection sensors to determine whether the gripping is successful, and various grippers, including vacuum suction cup grippers, electric grippers and pneumatic grippers. The control decision module is used to run visual recognition and grasping path planning algorithms, which work in conjunction with the robot body to control the movements of the robotic arm.

Images