A vision-based palletizing method

Abstract

This application relates to the technical field of robotic palletizing, and particularly to a vision-based palletizing method, which includes the following steps: Information acquisition: Obtain the top two-dimensional code image of the board and the board image; Information extraction: From the top two-dimensional code label image and the board image, obtain the physical boundary of the two-dimensional code label and its positional relationship with the board edge; Offset calculation: Calculate the grasping compensation angle according to the pose deviation of the two-dimensional code edge; Grasping adjustment: Adjust the offset pose of the manipulator according to the grasping compensation angle, so that the length direction of the grasping manipulator is parallel to the length direction of the board; Grasping execution: Execute the grasping action according to the compensated pose; Palletizing: Stack the grasped boards at the stacking position according to the program settings. This application has the effect of improving the work efficiency of palletizing.

Description

Technical Field

[0001] This application relates to the technical field of robotic palletizing, and particularly to a vision-based palletizing method. Background Art

[0002] Currently, in the current sheet processing industry, in order to make rational use of the remaining size of the sheet, generally multiple different specifications of sheets are processed on the same large sheet, and then the cut sheets are grasped and palletized. There are the following technical pain points for palletizing sheets with irregular sizes after cutting: The traditional mechanical positioning method relies on a preset program and cannot adapt to the randomly changing sheet sizes, resulting in a grasping failure rate as high as 15%-20%; Most existing vision positioning systems use contour recognition technology, but the recognition error for sheets with worn edges or uneven surfaces exceeds ±3 degrees; However, for the smaller sheets formed after processing, the deviation of the grasping angle will cause the stacks formed to easily collapse, and manual palletizing is required, or manual sheet picking or repair of the collapsed stack is needed, resulting in a decrease in work efficiency. Summary of the Invention

[0003] In order to improve the work efficiency of palletizing, this application provides a vision-based palletizing method.

[0004] A vision-based palletizing method provided by this application adopts the following technical solutions:

[0005] A vision-based palletizing method includes the following steps:

[0006] Information Acquisition: Obtain the top two-dimensional code image of the sheet and the sheet image;

[0007] Information Extraction: Obtain the physical boundary of the two-dimensional code label and its positional relationship with the edge of the sheet from the top two-dimensional code label image and the sheet image;

[0008] Offset Calculation: Calculate the grasping compensation angle according to the pose deviation of the two-dimensional code edge ;

[0009] Grasping Adjustment: Adjust the offset pose of the manipulator according to the grasping compensation angle so that the length direction of the grasping manipulator is parallel to the length direction of the sheet;

[0010] Grasping Execution: Execute the grasping action according to the compensated pose;

[0011] Palletizing: Stack the grasped sheets at the stacking position according to the program setting.

[0012] By adopting the above technical solution, by acquiring the QR code image and the sheet image at the top of the sheet, and extracting the physical boundary of the QR code label and its positional relationship with the sheet edge, this method can accurately determine the orientation and size of the sheet; this helps the manipulator to achieve higher precision when grasping the sheet, reducing grasping failures or sheet damage caused by inaccurate positions.

[0013] Calculate the grasping compensation angle according to the pose deviation of the QR code edge , and accordingly adjust the offset pose of the manipulator so that the length direction of the grasping manipulator is parallel to the length direction of the sheet; this adaptive adjustment mechanism can handle sheets of different sizes and directions, improving the flexibility and versatility of the palletizing operation.

[0014] Through precise grasping and adaptive adjustment, this method can ensure that the manipulator accurately stacks the sheet to the specified position after grasping the sheet; this helps to improve the efficiency of the palletizing operation, reducing manual intervention and error rate.

[0015] At the same time, it can reduce the difficulty of grasping small sheets, and can reduce the influence of grasping offset on stacking, facilitating the mixed stacking of small and large sheets, reducing the occurrence of easy toppling of small sheet stacking, and is more convenient compared to manual picking of sheets, and can improve the efficiency of stacking, and can record production data throughout the process for traceability, and can track the production status in real time, reducing the occurrence of missing or dropped sheets.

[0016] Optionally, in the information extraction step, QR code information recognition is also included and recorded as specification data;

[0017] In the palletizing step, it also includes performing position stacking according to the program set according to the specification data.

[0018] By adopting the above technical solution, adding QR code information recognition in the information extraction step and recording it as specification data, this improvement enables the system to obtain more information about the sheet; QR codes usually contain key data such as sheet size, material, batch, etc., and this information is crucial for subsequent palletizing operations; by identifying this information, the system can process the sheets more intelligently, improving the accuracy and efficiency of the operation.

[0019] In the palletizing step, performing position stacking according to the program set according to the specification data, this improvement realizes the intelligence of the palletizing operation; the system can automatically adjust the palletizing strategy according to different sheet specification data to ensure that the sheets are correctly stacked at the specified position; this reduces the need for manual intervention, improving the automation level and accuracy of the operation.

[0020] Through precise information extraction and intelligent palletizing operations, this method can significantly improve production efficiency; the system can quickly identify and process sheets, reducing waiting time and error rates, thus accelerating the palletizing speed; in addition, the intelligent palletizing strategy can also optimize the stacking method of sheets, improving the utilization rate of warehouse space.

[0021] The QR code information recognition also provides traceability for the sheets. Once problems occur during the subsequent use of the sheets, the information such as their source, batch, and production process can be quickly traced by scanning the QR code, which is of great significance for quality control and traceability management.

[0022] Optionally, the pasting of the QR code label satisfies:

[0023] The four sides of the label are parallel or collinear with the corresponding sides of the sheet;

[0024] The offset of the center point of the label from the geometric center of the sheet ≤ 2% of the sheet length.

[0025] By adopting the above technical solution, requiring the four sides of the QR code label to be parallel or collinear with the corresponding sides of the sheet ensures a stable and easily recognizable relative position relationship between the QR code label and the sheet during the image recognition process; such a label pasting method helps the image processing algorithm to more accurately extract the physical boundary of the QR code and its position relationship with the sheet edge, thus improving the accuracy of information extraction.

[0026] The offset of the center point of the label from the geometric center of the sheet is limited to ≤ 2% of the sheet length, which ensures the relative consistency between the position of the QR code label and the overall position of the sheet; in the offset calculation step, it helps to more accurately calculate the grasping compensation angle , and then optimize the grasping adjustment strategy of the manipulator; by reducing the grasping error caused by the label position deviation, the accuracy and stability of the grasping action can be improved.

[0027] By specifying the pasting requirements of the QR code label, the system can more reliably process sheets of different sizes and orientations; even under different working conditions, as long as the label meets the specified pasting requirements, the system can accurately identify and process the sheets, thus enhancing the robustness and adaptability of the system.

[0028] Due to more accurate information extraction, more optimized grasping adjustment, and stronger system robustness, the vision-based palletizing method with the addition of QR code label pasting requirements can significantly improve the operation efficiency; the system can more quickly identify and process sheets, reducing waiting time and error rates, thus accelerating the palletizing speed and improving the overall production efficiency.

[0029] Optionally, in the grasping execution step, grasp the middle part of the sheet.

[0030] By adopting the above technical solution, the middle part of the sheet is grabbed. On the one hand, it facilitates the subsequent stacking work. On the other hand, it facilitates the extrusion of the QR code label, reducing the occurrence of insecure sticking situations such as warping corners, curling edges, and folding, and facilitating the subsequent recognition of the QR code label.

[0031] Optionally, after the information extraction step, an offset distance calculation step is also provided;

[0032] The offset distance calculation includes an edge extension step, an offset quantization step, and an intervention step;

[0033] Edge extension: Obtain the edge of the QR code label from the top QR code label image, and draw an extension line for the edge of the QR code label and extend it to the edge of the sheet;

[0034] Offset quantization: Calculate the maximum offset according to the edge inclination angle, the length distance of the sheet, and the minimum distance between the edge of the QR code label and the edge of the sheet;

[0035] Intervention: Intervene in the stacking step. When stacking the sheet with an offset, adjust the angle. The side with the largest offset first contacts the sheet that has been stacked, and then execute the stacking step.

[0036] By adopting the above technical solution, through the edge extension step, the system can accurately identify the relative position relationship between the edge of the QR code label and the edge of the sheet, and calculate the offset on this basis; this helps to more precisely adjust the position of the sheet during the stacking process, reduce the inaccurate stacking problem caused by position deviation, and thus improve the stacking accuracy.

[0037] The offset quantization step calculates the maximum offset by comprehensively considering multiple factors such as the edge inclination angle, the length distance of the sheet, and the minimum distance between the edge of the QR code label and the edge of the sheet. This quantization index provides an important basis for the subsequent stacking strategy; in the intervention step, the system can adjust the stacking angle of the sheet according to the maximum offset, ensuring that the side of the sheet with a larger offset first contacts the sheet that has been stacked, thereby optimizing the stacking structure and improving the stacking stability and load-bearing capacity.

[0038] After adding the offset distance calculation step, the system can more flexibly handle sheets of different sizes, shapes, and offsets; even if the sheet has a certain offset during transportation or placement, the system can ensure the accuracy and stability of stacking through calculation and adjustment; this enhances the adaptability and flexibility of the system.

[0039] Through precise offset calculation and stacking strategy adjustment, the system can complete the stacking operation more quickly; reduce the repeated adjustment and error rate caused by inaccurate position, and thus improve the operation efficiency.

[0040] Optionally, after the intervention step, a feedback step for sheet material specifications is further provided;

[0041] Feedback on sheet material specifications: including a dimension difference calculation step, a dimension judgment step, a second judgment step, a feedback step, a recording step, a third judgment step, an angle correction step, a fourth judgment step, and a deviation correction step;

[0042] Dimension difference calculation: According to the maximum offset amounts of the two sides of the same sheet material and the two sides of the QR code label edge in the offset quantization step, calculate the offset difference between the two sides;

[0043] Dimension judgment: According to the offset difference between the two sides and the specified width of the sheet material, judge whether there is a cutting deviation. If so, execute the second acquisition step;

[0044] Second acquisition: Obtain the actual dimension specifications of the current sheet material through the sheet material image, and obtain the first deviation from the theoretical dimension specifications;

[0045] Second judgment: Calculate the ratio of the cutting deviation to the allowable deviation and record it as the deviation ratio. Judge whether the deviation ratio is greater than or equal to the first threshold. If so, execute the feedback step; otherwise, execute the third judgment step;

[0046] Feedback: Feed back the cutting deviation to the cutting side for cutting correction, and execute the recording step;

[0047] Recording: Record and store the cutting deviation data into the QR code label;

[0048] Third judgment: Judge whether there is a sheet material at one end in the length direction of the sheet material. If so, execute the angle correction; otherwise, execute the fourth judgment step;

[0049] Angle correction: Correct the grasping compensation angle After correction, execute the grasping adjustment step, and execute the fourth judgment step;

[0050] Fourth judgment: Judge whether the cutting deviation is the same as the first deviation. If so, execute the recording step; otherwise, execute the deviation correction step;

[0051] Deviation correction: Calculate the average value of the cutting deviation and the first deviation, and record it as the cutting deviation, and then execute the recording step.

[0052] By adopting the above technical solution, through the dimension difference calculation and dimension judgment steps, the system can accurately identify whether there is a deviation in the cutting process of the sheet material; once a cutting deviation is found, the system will feedback the deviation information to the cutting side through the feedback step for timely cutting correction; this helps to reduce sheet material waste and quality problems caused by inaccurate cutting and improve the cutting quality.

[0053] After recognizing the cutting deviation, the system not only performs cutting correction, but also corrects the grasping compensation angle according to the deviation situation (angle correction step); this ensures that even when the size of the sheet changes, the system can accurately adjust the grasping posture of the manipulator to ensure that the sheets are stacked correctly; this dynamic adjustment mechanism optimizes the stacking strategy and improves the stability and accuracy of stacking.

[0054] Through the recording step, the system records and stores the cutting deviation data into the QR code label; this not only provides important reference information for subsequent sheet processing, but also enhances the traceability of the system; once problems are found in the subsequent use process, the deviation information during cutting and stacking can be quickly traced by scanning the QR code, which is convenient for problem troubleshooting and quality traceability.

[0055] Although the sheet specification feedback step is added, the implementation of this step does not significantly increase the complexity of the operation; on the contrary, by accurately identifying and processing the cutting deviation, the system can reduce the waste of operation time caused by repeated adjustments and error rates, thereby improving the operation efficiency.

[0056] Optionally, between the third judgment step and the fourth judgment step, a fifth judgment step and a size difference correction step are also set;

[0057] Fifth judgment: Determine whether there is a sheet on one side in the width direction of the sheet. If so, execute the size difference correction step; otherwise, execute the fourth judgment step;

[0058] Size difference correction: Intervene in the size difference calculation step, recalculate the size difference, and then execute the fourth judgment step.

[0059] By adopting the above technical solution, when there are sheets on the side, due to the mutual influence between the sheets, the deviation value obtained by the original size difference calculation step may be inaccurate; at this time, by executing the size difference correction step, the system can recalculate the size difference to ensure the accuracy of cutting deviation recognition; this helps the system to adjust the stacking strategy more precisely and reduce stacking errors caused by inaccurate deviation recognition.

[0060] Optionally, after the recording step, an associated label establishment step and a placement feedback step are also included;

[0061] Associated label establishment: Record the sheets with large cutting deviations, determine the positions of the deviations, establish connection labels with adjacent cut sheets, and store them in the QR code label;

[0062] Placement feedback: Determine the connection between sheets according to the connection label, and determine the placement position according to the stacking program. Associate the placement position with the connection label and store it in the QR code label.

[0063] By adopting the above technical solution, through the connection label establishment step, the system can record the sheets with large cutting deviations and establish connection labels with adjacent cut sheets; this step helps to quickly trace the problem sheets and their adjacent sheets during subsequent production or quality inspection processes, thereby improving the efficiency of problem tracing.

[0064] The placement feedback step determines the stacking position of the sheets according to the connection label and the stacking program, and associates the stacking position with the connection label; this step helps the system to more reasonably arrange the layout of the sheets during the palletizing process, ensuring that the problem sheets and their adjacent sheets can be stacked in a specific order and position, thereby optimizing the palletizing layout.

[0065] By adding the connection label establishment step and the placement feedback step, the system can more intelligently handle the cutting and palletizing problems of the sheets; the system can automatically identify and process cutting deviations, establish connection labels, and optimize the palletizing layout according to the stacking program; this intelligent processing method helps to improve the flexibility and adaptability of the system and reduce the cost of manual intervention.

[0066] On the other hand, by adding the connection label establishment step and the placement feedback step, it is convenient for the purchaser to reasonably use the sheets with an associated relationship, facilitate making up for its own defects, and reduce the expansion of defects.

[0067] Optionally, in the palletizing step, it further includes optimizing the stacking order according to the cutting deviation and the offset position.

[0068] By adopting the above technical solution, in the palletizing step, optimizing the stacking order according to the cutting deviation and the offset position can ensure that the sheets are stacked in the best order during the palletizing process. This optimization helps to reduce palletizing errors caused by inconsistent sheet sizes or cutting deviations, thereby improving the palletizing efficiency.

[0069] By optimizing the stacking order, the system can more reasonably arrange the position of the sheets in the warehouse; after adding the function of optimizing the stacking order, the system can dynamically adjust according to the actual situation of the sheets. This flexibility helps the system to adapt to different types of sheets and different cutting deviation situations, improving the versatility and adaptability of the system.

[0070] Optionally, in the palletizing step, it further includes: same-layer deviation statistics step, quantity judgment step, order maintenance step, set position adjustment step, and stacking execution step;

[0071] Same - layer deviation statistics: Obtain the number of plates with cutting deviation on the same stacking layer, which is recorded as the deviation quantity.

[0072] Quantity judgment: Judge whether the deviation quantity is greater than or equal to the second threshold. If so, execute the order - maintaining step; otherwise, execute the set - position adjustment step.

[0073] Order maintaining: Maintain the order set by the set stacking program and execute the stacking execution step.

[0074] Set - position adjustment: Determine the stacking position of plates of the same specification according to the set stacking program, set the first weight of the plates according to the stacking position, determine the second weight according to the stacking order, then calculate the stacking importance value based on the size of the deviation ratio and the correlation degree of adjacent plates with complementary deviations, adjust the stacking order according to the stacking importance value, and then execute the stacking execution step.

[0075] By adopting the above - mentioned technical solution, by counting the number of plates with cutting deviation on the same stacking layer, the stacking quality of the current layer can be intuitively understood, and this step provides data support for subsequent judgment and adjustment; by judging whether the deviation quantity reaches or exceeds the set second threshold, it can be decided whether to adjust the stacking order; when the deviation quantity does not reach the second threshold, maintaining the original stacking order can maintain a high stacking efficiency and avoid production delays caused by frequent adjustments; when the deviation quantity reaches or exceeds the second threshold, this step adjusts the stacking order by calculating the stacking importance value; this adjustment takes into account the specifications, stacking positions, stacking orders of the plates and the complementary relationship between adjacent plates, which helps to optimize the stacking quality and reduce space waste or structural instability caused by cutting deviation; it ensures the implementation of the decisions and adjustments of all the previous steps; through precise stacking operations, it can ensure that the plates are kept neat and stable during the stacking process.

[0076] In summary, the present application includes at least one of the following beneficial technical effects:

[0077] 1. Through precise grasping and adaptive adjustment, this method can ensure that the manipulator accurately stacks the plates to the specified position after grasping the plates; this helps to improve the efficiency of the stacking operation, reduce manual intervention and error rate.

[0078] 2. Through precise stacking operations, it can ensure that the plates are kept neat and stable during the stacking process. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 is the flowchart of the stacking method in Embodiment 1 of the present application;

[0080] Figure 2It is a sub-step flowchart of the offset distance calculation step in Embodiment 2 of the present application;

[0081] Figure 3 It is a sub-step flowchart of the sheet material specification feedback step in Embodiment 2 of the present application;

[0082] Figure 4 It is a flowchart of the associated label establishment step and the placement feedback step in Embodiment 2 of the present application;

[0083] Figure 5 It is a sub-step flowchart of the palletizing step in Embodiment 2 of the present application. Detailed implementation mode

[0084] The following is combined with Figures 1 to 5 to further elaborate on the present application in detail.

[0085] This embodiment discloses a vision-based palletizing method.

[0086] Refer to Figure 1 , the vision-based palletizing method includes the following steps:

[0087] Information acquisition: Stick a two-dimensional code label on the board, and the sticking of the two-dimensional code label satisfies: the four sides of the label are parallel or collinear with the corresponding sides of the board, and the offset of the label center point from the geometric center of the board ≤ 2% of the board length, and obtain the two-dimensional code image at the top of the board and the board image;

[0088] Specifically, for the sticking of the two-dimensional code label: Stick the two-dimensional code label at the middle position of the board to ensure that the four sides of the label are parallel or collinear with the corresponding sides of the board; ensure that the offset of the label center point from the geometric center of the board does not exceed 2% of the board length.

[0089] Use a high-resolution industrial camera to vertically shoot from the top of the board to obtain the two-dimensional code image at the top of the board and the overall image of the board. The camera should be fixedly installed to ensure that the viewing angle is consistent for each shooting.

[0090] Information extraction: From the two-dimensional code label image at the top and the board image, obtain the physical boundary of the two-dimensional code label and its positional relationship with the board edge, identify the information stored in the two-dimensional code label, and record it as specification data;

[0091] Specifically, use image processing algorithms (such as edge detection, contour extraction, etc.) to identify the physical boundary of the two-dimensional code label and its positional relationship with the board edge; use the two-dimensional code decoding algorithm to read the information stored in the label, which usually includes the specifications, materials, etc. of the board, and record it as specification data.

[0092] For example, through an image processing algorithm, the four corner points of the QR code label are identified, and then its boundary is determined. After decoding the QR code, the specifications of the board are obtained as 1000mm * 500mm, and the material is wood.

[0093] Offset calculation: Calculate the grasping compensation angle based on the pose deviation of the QR code edge ;

[0094] Specifically, first, use an image processing algorithm (such as the Canny edge detector, etc.) to identify the edge of the QR code label; then, obtain the contour of the QR code label through a contour extraction algorithm (such as the Hough transform, etc.).

[0095] Definition of ideal position: The ideal position is usually defined as the relative position of the length direction of the QR code label and the grasping side of the manipulator, that is, the state that should be presented under camera shooting, such as completely horizontal, without inclination and located at the center of the image; in actual applications, the ideal position may vary according to specific application scenarios and requirements, and is set according to self - needs, such as completely horizontal, biased to one side, and located at the center of the length of the board.

[0096] Geometric transformation: Perform geometric transformations (such as rotation, translation, etc.) on the extracted contour of the QR code label to align it with the ideal position. The parameters of the geometric transformation (such as the rotation angle) are the compensation angles that need to be calculated 。

[0097] Angle calculation: Before or after performing geometric transformation, the compensation angle can be obtained by calculating the angle between specific points (such as the center point, corner points, etc.) of the contour and the ideal position The angle calculation usually involves vector operations and trigonometric functions. For example, assume that we have obtained the contour of the QR code label through edge detection and contour extraction, and the ideal position has been determined. The following is a simplified calculation formula for calculating the compensation angle :

[0098] Calculate the center point of the contour: Assume that the contour consists of a series of points (x1, y1), (x2, y2),..., (xn, yn).

[0099] The center point of the contour (Cx, Cy) can be obtained by calculating the average coordinates of all points:

[0100] Cx=(x1 + x2 +... + xn) / n;

[0101] Cy=(y1 + y2 +... + yn) / n;

[0102] Calculate the ideal center point: The ideal center point (Ix, Iy) is defined according to the application scenario and requirements.

[0103] In many cases, the ideal center point may be the center point of the image or a predetermined position;

[0104] Calculate the vector: The vector V from the center point of the contour (Cx, Cy) to the ideal center point (Ix, Iy) can be expressed as: V = (Ix - Cx, Iy - Cy);

[0105] Calculate the included angle: Suppose there is a horizontal vector H = (1, 0), which represents the ideal non-inclined state.

[0106] The included angle between vector V and vector H is the compensation angle It can be calculated by the following formula:

[0107] θ = arctan2(Vy, Vx) - π / 2 (if it is necessary to adjust to the range from 0 to π);

[0108] where arctan2 is a variant of the arctangent function that accepts two parameters and returns the included angle between them (in radians). Vx and Vy are the x and y components of vector V respectively.

[0109] Adjust the angle: According to the requirements of the application scenario, it may be necessary to adjust the calculated included angle.

[0110] For example, if it is necessary to rotate the QR code label to a completely horizontal position, then it may be necessary to take the inverse or perform other transformations.

[0111] Grasping adjustment: Adjust the offset posture of the manipulator according to the grasping compensation angle so that the length direction of the grasping manipulator is parallel to the length direction of the plate;

[0112] Specifically, according to the calculated compensation angle , adjust the posture of the manipulator. This is usually achieved through the control system of the manipulator to ensure that the length direction of the manipulator is parallel to the length direction of the plate.

[0113] Grasping execution: Execute the grasping action according to the compensated posture, and when grasping, grasp the middle part of the plate;

[0114] Specifically, according to the plate image and specification data, determine the grasping position, usually select the middle part of the plate as the grasping point to ensure stable grasping.

[0115] Execute the grasping action: Control the manipulator to move to the grasping position and execute the grasping action.

[0116] Palletizing: According to the program setting, based on the specification data, execute the position set by the program and stack the grasped plates to the stacking position.

[0117] Specifically, according to the program settings and specification data, the stacking position is determined. This usually involves position calculations in three-dimensional space; perform the stacking action: control the manipulator to move the grabbed sheet to the stacking position and then release the sheet.

[0118] In this embodiment, the position of the sheet is determined by the angle between the two-dimensional code label and the sheet, the deflection angle between the sheet and the manipulator is determined, angle compensation is performed to achieve the accuracy of grasping, which can reduce the occurrence of missing or fewer sheets, and facilitate the grasping of small sheets. At the same time, it can meet the mixed stacking of large and small sheets, reduce the occurrence of re-stacking, and improve work efficiency.

[0119] Embodiment 2: Refer to Figure 2 , which is different from Embodiment 1 in that after the information extraction step, an offset distance calculation step is further provided;

[0120] The offset distance calculation includes an edge extension step, an offset quantization step, and an intervention step;

[0121] Edge extension: Obtain the edge of the two-dimensional code label from the top two-dimensional code label image, and extend the edge of the two-dimensional code label to the edge of the sheet;

[0122] Offset quantization: Calculate the maximum offset according to the edge inclination angle, the length distance of the sheet, and the minimum distance between the edge of the two-dimensional code label and the edge of the sheet;

[0123] Intervention: Intervene in the stacking step. When stacking the sheet with an offset, adjust the angle so that the maximum offset side first contacts the sheet that has been stacked, and then perform the stacking step.

[0124] Specifically, use image processing algorithms (such as edge detection, line fitting, etc.) to accurately obtain the edge of the two-dimensional code label from the top two-dimensional code label image, and then extend these edges until they intersect with the edge of the sheet or reach a predetermined extension distance; the purpose of this step is to determine the relative position relationship between the two-dimensional code label and the edge of the sheet; according to the edge extension line obtained in the edge extension step, calculate the edge inclination angle (i.e., the angle between the edge extension line and the edge of the sheet); then, combine the length distance of the sheet (i.e., the distance from the position where the two-dimensional code label is located to one end of the sheet) and the minimum distance between the edge of the two-dimensional code label and the edge of the sheet, and use geometric transformation and trigonometric functions to calculate the maximum offset. This maximum offset represents the degree of offset of the two-dimensional code label relative to the ideal position, that is, the maximum offset of the end of the sheet compared with the correct grasping after grasping according to the two-dimensional code label.

[0125] Before the palletizing step, intervene in the palletizing process according to the maximum offset calculated in the offset quantization step. For the offset board, adjust the grasping angle of the manipulator or the placement angle of the board during palletizing, so that the maximum offset side contacts the already palletized board first, and then gradually move to adjust and place the board. This can reduce the problems of unstable palletizing or space waste caused by offset, and can also reduce the pushing of other boards caused by placing according to the standard grasping position, reducing the possibility of toppling.

[0126] Refer to Figure 3 , in other embodiments, after the intervention step, there is also a board specification feedback step;

[0127] Board specification feedback: includes size difference calculation step, size judgment step, second judgment step, feedback step, recording step, third judgment step, angle correction step, fourth judgment step, deviation correction step, fifth judgment step and size difference correction step;

[0128] Size difference calculation: According to the maximum offset of the two sides of the same board and the edges of the two sides of the QR code label at the edge of the QR code label in the offset quantization step, calculate the offset difference between the two sides;

[0129] Size judgment: According to the offset difference between the two sides and the specified width of the board, judge whether there is a cutting deviation. If so, execute the second acquisition step;

[0130] Specifically, in the offset quantization step, the maximum offset of the two sides of the same board and the edge of the QR code label has been calculated; in this step, we subtract these two offsets to obtain the offset difference between the two sides; this difference reflects the deviation of the board in the width direction due to inaccurate cutting or placement; according to the known specified width of the board (such as the standard width is 1200mm) and the offset difference calculated in the previous step, we can judge whether the board has a cutting deviation; if the offset difference exceeds the allowable range (set according to process requirements), it is considered that there is a cutting deviation.

[0131] Second acquisition: Obtain the actual size specification of the current board through the board image, and obtain the first deviation from the theoretical size specification;

[0132] Second judgment: Calculate the ratio of the cutting deviation to the allowable deviation, and record it as the deviation ratio. Judge whether the deviation ratio is greater than or equal to the first threshold. If so, execute the feedback step, otherwise, execute the third judgment step;

[0133] Specifically, when it is determined that there is a cutting deviation, the image of the sheet is taken again, and the actual size specifications of the current sheet are obtained using image processing technology; then, this actual size is compared with the theoretical size (i.e., the standard size) to obtain the first deviation; calculate the ratio of the cutting deviation (i.e., the deviation determined in the size judgment step) to the allowable deviation (the maximum acceptable deviation set according to the process requirements), denoted as the deviation ratio; if the deviation ratio is greater than or equal to the set first threshold (such as 5%), and the determination of the first threshold is set according to the customer's requirements, it is considered that the deviation is large and the feedback step needs to be executed.

[0134] Feedback: Feed the cutting deviation back to the cutting side for cutting correction and execute the recording step;

[0135] Recording: Record and store the cutting deviation data into the QR code label;

[0136] Specifically, feedback the information of the cutting deviation to the person in charge of the cutting equipment or the cutting process for cutting correction; at the same time, execute the recording step, record and store the cutting deviation data into the QR code label for subsequent tracking and analysis; use methods such as a database or a file system to record information such as the cutting deviation data, the sheet number, and the time, and store the data in the QR code label pasted on the sheet for subsequent operation and tracking.

[0137] Third judgment: Judge whether there is a sheet at one end in the length direction of the sheet. If so, execute the angle correction; otherwise, execute the fifth judgment step;

[0138] Specifically, the extension line of the edge of the QR code label extends along the length direction of the sheet. Judge whether there is a sheet below the length direction of the sheet. If there is, it means that during the process of conveying the sheet after cutting, there is stacking of the sheets, resulting in one end of the length direction of the sheet being higher and the other end being lower from the placement plane, which will cause a change in the deviation angle between the photographed QR code label and the sheet. Therefore, it is necessary to correct the grasping compensation angle to ensure the stability of the palletizing.

[0139] Angle correction: Correct the grasping compensation angle After correction, execute the grasping adjustment step and execute the fourth judgment step;

[0140] Specifically, according to the result of the third judgment, correct the grasping compensation angle Make necessary corrections. When making corrections, since the sheet has a thickness, after one end is lifted, the side close to the placement plane will be exposed. By obtaining the exposed width and then performing geometric calculations, the deviation angle between the QR code label and the actual pasting angle of the sheet can be obtained. The corrected angle should be able to reduce the offset and tilt of the sheet during palletizing and improve the stability of palletizing. After correction, execute the grasping adjustment step and perform the grasping and placement operations according to the new angle.

[0141] Fifth judgment: Determine whether there is a sheet in the width direction of the sheet. If so, execute the dimension difference correction step; otherwise, execute the fourth judgment step.

[0142] Fourth judgment: Determine whether the cutting deviation is the same as the first deviation. If so, execute the recording step; otherwise, execute the deviation correction step.

[0143] Deviation correction: Calculate the average value of the cutting deviation and the first deviation, record it as the cutting deviation, and then execute the recording step.

[0144] Specifically, after the angle correction, determine again whether the cutting deviation is the same as the first deviation. If they are the same, it means the correction is effective and the recording step can be executed; if they are different, it means there may be other factors affecting the size of the sheet, and the deviation correction step needs to be executed. By taking the average of the first deviation and the cutting deviation, the excessive error can be reduced and the data accuracy can be improved.

[0145] Dimension difference correction: Intervene in the dimension difference calculation step, recalculate the dimension difference, and then execute the fourth judgment step.

[0146] Specifically, determine whether there is a sheet on one side of the width direction of the sheet. If there is, it means that during the sheet conveying process, there is another sheet under this sheet, making one side of the width direction of the photographed sheet higher and the other side lower compared to the placement plane, which affects the acquisition of the dimension difference. Therefore, it is necessary to intervene in the dimension difference calculation step and recalculate the dimension difference.

[0147] Refer to Figure 4 , in other embodiments, after the recording step, there are also an associated label establishment step and a placement feedback step;

[0148] Associated label establishment: Record the sheets with large cutting deviations, determine the positions of the deviations, establish connection labels with adjacent cut sheets, and store them in the QR code label.

[0149] Specifically, for plates with large cutting deviations, record them and determine the positions of the deviations. Then, establish connection tags with adjacent cut plates. This tag contains the relative position relationship and deviation information between the plates. The relative position relationship includes the deviation of the plate and the corresponding deviation of the adjacent plate. Store this information in a QR code tag for subsequent tracking and analysis. At the same time, it is convenient for the purchaser of the plates to arrange the arrangement and use of the plates according to the requirements.

[0150] Placement feedback: Determine the connection between the plates according to the connection tag, and determine the placement position according to the stacking procedure. Associate the placement position with the connection tag and store it in the QR code tag.

[0151] Specifically, according to the information in the connection tag, determine the connection and relative position between the plates; then, according to the requirements of the stacking procedure, determine the stacking position of the plates. Associate the stacking position with the connection tag and store it in the QR code tag; during subsequent palletizing, these tag information can be used to optimize the palletizing strategy and improve the palletizing efficiency.

[0152] In other embodiments, in the palletizing step, it also includes optimizing the stacking order according to the cutting deviation and offset position.

[0153] Refer to Figure 5 , in other embodiments, in the palletizing step, it also includes: the same-layer deviation statistics step, the quantity judgment step, the order-keeping step, the set position adjustment step, and the stacking execution step;

[0154] Same-layer deviation statistics: Obtain the number of plates with cutting deviations on the same stacking level, denoted as the deviation quantity;

[0155] Quantity judgment: Judge whether the deviation quantity is greater than or equal to the second threshold. If so, execute the order-keeping step; otherwise, execute the set position adjustment step;

[0156] Order-keeping: Keep the order set by the set stacking procedure and execute the stacking execution step;

[0157] Set position adjustment: Determine the stacking position of the same-specification plates according to the set stacking procedure, set the first weight of the plates according to the stacking position, determine the second weight according to the stacking order, then calculate the stacking importance value according to the size of the deviation ratio and the correlation degree of adjacent plates with complementary deviations, adjust the stacking order according to the stacking importance value, and then execute the stacking execution step.

[0158] Specifically, by obtaining all the information of the plates stacked on the same stacking layer and counting the number of plates with cutting deviations, where the cutting deviations can include dimensional deviations, shape deviations, or position deviations, etc. In this embodiment, the dimensional deviation is preferably used. The number of plates with dimensional deviations on the same layer is counted, and according to a preset second threshold, it is determined whether the deviation quantity has reached the critical value for adjusting the stacking order, where the second threshold is determined according to the area of the stacking layer and the total stacking quantity; if the deviation quantity is greater than or equal to the second threshold, it indicates that the deviation situation is relatively serious, and the original stacking order needs to be maintained to reduce the overall impact; if the deviation quantity is less than the second threshold, it indicates that the deviation situation is relatively light, and the stacking effect can be optimized by adjusting the stacking positions; when the deviation quantity is greater than or equal to the second threshold, the system will keep the order set by the original stacking program unchanged and directly execute the stacking execution step. This is to avoid greater deviations or chaos that may be caused by adjusting the order; when the deviation quantity is less than the second threshold, the system will determine the stacking positions of the plates of the same specification according to the set stacking program, and set the first weight and the second weight for each plate according to the stacking positions and the stacking order. Then, the system will calculate the stacking importance value of each plate according to the magnitude of the deviation ratio and the correlation degree of adjacent plates with complementary deviations. Finally, the stacking order is adjusted according to the stacking importance value to optimize the stacking effect.

[0159] After determining the final stacking order and positions, the system will control the execution mechanisms such as the stacking robotic arm or the conveyor belt to stack the plates to the specified positions according to the adjusted order and positions.

[0160] Among them, The calculation model representing the stacking importance value is as follows:

[0161] ;

[0162] Among them, the stacking importance value, represents the number of the current plate, represents the number of the plate adjacent to the current plate, is the weight coefficient set according to the stacking position, is the weight coefficient set according to the stacking order, is the correlation degree coefficient between adjacent plates (which can be set according to the actual situation, such as a higher correlation degree when adjacent and of the same specification), is the plate numbered of the deviation ratio, the plate numbered of the deviation ratio, deviation ratio = cutting deviation amount / allowable deviation amount.

[0163] The calculation of the compensation angle can be carried out based on the specific type and degree of the deviation. For example, for dimensional deviations, the compensation angle can be calculated according to the magnitude and direction of the deviation; for shape deviations, the compensation angle can be calculated according to the shape characteristics of the deviation. At the same time, the degree of association between adjacent plates also needs to be considered to ensure that the adjusted stacking effect is more stable.

[0164] The above are all preferred embodiments of this application, and the protection scope of this application is not limited thereby. Therefore, all equivalent changes made according to the structure, shape, and principle of this application shall be covered within the protection scope of this application.

Claims

1. A vision-based palletizing method, characterized in that: The following steps are involved: Information acquisition: Get the QR code image on the top of the plate and the plate image; Information extraction: From the top QR code label image and the plate image, obtain the physical boundary of the QR code label and its positional relationship with the plate edge; Offset calculation: Calculate the grab compensation angle based on the position deviation of the QR code edge ; Grasping adjustment: adjust the offset posture of the manipulator according to the grasping compensation angle so that the length direction of the grasping manipulator is parallel to the length direction of the plate; Grasp execution: Execute the grasping action according to the compensated posture; Palletizing: According to the program settings, the grabbed boards are stacked at the stacking position; After the information extraction step, an offset distance calculation step is also provided; The offset distance calculation includes an edge extension step, an offset quantification step, and an intervention step; Edge extension: Get the edge of the QR code label from the top QR code label image, extend the edge of the QR code label, and extend it to the edge of the plate; Offset quantification: Calculate the maximum offset based on the edge inclination, the length of the plate, and the minimum distance between the edge of the QR code label and the edge of the plate; Intervention: Intervene in the palletizing step. When palletizing offset plates, adjust the angle so that the side with the largest offset contacts the already palletized plates first, and then execute the palletizing step.

2. The vision-based palletizing method according to claim 1, characterized in that: In the information extraction step, it also includes identifying the QR code information and recording it as specification data; The palletizing step also includes palletizing at the position set by the program according to the specification data.

3. The vision-based palletizing method according to claim 1, characterized in that: The affixing of the QR code label satisfies: The four sides of the label should be parallel or collinear with the corresponding sides of the plate; The offset between the center point of the label and the geometric center of the plate is ≤ 2% of the plate length.

4. The vision-based palletizing method according to claim 1, characterized in that: In the grasping execution step, the middle of the plate is grasped.

5. The vision-based palletizing method according to claim 1, characterized in that: After the intervention step, there is also a board specification feedback step; Plate specification feedback: including size difference calculation step, size judgment step, second judgment step, feedback step, recording step, third judgment step, angle correction step, fourth judgment step and deviation correction step; Calculation of size difference: Calculate the offset difference between the two sides according to the maximum offset between the two sides of the QR code label edge of the same plate in the offset quantification step; Size judgment: According to the offset difference on both sides and the specification width of the plate, it is judged whether there is a cutting deviation. If so, the second acquisition step is executed; Second acquisition: obtaining the actual size specification of the current plate through the plate image, and obtaining the first deviation from the theoretical size specification; Second judgment: Calculate the ratio of cutting deviation to allowable deviation and record it as deviation ratio, and judge whether the deviation ratio is greater than or equal to the first threshold. If so, execute the feedback step, otherwise, execute the third judgment step; Feedback: Feedback the cutting deviation to the cutting side, make cutting corrections, and execute the recording steps; Record: store the cutting deviation data in the QR code label; The third judgment: judging whether there is a plate at one end of the length direction of the plate, if so, executing the angle correction, otherwise, executing the fourth judgment step; Angle correction: Compensation angle for grabbing Perform correction, and after the correction, perform a grasping adjustment step, and perform a fourth judgment step; Fourth judgment: judging whether the cutting deviation is the same as the first deviation, if so, executing the recording step, otherwise, executing the deviation correction step; Deviation correction: Calculate the average of the cutting deviation and the first deviation and record it as the cutting deviation, then perform the recording step.

6. The vision-based palletizing method according to claim 5, characterized in that: Between the third judging step and the fourth judging step, a fifth judging step and a size difference correction step are also provided; Fifth judgment: judging whether there is a plate on one side of the plate width direction, if yes, executing the size difference correction step, otherwise, executing the fourth judgment step; Size difference correction: intervene in the size difference calculation step, recalculate the size difference, and then execute the fourth judgment step.

7. The vision-based palletizing method according to claim 5, characterized in that: After the recording step, there are also steps of establishing associated tags and placing feedback; Establishing associated labels: Record the plates with large cutting deviations, determine the location of the deviations, establish associated labels with adjacent cutting plates, and store them in QR code labels; Placement feedback: Determine the connection between the panels based on the contact tags, and determine the stacking position based on the stacking procedure. Associate the stacking position with the contact tags and store them in the QR code tags.

8. The vision-based palletizing method according to claim 5, characterized in that: The palletizing step also includes optimizing the stacking sequence based on cutting deviation and offset position.

9. The vision-based palletizing method according to claim 8, characterized in that: The palletizing step also includes: a same-layer deviation counting step, a quantity judging step, a sequence keeping step, a position setting adjustment step and a palletizing execution step; Deviation statistics on the same layer: obtain the number of plates with cutting deviations on the same stacking layer, and record it as the number of deviations; Quantity judgment: judging whether the deviation quantity is greater than or equal to the second threshold value, if so, executing the sequence maintaining step, otherwise, executing the setting position adjustment step; Sequence retention: maintain the sequence set by the stacking program and execute the stacking execution steps; Set position adjustment: determine the stacking position of plates of the same specification according to the set stacking procedure, set the first weight of the plates according to the stacking position, determine the second weight according to the stacking order, and then calculate the stacking importance value according to the size of the deviation ratio and the degree of association between adjacent plates with complementary deviations, adjust the stacking order according to the stacking importance value, and then execute the stacking execution steps.

Images