Material positioning method for stacker crane and stacker crane
By combining the improved U-Net model and Canny edge detection algorithm with the OpenCV software library, the problem of inaccurate material positioning was solved, fast and precise positioning of materials was achieved, and the production efficiency of automated palletizing equipment was improved.
Patent Information
- Application Number
- CN202510601406.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-12
- Publication Date
- 2025-09-16
AI Technical Summary
Existing automatic palletizing equipment cannot accurately locate the position of materials on the conveyor belt, resulting in the gripper being unable to accurately grip the materials, affecting production efficiency.
The improved U-Net model is combined with the OpenCV software library for material segmentation and edge detection. The depth camera is used to calculate the actual position of the material. The improved Canny edge detection algorithm is combined to eliminate background interference and obtain the accurate location information of the material.
It achieves fast and accurate positioning of materials, improves production efficiency, ensures that the grippers accurately grasp materials, and improves the operating efficiency of automated palletizing equipment.
Smart Images
Figure CN120646486A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a material positioning method, in particular to a material positioning method for a palletizer and a palletizer. Background Art
[0002] With the development of productivity, automated equipment has been widely used in the field of material palletizing, replacing previous manual palletizing methods. Existing automatic palletizing equipment typically consists of a conveyor belt and a palletizing gripper. The material is transported via the conveyor belt to the gripper, which then palletizes the material. However, this method cannot accurately locate the material on the conveyor belt, resulting in the gripper being unable to accurately grasp the material, causing production line interruptions and affecting production efficiency.
[0003] Therefore, how to accurately locate the position of the material on the conveyor belt to ensure that the clamping claws accurately clamp the material and improve production efficiency has become an urgent problem to be solved. Summary of the Invention
[0004] The present invention provides a material positioning method for a palletizer and a palletizer, which can accurately locate the position of materials and improve production efficiency.
[0005] In a first aspect, a method for material positioning in a palletizer is provided. The method comprises: S1: collecting and labeling material images, and creating a material dataset; S2: using the material dataset to train an improved U-Net model to obtain a material detection model; S3: using a visual positioning module to collect images of materials in a real-world scenario, inputting the material images into the material detection model, and obtaining a segmented image of the material; S4: using an improved Canny algorithm in conjunction with the OpenCV software library to perform edge detection on the segmented image to obtain first position information of the material; S5: processing the first position information to obtain second position information of the material. This method uses an improved U-Net model to segment the material image, eliminating interference from background conveyor belts and other debris, then uses an improved Canny edge algorithm in conjunction with the OpenCV software library to accurately obtain the first position information of the material. Finally, the first position information is processed to obtain the second position information, thereby improving positioning accuracy.
[0006] In combination with the first aspect, in some implementations of the first aspect, step S1 includes: S101: collecting pictures of materials of different specifications and sizes in various postures in actual scenes; S102: labeling the material pictures, labeling the materials as foreground and other content as background.
[0007] It should be understood that the actual specifications and sizes of materials are usually different, and in actual application scenarios, they may be placed in different postures, such as flat, tilted, stacked, etc. Therefore, it is necessary to collect images of materials in these different postures to improve the generalization ability of the model.
[0008] It should be understood that the foreground refers to the object or area of main concern in the image, usually the part that needs to be detected, identified or analyzed. In the scenario of this application, the foreground is the material that needs to be stacked; the background refers to the other parts of the image other than the foreground object, usually including the environment and background debris. In the scenario of this application, the background includes debris, conveyor belts, etc.
[0009] In combination with the first aspect, in certain implementations of the first aspect, the improvement measures for improving the U-Net model include: reconstructing the encoding module of the U-Net model using depthwise separable convolution, improving the downsampling operation in the encoding module, and adding an attention module to the decoding module of the U-Net model.
[0010] It should be understood that the U-Net model is an "encoder-decoder" structure. In its encoder part, the input features are downsampled four times using the maximum pooling method.
[0011] In combination with the first aspect, in some implementations of the first aspect, depthwise separable convolution is used to replace ordinary convolution in the encoding module to reduce the parameters of the model and improve the running speed of the model; maximum pooling is used for the first two downsampling operations, and average pooling is used for the last two downsampling operations. It should be understood that the first two maximum pooling operations can filter low-level semantic features and retain key features, while the last two average pooling operations can retain more key detail features, thereby making the model have better performance; adding an attention module before the first upsampling operation in the decoding module of the U-Net model can enhance the model's attention to important features and improve the accuracy and robustness of feature extraction.
[0012] In combination with the first aspect, in some implementations of the first aspect, step S4 includes: S401: using a bilateral filter to replace the Gaussian filter to filter the segmented image; S402: calculating the gradient intensity and direction of each pixel point in the segmented image; performing non-maximum suppression on the segmented image to obtain a gradient image; S403: using the Otsu algorithm to calculate the gradient image and obtain a double threshold; S404: using double threshold processing and connected domain analysis to detect and connect edges to obtain the contour of the material; S405: using the moments function in the OpenCV software library to obtain the first position information of the contour.
[0013] In combination with the first aspect, in certain implementations of the first aspect, the first position information obtained in step S4 is processed to obtain second position information of the material.
[0014] It should be understood that the first position information includes the position coordinates (x img ,y imgThe second position information includes the coordinates of the center point of the material in the actual space (x world ,y world , z world ).
[0015] In a second aspect, a palletizer is provided, comprising: a visual positioning module, the visual positioning module executing the palletizer material positioning method according to any one of claims 1 to 5; a palletizing module, the palletizing module being used to palletize materials; and the palletizing module (320) being capable of palletizing materials according to second position information obtained by the visual positioning module (310).
[0016] In combination with the second aspect, in certain implementations of the second aspect, the palletizing module includes a moving component, the moving component is used to control the movement of the palletizing module, the visual positioning module is installed at the midpoint of the moving component, and the component can move quickly to the material; a rotating component, including a rotating motor and a hollow rotating platform, the rotating component is connected to the moving component, and the component can palletize materials of different specifications in a specified manner as needed; and a clamping component, the clamping component includes an adjusting device and a clamping device, and the clamping component is connected to the rotating component.
[0017] In conjunction with the second aspect, in certain implementations of the second aspect, the adjustment device includes an adjustment motor and a transmission device, the adjustment motor being transversely connected to the transmission device; the clamping device includes opposing clamping plates, the clamping plates including a clamping motor and clamping plates, the clamping motor controlling the opening and closing of the clamping plates; the clamping plates are slidably connected to the transmission device. The adjustment assembly can be used to control the distance between the clamping plates to accommodate materials of varying sizes; and the clamping structure can be used to quickly palletize materials.
[0018] In combination with the second aspect, in certain implementations of the second aspect, the palletizer further includes a conveying module, the conveying assembly includes a conveyor belt, a tensioning mechanism, and a conveying motor, and the conveyor belt and the incoming material assembly are on the same horizontal plane.
[0019] In a third aspect, a control device is provided, which includes a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions in the memory, so that any method in the first aspect is executed.
[0020] Compared with the existing technology, the beneficial effects of this application are: this method combines the advantages of deep learning and edge detection, obtains the segmented image of the material by improving the U-Net model, eliminates the interference of the conveyor belt background and other debris, and uses the improved Canny edge detection algorithm combined with the OpenCV software library to accurately obtain the contour and position information of the material. Finally, the actual position of the material is calculated through the depth camera, thereby controlling the palletizer to quickly and accurately clamp the stacked materials, effectively improving production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 A schematic structural diagram of a material palletizing system provided in an embodiment of the present application.
[0022] Figure 2 A flow chart of a material positioning method for a palletizer provided in an embodiment of the present application.
[0023] Figure 3 A flow chart for producing a material data set provided in an embodiment of the present application.
[0024] Figure 4 A schematic diagram of the improved U-Net model structure provided in an embodiment of the present application.
[0025] Figure 5 A flow chart of a method for obtaining material location information provided in an embodiment of the present application.
[0026] Figures 6 to 9 A schematic diagram of the structure of a palletizer provided in an embodiment of the present application.
[0027] Figure 10 A schematic diagram of the structure of a control device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0028] The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of this application and the appended claims, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.
[0029] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.
[0030] Accurately locating material positions is crucial for automated palletizers. However, existing methods for locating material positions often have problems such as low efficiency and prone to errors, which restricts production efficiency.
[0031] The embodiments of the present application provide a material positioning method for a palletizer and a palletizer, which can accurately locate the position of materials, achieve rapid and precise palletizing of materials, and improve production efficiency.
[0032] The technical solution in this application will be described below with reference to the accompanying drawings.
[0033] Figure 1 1 is a schematic structural diagram of a material palletizing system 100 provided in an embodiment of the present application. In some examples, the system may include an acquisition system 110, a control system 120, and an operating device 130. It should be understood that the material palletizing system 100 may also include other subsystems, and each of its subsystems may also include one or more smaller subsystems.
[0034] refer to Figure 2 An embodiment of the present application provides a material positioning method for a palletizer, comprising: S1: collecting and labeling material images, and creating a material dataset; S2: using the material dataset to train an improved U-Net model to obtain a material detection model; S3: using a visual positioning module to collect images of materials in a real-world scenario, inputting the material images into the material detection model, and obtaining a material segmentation image; S4: using an improved Canny algorithm combined with the OpenCV software library to perform edge detection on the segmented image to obtain first position information of the material; and S5: processing the first position information to obtain second position information. This method uses an improved U-Net model to segment the material image, eliminating interference from background conveyor belts and other debris, and then uses an improved Canny edge algorithm combined with OpenCV to accurately obtain the material's outline and midpoint information, thereby improving positioning accuracy.
[0035] refer to Figure 3In some examples, step S1 includes, S101: collecting pictures of materials of different specifications and sizes in various postures in actual scenes; S102: annotating the material pictures, annotating the materials as foreground and other contents as background.
[0036] In one possible implementation, LabelMe labeling tool is used to label materials as foreground and other contents as background to obtain a material dataset.
[0037] In some examples, the improvement measures for improving the U-Net model in step S2 include: reconstructing the encoding module of the U-Net model using depthwise separable convolution, improving the downsampling operation in the encoding module, and adding an attention module to the decoding module of the U-Net model.
[0038] It should be understood that the U-Net model is an "encoder-decoder" structure. In its encoder part, the input features are downsampled four times using the maximum pooling method.
[0039] In one possible implementation, the ordinary convolution in the encoding module is replaced by depthwise separable convolution, the first two downsampling operations are performed using maximum pooling, the last two downsampling operations are performed using average pooling, and an attention module is added before the first upsampling operation in the decoding module of the U-Net model. The structure of the improved U-Net model is as follows: Figure 4 shown.
[0040] It should be understood that by replacing the ordinary convolution in the encoding module with depth-separable convolution, the number of parameters and the amount of calculation of the model are significantly reduced, thereby improving the running speed and making the model more suitable for running in an environment with limited computing resources. Use maximum pooling for the first two downsampling operations and use average pooling for the last two downsampling operations. It should be understood that the first two maximum pooling operations can filter low-level semantic features and retain key features, while the last two average pooling operations can retain more key detail features, so that the model has better performance. In addition, adding an attention module before the first upsampling operation in the decoding module of the U-Net model can enhance the model's attention to important features, enhance its adaptability and accuracy in complex scenarios, and thus improve the overall segmentation performance. The embodiment of the present application does not limit the type of attention module added.
[0041] In some examples, reference Figure 5Step S4 includes: S401: using a bilateral filter to replace the Gaussian filter to filter the segmented image; S402: calculating the gradient intensity and direction of each pixel in the segmented image; S403: performing non-maximum suppression on the segmented image to obtain a gradient image; S404: using the Otsu algorithm to calculate the gradient image and obtain a double threshold; S405: using double threshold processing and connected domain analysis to detect and connect edges to obtain the contour of the material; S406: using the moments function in the OpenCV software library to obtain the first position information of the contour.
[0042] In some examples, the first location information obtained in step S4 is processed to obtain second location information of the material.
[0043] It should be understood that the first position information includes the position coordinates (x img ,y img The second position information includes the position coordinates of the center point of the material in the actual space (x world ,y world , z world ).
[0044] In one possible implementation, a depth camera is used to obtain (x img ,y img ) corresponds to the depth coordinate z img , calibrate the 3D depth camera through the calibration board to obtain the camera's internal parameters (focal length f x , f y , principal point coordinates c x , c y ) and external parameters (rotation matrix R and translation vector T), the internal parameter matrix K is: , (x img ,y img , z img ) is converted to the coordinates of the depth camera coordinate system (x can ,y can , z can ), the specific conversion formula is as follows: , Finally, the coordinates in the depth camera coordinate system (x can ,y can , z can ) is converted to real space coordinates (x world ,y world , z world ), the specific conversion formula is as follows: , (x world,y world , z world ) is the position information of the material center point in the actual space.
[0045] Figure 6 Schematic diagram of the structure of a palletizer 300 provided in an embodiment of the present application.
[0046] In some examples, the palletizer may include: a vision positioning module 310, which performs Figures 1 to 5 A palletizer material positioning method according to any one of the above; a palletizing module 320, the palletizing module 320 is used to palletize materials; the palletizing module (320) can palletize the materials according to the second position information obtained by the visual positioning module (310).
[0047] In some examples, the palletizing module 320 may include a moving component 321, which is used to control the movement of the palletizing module 320, and the visual positioning module 310 is installed on the moving component 321; a rotating component 322, which includes a rotating motor 3221 and a hollow rotating platform 3222, and the rotating component 322 is connected to the moving component 321; and a clamping component 323, which includes an adjusting device 3231 and a clamping device 3232, and the clamping component 323 is connected to the rotating group 322.
[0048] refer to Figure 6 In some examples, the adjusting device 3231 includes an adjusting motor 3231-1 and a transmission device 3231-2, and the adjusting motor 3231-1 is connected to the transmission transposition 3231-2; the clamping device 3232 includes a relatively arranged splint structure, and the splint structure includes a clamping motor 3232-1 and a splint 3232-2, and the clamping motor 3232-1 controls the opening and closing of the splint 3232-2; the splint 3232-1 is slidably connected to the transmission device 3231-2.
[0049] In one possible implementation, refer to Figure 7 and Figure 8 The moving assembly 321 includes a moving device 3211 and a moving motor 3212. The moving assembly 321 is provided with a slide rail 3213, and the rotating assembly 322 is provided with a slide rail interface 3223. The rotating assembly 322 is slidably connected to the moving assembly 321.
[0050] In one possible implementation, refer to Figure 9 The clamping assembly 323 is provided with a connection interface 3233 , and the clamping assembly 323 is connected to the rotating assembly 322 through the connection interface 3233 .
[0051] In some examples, the palletizer 300 further includes a conveying module 330 , which includes a conveyor belt 331 , a tensioning mechanism 332 , and a conveyor motor 333 . The conveyor belt 331 is on the same horizontal plane as the incoming material assembly.
[0052] Figure 10 This is a schematic diagram of a control device 400 provided in an embodiment of the present application. The device includes a processor 410 and a memory 420. The processor 410 is coupled to the memory 420. The memory 420 is used to store computer programs or instructions. The processor 410 is used to execute the computer programs or instructions in the memory 420 so that any one of the methods in the above embodiments is executed.
[0053] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed in the present invention should be included in the protection scope recorded in the claims.
Claims
1. A material positioning method for a palletizer, characterized in that: include: S1: Collect material images, annotate the material images, and create a material dataset; S2: Using the material dataset to train the improved U-Net model to obtain a material detection model; S3: Using the visual positioning module to collect actual images, inputting the actual material images into the material detection model to obtain material segmentation images; S4: performing edge detection on the segmented image using an improved Canny algorithm combined with the OpenCV software library to obtain first position information of the material; S5: Process the first location information to obtain the second location information of the material.
2. The material positioning method for a palletizer according to claim 1, characterized in that: The step S1 comprises: S101: Collecting pictures of the material of different specifications and sizes in various postures in actual scenes; S102: Label the material image, label the material as the foreground, and label other contents as the background.
3. The material positioning method for a palletizer according to claim 1, characterized in that: The improvement measures for improving the U-Net model in step S2 include: reconstructing the encoding module of the U-Net model using depthwise separable convolution, improving the downsampling operation in the encoding module, and adding an attention module to the decoding module of the U-Net model.
4. The material positioning method for a palletizer according to claim 3, characterized in that: The depth-wise separable convolution is used to replace the ordinary convolution in the encoding module, the maximum pooling is used for the first two downsampling operations, the average pooling is used for the last two downsampling operations, and an attention module is added before the first upsampling operation in the decoding module of the U-Net model.
5. The material positioning method for a palletizer according to claim 1, characterized in that: The step S4 comprises: S401: Using a bilateral filter to replace the Gaussian filter to filter the segmented image; S402: Calculating the gradient intensity and direction of each pixel in the segmented image; S403: Perform non-maximum suppression on the segmented image to obtain a gradient image; S404: Calculate the gradient image using the Otsu algorithm to obtain a double threshold; S405: using double threshold processing and connected domain analysis to detect and connect edges to obtain the outline of the material; S406: Utilize the moments function in the OpenCV software library to obtain first position information of the contour.
6. A palletizer, characterized in that: include, A visual positioning module (310), wherein the visual positioning module (310) executes the palletizer material positioning method according to any one of claims 1 to 5; A palletizing module (320), the palletizing module (320) is used to palletize the materials; The palletizing module (320) is capable of palletizing the material according to the second position information acquired by the visual positioning module (310).
7. The palletizer according to claim 6, characterized in that The palletizing module (320) comprises: A moving component (321), the moving component (321) is used to control the movement of the palletizing module (320), and the visual positioning module (310) is installed on the moving component (321); A rotating assembly (322), the rotating assembly (322) comprising a rotating motor (3221) and a hollow rotating platform (3222), the rotating assembly (322) being connected to the moving assembly (321); A clamping assembly (323), comprising an adjusting device (3231) and a clamping device (3232), and the clamping assembly (323) is connected to the rotating assembly (322).
8. The palletizer according to claim 7, characterized in that: The adjusting device (3231) comprises an adjusting motor (3231-1) and a transmission device (3231-2), wherein the adjusting motor (3231-1) is connected to the transmission transposition (3231-2); the clamping device (3232) comprises a clamping plate structure arranged opposite to each other, wherein the clamping plate structure comprises a clamping motor (3232-1) and a clamping plate (3232-2), wherein the clamping motor (3232-1) controls the opening and closing of the clamping plate (3232-2); and the clamping plate (3232-2) is slidably connected to the transmission device (3231-2).
9. The palletizer according to claim 6, characterized in that: The palletizer further comprises a conveying module (330), the conveying assembly comprising a conveying belt (331), a tensioning mechanism (333) and a conveying motor (333), and the conveying belt (331) and the incoming material assembly are on the same horizontal plane.
10. A control device comprising a processor (410) and a memory (420), wherein the processor (410) is coupled to the memory (420), the memory (420) is used to store computer programs or instructions, and the processor (410) is used to execute the computer program or instructions in the memory (420), so that the method according to any one of claims 1 to 5 is executed.