System for placing a label on an object and method therefor

Abstract

A system for placing a label on an object, comprising: an image capturing module for capturing an image of a real-world environment having an object with a label placed thereon; an object recognition processor for processing the captured image to identify a point cloud associated with a region on a surface of the object, wherein the object recognition processor is configured to determine a suitable label placement location based on a combination of the identified point cloud and pre-set parameters associated with the label; and a robotically controlled device for applying the label to the suitable label placement location such that the label adheres to the object. A method for placing a label on an object is also disclosed.

Description

Technical Field

[0001] This disclosure relates to systems and methods for placing tags on objects, and more specifically, but not exclusively, to a system and method for placing tags on objects, wherein the properties of the objects are unknown prior to scanning the objects. Background Technology

[0002] E-commerce encompasses a wide range of businesses, from consumer-facing retail websites to auction or music sites, and commercial transaction platforms facilitating the trading of goods and services between overseas companies. E-commerce warehouses have high turnover rates for goods or stored items, with a small quantity but a wide variety of products. However, existing automation or robotic technologies may not be suitable for the daily operations of e-commerce warehouses.

[0003] For example, stored items and the boxes used to pack them often have irregular shapes, forms, and colors, and are made of materials with varying degrees of hardness. Scanning objects based on preset parameters associated with labels to find suitable label placement locations on these objects becomes inefficient and slow. Summary of the Invention

[0004] According to a first aspect of this disclosure, a system for placing a label on an object is provided, comprising:

[0005] An image capture module, used to capture images of a real-world environment with objects for placing labels;

[0006] An object recognition processor is used to process a captured image to identify a point cloud associated with a region on the surface of the object, wherein the object recognition processor is used to determine a suitable label placement position based on a combination of the identified point cloud and preset parameters associated with the label; and

[0007] A robot-controlled device for applying the label to the appropriate label placement location so that the label adheres to the object.

[0008] In one embodiment of the first aspect, the object recognition processor is used to filter data points of the recognized point cloud based on preset parameters, thereby removing data points unrelated to the object from the captured image.

[0009] In one embodiment of the first aspect, the object recognition processor is used to perform local filtering based on the relationship between adjacent data points.

[0010] In one embodiment of the first aspect, the object recognition processor is configured to merge filtered data points into one or more segments associated with a region on the surface of the object.

[0011] In one embodiment of the first aspect, the object recognition processor is used to compare merged segments with one or more preset criteria to identify candidate label placement locations on the surface of the object.

[0012] In one embodiment of the first aspect, the object recognition processor is used to process larger merged segments with a higher priority than smaller merged segments.

[0013] In one embodiment of the first aspect, the object recognition processor is used to remove data points on surfaces that are far from the identified candidate label placement location, thereby reducing the dimensionality of the data points associated with the identified candidate label placement location.

[0014] In one embodiment of the first aspect, the object recognition processor is used to prioritize the data points among multiple identified candidate label placement locations based on the one or more preset criteria.

[0015] In one embodiment of the first aspect, the preset criteria include sorting the data points based on the distance of the candidate label placement position from the centroid of the object.

[0016] In one embodiment of the first aspect, the object recognition processor is used to compare the surface area of ​​the candidate label placement location with the surface area of ​​the label.

[0017] According to a second aspect of this disclosure, a method for placing a label on an object is provided, comprising:

[0018] Capture images of a real-world environment with objects for placing labels;

[0019] The captured image is processed using an object recognition process to identify point clouds associated with regions on the surface of the object;

[0020] Based on the identified point cloud and a combination of preset parameters associated with the label, the appropriate label placement location is determined; and

[0021] The label is applied to the appropriate label placement location so that the label adheres to the object.

[0022] In one embodiment of the second aspect, the object recognition process is used to filter data points of the recognized point cloud based on preset parameters, thereby removing data points unrelated to the object from the captured image.

[0023] In one embodiment of the second aspect, the object identification process is used to perform local filtering based on the relationship between adjacent data points.

[0024] In one embodiment of the second aspect, the object identification process is used to merge filtered data points into one or more segments associated with a region on the surface of the object.

[0025] In one embodiment of the second aspect, the object identification process is used to compare merged segments with one or more preset criteria to identify candidate label placement locations on the surface of the object.

[0026] In one embodiment of the second aspect, the object identification process is used to process larger merged segments with a higher priority than smaller merged segments.

[0027] In one embodiment of the second aspect, the object recognition process is used to remove data points on surfaces that are far from the identified candidate label placement location, thereby reducing the dimensionality of the data points associated with the identified candidate label placement location.

[0028] In one embodiment of the second aspect, the object recognition process is used to prioritize the data points among multiple identified candidate label placement locations based on the one or more preset criteria.

[0029] In one embodiment of the second aspect, the object identification process is used to sort the data points based on the distance of the candidate label placement position from the centroid of the object.

[0030] In one embodiment of the second aspect, the object recognition process is used to compare the surface area of ​​the candidate label placement location with the surface area of ​​the label. Attached Figure Description

[0031] The embodiments of this disclosure will now be described by way of example with reference to the accompanying drawings.

[0032] Figure 1 A schematic diagram of a system according to an embodiment of the present disclosure is shown;

[0033] Figure 2 The system according to an embodiment of the present disclosure is shown for detecting the optimal label position on a curved suitcase;

[0034] Figure 3 The system according to an embodiment of the present disclosure is shown for detecting the optimal label position on a box;

[0035] Figure 4 The system according to an embodiment of the present disclosure is shown detecting the optimal label position on a chair;

[0036] Figure 5 A schematic diagram illustrating conventional template matching that suffers from several drawbacks of the solutions provided in this disclosure;

[0037] Figure 6 A flowchart of a method according to an embodiment of this disclosure;

[0038] Figure 7 A schematic diagram of a method according to an embodiment of the present disclosure; and

[0039] Figure 8 This is a flowchart of a method according to another embodiment of the present disclosure. Detailed Implementation

[0040] Without wanting to be limited by theory, the inventors, through their own research, experimentation, and research, discovered that existing systems and methods for placing tags on objects require scanning every single point of the object, which is computationally intensive and slow. The efficiency of existing systems and methods is limited by the tedious scanning and processing time, and therefore may not be suitable for the daily operations of e-commerce warehouses.

[0041] To address or mitigate at least some of the aforementioned problems, this disclosure provides a novel label adhesion system and method. This system and method employ enhanced computer vision algorithms.

[0042] Advantageously, the system and method can utilize low-cost depth sensors and run on non-GPU machines, making them suitable for edge computing. By employing computational algorithms including region growing and template matching, this disclosure provides a deterministic label adhesion method.

[0043] refer to Figure 1 A system 10 for placing a label 20 on an object 30 is provided, comprising: an image capture module 110 for capturing an image of a real-world environment having the object 30 on which the label 20 is placed; an object recognition processor 120 for processing the captured image to identify a point cloud 40 associated with a region on the surface of the object 30, wherein the object recognition processor 120 is used to determine a suitable label placement position 42 based on a combination of the identified point cloud 40 and preset parameters associated with the label 20; and a robot-controlled device 200 for applying the label 20 to the suitable label placement position 42, such that the label 20 adheres to the object 30.

[0044] In this embodiment, a system 10 is provided, incorporated in the form of a service robot. System 10 includes a scanning module 100, a robot-controlled device 200, a target object 30 (e.g., luggage to be scanned by the scanning module 100), a tag 20 (e.g., a barcode, RFID tag, or any other identifier with a delivery address or other stored information) for affixing to the scanned object 30, and a tag substrate (not shown) (e.g., an automated tag dispenser for providing the tag 20). System 10 can effectively replicate manual tag affixing tasks, improving efficiency and the overall workflow of luggage handling. Before the scanning module 100 scans the object 30, the properties of the object 30, such as its size, shape, surface profile, functional orientation, material composition, stiffness, deflection characteristics, etc., are unknown.

[0045] Preferably, the scanning module 100 further includes an image capture module 110 and an object recognition processor 120, such as a computer 122. The image capture module 110 is used to capture an image of the object 30, while the object recognition processor 120 is used to identify areas on the surface of the object 30 as suitable label placement locations. In an example embodiment, such a suitable label placement location may be a planar area or surface of the object 30 with an irregular shape. A label 20 of arbitrary shape can then be applied to the suitable label placement location and uniformly fixed to the object 30, thereby assigning a unique identifier to the object 30.

[0046] Optionally, the image capture module 110 may also include a depth sensing module or a depth camera 110 to capture images of the object 30 from various orientations, thereby more effectively identifying the most suitable label placement position among multiple planes of the three-dimensional object 30. For example, the depth camera 110 can provide the object recognition processor 120 with the position (e.g., depth information) of the object 30, enabling timely and accurate tracking of the relative position of the object 30 even when the object 30 is in arbitrary, unpredictable, non-uniform motion. More importantly, the shape and contour of the object 30 can also be determined even when the object 30 is not stationary.

[0047] On the other hand, the robot-controlled device 200 may include a robot component 210, such as an industrial arm, like a triangular robot or a Cartesian robot, and a controller 220, such as a microcontroller for embedded applications that manipulates the movement of the robot component 210 in three-dimensional space based on programmable input / output and data algorithms. The robot component 210, actuated by the controller 220, can extract a label 20 from a label substrate and apply the label 20 to a suitable label placement location. For example, the robot component 210 may include a pneumatic actuator 300. The pneumatic actuator 300 is used to pneumatically extract / pick up the label 20 from the label substrate by applying pneumatic suction pressure in a suction manner, and to pneumatically apply / place the label 20 onto the object 30 by applying pneumatic blowing pressure in a blowing manner. Alternatively, a movable end effector 300 may be used to roll the label 20 onto the object 30. More advantageously, a flexible end effector 300 may also be used to roll the label 20 onto an irregularly shaped object 30.

[0048] Preferably, the object recognition processor 120 can identify data points on the object 30 to identify a general planar area on the object 30 as a suitable label placement location. Since the attributes of the object 30 (such as the size, shape, surface profile, functional orientation, material composition, deflection characteristics, etc.) are unknown, identifying planar areas on the object 30 has a significant advantage, thereby enabling the selection of the most suitable label placement location, such as the location with the largest available planar area.

[0049] refer to Figures 2 to 4 Examples are shown for various objects 30, which are suitable for detecting the optimal label position 50 by the application system 10. For example, the depth camera 110 can capture a 3D point cloud 40 of an object 30 in a real-world environment, and the object recognition processor 120 can determine the optimal label position 50 on the object 30 based on the data points of the 3D point cloud 40.

[0050] For example Figure 2 Using the curved suitcase 30a as an example, the depth camera 110 can capture a 3D point cloud 40 of the curved suitcase 30a and identify the curvature of the surface of the suitcase 30a. Surfaces that meet preset parameters are marked as candidate label placement positions 42, and the positions that meet the conditions will be the optimal label positions 50 for attaching the labels 20. The optimal label positions 50 for the box 30b and the chair 30c can also be determined accordingly using a similar algorithm.

[0051] For a detailed description of the functions of the object recognition processor 120, please refer to [link / reference]. Figures 5 to 6 .

[0052] Figure 5A schematic diagram 500 depicts a typical flowchart for determining the optimal location to attach label 20 to object 30. A simple approach is to apply template matching to each point in point cloud 40. However, this is a computationally intensive and slow method and does not solve the problem.

[0053] Figure 6 A method and system 600 for automatic label adhesion using point cloud data 40, according to an exemplary embodiment of this disclosure, are described. The method 600 ensures efficient and accurate identification of the optimal surface for label adhesion by utilizing point cloud data 40, local screening, and advanced geometry and template matching techniques.

[0054] Method 600 begins with step 610, which involves capturing a 3D point cloud 40. The system may use depth sensor 110 to capture a 3D point cloud 40 of the scene, which includes the object 30 to be labeled. After step 610, method 600 may proceed to step 620, which involves applying local filters. The system may apply local filters to each data point in the point cloud 40. For example, the system may implement criteria to significantly remove unsuitable points, ensuring that only relevant data points are retained.

[0055] Next, method 600 can proceed to step 630, which involves classification based on local attributes. The system can sort the remaining points based on their local attributes. In subsequent steps 640 to 650, the system can prioritize testing the most suitable points. Next, method 600 can proceed to step 640, which involves candidate point identification. For each point, the system can identify nearby points to form candidate regions. The system can also apply local fitting to reduce the data dimensionality by one dimension, as fitting typically reduces dimensionality.

[0056] Finally, method 600 can proceed to step 650, which involves geometric alignment and template matching. The system can use geometric transformations to align the shape of the candidate region with the target template. For example, the system can apply correlation-based template matching to ensure that the candidate region matches the label template 20.

[0057] The following reference is used to place label 20 as follows: Figure 7 The workflow of method 700 on object 30 is shown, and method 600 is further described.

[0058] Method 700 begins with steps 710 of capturing multidimensional data and filtering the captured multidimensional data. Based on the specific problem, local filtering is applied to remove most of the data. This filtering process is consistent with rejection criteria at the local level and can be efficiently computed using nearest neighbor points. For example, the local curvature of the surface, i.e., the flatness of the surface on object 30, can be determined based on data point 712. Complex shapes 714 are excluded from potential candidate points, and data points are removed from region 716.

[0059] Following step 710, method 700 may proceed to step 720, which involves running an optional merging algorithm to cluster the input data points into segments for parallel processing or reject segments that do not meet suitable criteria. These points are merged into multiple distinct regions, such as 721 to 726, and the merged segments 721 to 726 are selected based on criteria. For example, a merged segment is only considered sufficient to adhere to the label 20 if its surface area exceeds a preset threshold. If the surface area of ​​segment 722 is too small, all data points in that segment are discarded. Method 700 may then proceed to step 730, which prioritizes the merged segments. To allocate resources more efficiently, the larger segment 722 is processed first by processor 120 on top of the remaining segments 721 and 723 to 726.

[0060] Following step 730, method 700 can then proceed to step 740, which projects data points onto a 2D plane and removes points far from segment 722. After projection, the dimensionality of the data points is reduced, making template matching in subsequent step 770 more efficient. In this process, the 3D region of segment 722 is approximated as a 2D plane. For example, a cross-sectional side view of 722 can be simulated by a surface 732 formed by data points relative to axis 734, showing that segment 722 is not flat relative to axis 734. To reduce the number of data points to be processed by processor 120, those data points 736 furthest from axis 734 are removed.

[0061] Then, method 700 can proceed to step 750, which transforms the data from 3D to 2D. Here, a portion of segment 754 is removed from segment 722 to obtain a 2D plane 752 composed of reduced data points. Furthermore, the data points can be sorted based on a criterion of prioritizing candidates most likely to meet the requirements. Once enough data points are found to solve the problem, the algorithm terminates.

[0062] Subsequently, method 700 can proceed to step 760, determining the distance from the centroid standard and classifying the points based on the centroid standard. A label 761 is placed on a 2D plane 762, and multiple potential points 764, 766, and 768 are identified. Preferably, label 761 will adhere better to the center of the area formed by potential points 764, 766, and 768. These points will be sorted based on their distance from the center of the 2D plane 762. The potential point with the shortest distance to the center of the 2D plane 762 will be selected.

[0063] Finally, method 700 can proceed to the template matching step 770. For example, a mesh-based solution is provided for template matching between label 772 and candidate surface 774 to adhere label 772 thereto. When counting support points, a bin counting method can be used to count sufficient surfaces for adhering label 772. The mesh density of bin 776 can be based on the attribute definition of label 772, i.e., the mesh roughness can be adjusted based on the problem. If the occupancy count meets a threshold, the test point is a selected candidate point. In the template matching step 770, the data can be further refined based on criteria such as transitioning from dense shapes to peripheral points to accelerate processing by processor 120.

[0064] In one exemplary embodiment, an automated label adhesion system 10 for conveyor belt operation is provided for placing labels 20 onto objects 30, such as items to be transported along a transport system 400, such as a conveyor belt system 400 for transporting luggage 30. Figure 1 As shown. System 10 preferably includes a depth sensor 110, a micro workstation 122, and a robotic arm 200 equipped with a pneumatic end effector 300. The micro workstation 122 may be a computer console / processor 120 for controlling the depth sensor 110 and the robotic arm 200. The depth sensor 110 captures 3D point clouds 40 of objects 30 on the conveyor belt 400. In system 10, a large number of objects 30 with irregular shapes, forms, colors, and made of materials with different stiffnesses are transported along the conveyor belt system 400, therefore the micro workstation 122 needs a fast processing speed. The micro workstation 122 may store computational algorithms for locating the optimal surface for label adhesion.

[0065] Preferably, the scanning module 100 is used to scan each of a plurality of objects 30 moving along the conveyor belt by the conveyor belt system 400, for example by capturing images of the objects 30 for a suitable label placement position 42 using the depth sensor 110 and based on preset parameters associated with the plurality of labels 20. The captured images can be sent to the computer console 120 for image analysis and object recognition. In an exemplary embodiment, the object recognition processor 120 can, for example, identify point cloud data 40 of irregularly shaped objects 30 by using a computational algorithm to identify a generally planar region on the object 30 and thereby select the most suitable planar region, for example, the largest planar region that can be used as the suitable label placement position 42. Alternatively, the suitable label placement position 42 can be determined based on other user-preset criteria, for example, the shape of the label 20 in addition to the size of the planar region or in addition to the size of the planar region.

[0066] A robotic arm 200, commanded by a computer console 120, extracts each tag 20 from a tag source and moves the tag 20 adjacent to a suitable tag placement position 42 for each scanned object 30. Optionally, the speed of the conveyor belt 400 can be monitored and adjusted by the computer console 120 to allow for sufficient separation of the objects 30.

[0067] The robotic arm 200 is also used to apply each label 20 to a suitable label placement position 42, such that each label 20 can be individually adhered to and placed on multiple objects 30. Optionally, the label 20 can be placed on the object by the robotic arm 200 with the assistance of an effector 300 attached thereto. Those skilled in the art will understand that system 10 can also be implemented in a transport system including multiple conveyor belt systems 400.

[0068] Final Reference Figure 8 The following describes a computational algorithm operation workflow 800 according to an exemplary embodiment of the present disclosure. The operation workflow 800 for locating the optimal surface for label adhesion may include steps 810 to 850.

[0069] Method 800 begins with frame analysis and object detection step 810. Initially, step 810 may begin with substep a) of capturing raw point cloud data for each frame. First, the raw point cloud for each frame contains thousands of points, and not every frame includes the object of interest 30. This initial substep a) is used to determine whether a frame contains object 30. Then, step 810 may proceed to substep b) of calibrating camera 110 using a checkerboard pattern. Based on the hardware setup, the computational algorithm can calibrate camera 110 using a checkerboard pattern.

[0070] Next, step 810 can proceed to substep c) transforming the point cloud coordinates 40 to align with the conveyor belt 400. For each point cloud frame, the computation algorithm can apply a transformation to align the coordinates with the conveyor belt 400. Then, step 810 can proceed to substep d) defining a space that matches the width of the conveyor belt 400 and the maximum permissible luggage height. The computation algorithm can define a space that matches the width of the conveyor belt 400, where the height is determined by the maximum length of the permissible luggage size.

[0071] Next, step 810 can proceed to substep e) to remove points outside the defined space, and then to substep f) to apply a voxel grid filter to ensure a uniform point distribution and reduce the point cloud size. Here, the computational algorithm can apply a voxel grid filter to ensure a uniform point distribution and further reduce the number of points in cloud 40. This process helps maintain a manageable and efficient dataset for subsequent processing steps 820 to 850. After substep f), if the number of remaining points is below a preset threshold, the computational algorithm can conclude that there is no object 30 on the conveyor belt 400 below camera 110. Therefore, the computational algorithm will skip the current frame.

[0072] Then, method 800 can proceed to step 820, which identifies the primary flat surfaces. Initially, step 820 may begin with sub-step a) of estimating the local surface properties at each 3D point using parallel processing. The computational algorithm can estimate the local surface properties at each 3D point in parallel using the OpenMP standard.

[0073] Then, step 820 can proceed to substep b), where the neighborhood of each point is analyzed using a K-dimensional tree (KD-tree) to compute the normal of each point for an efficient nearest neighbor search. To compute the normal of a specific point, its neighborhood must be analyzed, including surrounding points. Typically, this involves selecting the N nearest points or all points within a certain radius around the target point. If the point cloud 40 is unstructured, identifying these neighboring points can be challenging. KD-trees are used to accelerate this search process because they are optimized data structures capable of enabling very fast nearest neighbor searches.

[0074] Next, step 820 can proceed to substep c) to filter out points that deviate from the viewpoint by more than a certain angle. After estimating the normal of each point based on its neighboring points, the computation algorithm can perform a dot product operation between the normal vector of each point and a unit vector along the z-axis. This substep filters out points that deviate from the viewpoint by more than a certain angle because such points may be downward and inaccessible to the robot arm 200.

[0075] Next, step 820 can proceed to substep d) of applying region growing segmentation to merge points into clusters representing smooth surfaces. The computational algorithm can apply region growing segmentation. This algorithm merges points that are sufficiently close in terms of smoothness constraints to produce clusters, where each cluster represents a group of points belonging to the same smooth surface. The algorithm operates by comparing the angles between the normals of the points. The computational algorithm empirically sets a smoothness threshold, a curvature threshold, and a minimum cluster size. Finally, step 820 can proceed to substep e) of selecting the largest flat surface cluster for further processing. Therefore, the resulting clusters are primarily planar surfaces. From these clusters, the computational algorithm can select the largest surface for further processing.

[0076] Then, method 800 can proceed to step 830, which transforms the working coordinates from 3D to 2D. Initially, step 830 can begin with sub-step a) of applying plane segmentation to identify planar support points. After identifying clusters on the planar surface, the computational algorithm can apply the plane segmentation to the set of points using the Random Sample Consensus (RANSAC) algorithm. This algorithm helps to find all points within the point cloud 40 that support the planar model.

[0077] Then, step 830 can proceed to substep b) to calculate the angle between the identified plane and the vertical plane. Once the plane model coefficients are obtained, the calculation algorithm can calculate the angle between the identified plane and the vertical plane. Then, step 830 can proceed to substep c) to apply an appropriate transformation matrix to the motion matrix of the robot arm 200. This calculation allows the application of an appropriate transformation matrix when determining the motion matrix of the robot arm 200. Finally, step 830 can proceed to substep d) to work in the planar coordinate system to identify the optimal label position 50.

[0078] Then, method 800 can proceed to step 840, which involves sequentially filtering candidate points for label adhesion. Initially, step 840 may begin with sub-step a) of identifying planar regions and locating sufficiently flat areas for label adhesion. At this stage, the computational algorithm has identified the planar regions, and the next task is to locate sufficiently flat areas within these regions for label adhesion. Generally, the selected locations should avoid edge areas.

[0079] Then, step 840 can proceed to substep b) to apply template matching by calculating the average value of points within a cluster and normalizing the data. To find regions, the computational algorithm can apply the classic method of template matching. The computational algorithm can first calculate the average value of points within a cluster in the x, y, and z coordinates, and normalize the data by subtracting this average value from each point. For template matching in substep b), the computational algorithm can employ a greedy algorithm, sequentially testing candidate points from the center of the region outwards. Next, the computational algorithm can create a point cloud of labels 20 to be used as templates. Labels 20 are typically rectangular, so the computation can programmatically generate rectangular templates. The computation can define the shape formed by these points as (T).

[0080] Next, step 840 can proceed to sub-step c) to classify points according to their distance from the centroid and test candidate points from the center outwards. Subsequently, the calculation algorithm can sort the points in the point cloud 40 according to their distance from the centroid, starting with those points closest to the center of the plane.

[0081] Then, step 840 can proceed to substep d) of using a KD tree to identify points within a specified radius and evaluating candidate regions for label adhesion. For each candidate point cloud, the computational algorithm can utilize the KD tree to identify points within a specified radius, which is equivalent to half the diagonal of the rectangular label 20, and count the number of points within that circle. The computational algorithm can define the shape formed by these points as (A).

[0082] The computational algorithm can then apply a formula to evaluate whether the candidate region has sufficient surface area for label adhesion. For example, if the candidate point is at the edge of the region, the circle (A) can resemble a half-pie shape. In this case, imagine a rectangular label 20 within that region; there might not be enough points to firmly adhere the label 20. In this way, unsuitable candidate regions can be quickly rejected.

[0083] The first criterion is that if the number of points in (A) is less than the number of points in the label template, the candidate region can be ignored. The second criterion further eliminates candidate regions based on geometric properties. Since the rectangle (T) is inscribed in the circle (A), the diagonal of the rectangle is equal to the diameter of the circle. Therefore, the radius (r) of the circle can be represented by the length (x) and width (y) of the rectangle:

[0084]

[0085] Using the area formula:

[0086] Area of ​​a circle = πr 2

[0087] Area of ​​rectangle = π × y

[0088] We can calculate the ratio between the area of ​​the circle and the area of ​​the rectangle. The ratio is given by the following formula:

[0089] Ratio = Area of ​​circle / Area of ​​rectangle = πr 2 / π×y

[0090] Now, we can substitute the expression for (r) into the formula and simplify it to:

[0091] Ratio = π(x) 2 +y 2 ) / 4xy

[0092] Therefore, by inputting the dimensions of label 20 into the formula, the calculation algorithm can determine the precise ratio under ideal conditions.

[0093] Next, step 840 can proceed to the sub-step e) of applying criteria to reject unsuitable candidates based on point counts and geometric characteristics. The calculation algorithm can apply strict criteria: if the ratio of the current candidate points is not greater than a certain factor (e.g., 1.3 times), the calculation algorithm can reject the candidate point due to its less-than-ideal position for label attachment.

[0094] Finally, method 800 can then proceed to the final step 850 of label template matching. Initially, step 850 can begin with substep a) of performing eigenvector decomposition to align candidate shapes with label template 20. The computation algorithm can apply eigenvector decomposition to find the longest axis of shapes (T) and (A). The computation algorithm can then calculate the angle between the label template shape (T) and the candidate shape (A) and align (A) and (T) along their longest axis.

[0095] Then, step 850 can proceed to substep b) to create a discrete bin table for the label template and check whether all bins are filled with candidate points. Subsequently, the computational algorithm can create a discrete bin table inspired by the Hough Transform, albeit in a simplified manner. The label template shape (T) is discretized into bins of n rows by m columns. For each point in the candidate shape (A), the computational algorithm can determine whether it falls into any bin in (T).

[0096] Finally, step 850 can proceed to sub-step c) to confirm that the candidate shape meets the criteria of a sufficiently flat and dense location for label adhesion. After iterating over all points, the computational algorithm can check whether all boxes are filled. If so, the candidate shape meets the criteria of a sufficiently flat and dense location for label adhesion.

[0097] In an alternative exemplary embodiment, another system is provided for attaching a label to an irregularly shaped object 30. The system also includes a depth sensor 110, a micro-workstation 122, and a robotic arm 200 with a flexible end effector 300. The depth sensor 110 can capture a 3D point cloud of the irregularly shaped object.

[0098] The operation of this alternative embodiment is similar to the operation workflow 800 described above, except that step 810 includes a sub-step c) of converting point cloud coordinates to align with the workspace rather than the conveyor belt 400, and a sub-step d) of defining a space that matches the expected size of the object rather than the width of the conveyor belt 400 and the maximum allowable luggage height. The remainder of steps 810 to 850 and their sub-steps are the same as those in operation workflow 800 and are not repeated.

[0099] Preferably, all these embodiments illustrate the practical application of the method in different scenarios, thereby ensuring efficient and accurate label adhesion on various objects 30 using point cloud data 40 and advanced processing techniques.

[0100] Those skilled in the art will understand that although embodiments including system 10 or effector 300 are applied to adhering labels to multiple irregular objects, these embodiments can also be applied to any other application, such as automated picking and placing in warehouses, airports, outdoor games, logistics, food and beverage ordering, manufacturing processes, etc., with minor modifications and without departing from the concept of this disclosure.

[0101] Those skilled in the art will understand that various changes and / or modifications can be made to the present disclosure as illustrated in the specific embodiments without departing from the spirit or scope of the present disclosure as broadly described. Therefore, the above-described embodiments are to be considered illustrative rather than restrictive in all respects.

[0102] Unless otherwise specified, any references to prior art contained herein should not be construed as an admission that the information is common general knowledge.

Claims

1. A system for placing a label on an object, characterized in that, include: An image capture module, used to capture images of a real-world environment with objects labeled; An object recognition processor is used to process a captured image to identify a point cloud associated with a region on the surface of the object; wherein the object recognition processor is used to determine a suitable label placement position based on a combination of the identified point cloud and preset parameters associated with the label; and A robot-controlled device for applying the label to the appropriate label placement location so that the label adheres to the object.

2. The system for placing a label on an object according to claim 1, characterized in that, in, The object recognition processor is used to filter data points from the recognized point cloud based on preset parameters, thereby removing data points unrelated to the object from the captured image.

3. The system for placing a label on an object according to claim 2, characterized in that, in, The object recognition processor is used to perform local filtering based on the relationship between adjacent data points.

4. The system for placing a label on an object according to claim 2, characterized in that, in, The object recognition processor is used to merge the filtered data points into one or more segments associated with a region on the surface of the object.

5. The system for placing a label on an object according to claim 4, characterized in that, in, The object recognition processor is used to compare the merged segment with one or more preset criteria to identify the placement position of the candidate label on the surface of the object.

6. The system for placing a label on an object according to claim 4, characterized in that, in, The object recognition processor is used to process larger merged segments with a higher priority than smaller merged segments.

7. The system for placing a label on an object according to claim 5, characterized in that, in, The object recognition processor is used to remove data points from surfaces that are far from the identified candidate label placement location, thereby reducing the dimensionality of the data points associated with the identified candidate label placement location.

8. The system for placing a label on an object according to claim 5, characterized in that, in, The object recognition processor is used to sort the priority of the data points among multiple identified candidate label placement positions based on one or more preset criteria.

9. The system for placing a label on an object according to claim 8, characterized in that, in, The preset criteria include sorting the data points based on the distance between the candidate label placement position and the centroid of the object.

10. The system for placing a label on an object according to claim 5, characterized in that, in, The object recognition processor is used to compare the surface area of ​​the candidate label placement location with the surface area of ​​the label.

11. A method for placing a label on an object, characterized in that, include: Capture images of a real-world environment with objects for placing labels; The captured image is processed using an object recognition process to identify point clouds associated with regions on the surface of the object; Based on the combination of the identified point cloud and preset parameters related to the label, the appropriate label placement position is determined; as well as The label is applied to the appropriate label placement location so that the label adheres to the object.

12. The method for placing a label on an object according to claim 11, characterized in that, in, The object recognition process is used to filter data points from the recognized point cloud based on preset parameters, thereby removing data points unrelated to the object from the captured image.

13. The method for placing a label on an object according to claim 12, characterized in that, in, The object identification process is used to perform local filtering based on the relationship between adjacent data points.

14. The method for placing a label on an object according to claim 12, characterized in that, in, The object identification process is used to merge the filtered data points into one or more segments that are associated with a region on the surface of the object.

15. The method for placing a label on an object according to claim 14, characterized in that, in, The object recognition process is used to compare the merged segment with one or more preset criteria to identify the placement position of the candidate label on the surface of the object.

16. The method for placing a label on an object according to claim 14, characterized in that, in, The object identification process is used to process larger merged segments with a higher priority than smaller merged segments.

17. The method for placing a label on an object according to claim 15, characterized in that, in, The object recognition process is used to remove data points from surfaces that are far from the identified candidate label placement location, thereby reducing the dimensionality of the data points associated with the identified candidate label placement location.

18. The method for placing a label on an object according to claim 15, characterized in that, in, The object recognition process is used to sort the priority of the data points among multiple identified candidate label placement positions based on one or more preset criteria.

19. The method for placing a label on an object according to claim 18, characterized in that, in, The object recognition process is used to sort the data points based on the distance between the candidate label placement position and the centroid of the object.

20. The method for placing a label on an object according to claim 15, characterized in that, in, The object recognition process is used to compare the surface area of ​​the candidate label placement location with the surface area of ​​the label.

Images