Pallet positioning method and pallet positioning system

Abstract

The invention discloses a pallet positioning method and a pallet positioning system. The pallet positioning method comprises the following steps: defining a size parameter of a pallet; defining a pallet characteristic of the pallet according to the dimension parameter; obtaining a field image; defining a projection reference object in the field image; calculating a projection plane coordinate system according to the projection reference object; converting the field image into a visual angle adjusting image according to the projection plane coordinate system; obtaining a first pallet image according to the pallet characteristics in the view angle adjustment image; calculating an inclination angle of the first pallet image; rotating the first pallet image according to the inclination angle; obtaining a second pallet image; obtaining position information of the pallet according to the second pallet image; and obtaining three-dimensional information of the pallet according to the inclination angle and the position information. The invention also discloses a pallet positioning system.

Description

Technical Field

[0001] The present invention relates to a positioning method and a positioning system, and in particular to a pallet positioning method and a pallet positioning system. Background Art

[0002] The current logistics industry mostly uses pallet trucks to move goods. Even if some of them are powered, they still require human operation. The human intervention in the moving process is not only costly and unsafe, but also cannot achieve a high level of absolute accuracy and repeatability.

[0003] Therefore, some companies have introduced Automated Guided Vehicles (AGVs) to automatically move cargo pallets, but pallet positioning is mainly achieved using binocular vision, radar, or RGB-D cameras. Binocular vision is easily affected by light and is not conducive to fields without obvious features. RGB-D cameras and high-precision radars are relatively expensive.

[0004] Therefore, how to provide a pallet positioning method and a pallet positioning system that can automatically position and has low cost is one of the problems that need to be solved urgently. Summary of the invention

[0005] The invention provides a pallet positioning method and a pallet positioning system which can automatically position and have low cost.

[0006] The present invention provides a pallet positioning method, comprising: defining a size parameter of a pallet; defining a pallet feature of the pallet according to the size parameter; obtaining a live image; defining a projection reference object in the live image; calculating a projection plane coordinate system according to the projection reference object; converting the live image into a perspective adjustment image according to the projection plane coordinate system; obtaining a first pallet image in the perspective adjustment image according to the pallet feature; calculating an inclination angle of the first pallet image; rotating the first pallet image according to the inclination angle; obtaining a second pallet image; obtaining position information of the pallet according to the second pallet image; and obtaining three-dimensional information of the pallet according to the inclination angle and the position information.

[0007] In some embodiments, rotating the first pallet image according to the tilt angle includes: comparing the rotated first pallet image with the pallet feature.

[0008] In some embodiments, obtaining the second pallet image includes: determining whether the second pallet image corresponds to the pallet based on the size parameter and the pallet feature; and confirming to obtain the second pallet image if it is determined that the second pallet image corresponds to the pallet.

[0009] In some embodiments, obtaining the second pallet image further includes: comparing an edge distribution of the rotated first pallet image with an edge distribution of the pallet feature; and aligning the edge distribution of the rotated first pallet image with the edge distribution of the pallet feature.

[0010] In some embodiments, the position information is two-dimensional information.

[0011] In some embodiments, obtaining the three-dimensional information of the pallet according to the tilt angle and the position information includes: obtaining an inverse transformation matrix according to the tilt angle and the position information; and converting the position information into the three-dimensional information using the inverse transformation matrix.

[0012] The present invention provides another pallet positioning method, which includes: defining a size parameter of a pallet; defining a pallet feature of the pallet according to the size parameter; obtaining a live image; defining a projection reference object in the live image; calculating a projection plane coordinate system according to the projection reference object; converting the live image into a perspective adjustment image according to the projection plane coordinate system; performing a first positioning procedure to obtain a first pallet image in the perspective adjustment image; calculating a tilt angle of the first pallet image; rotating the first pallet image according to the tilt angle; obtaining a second pallet image; obtaining position information of the pallet according to the second pallet image; and obtaining three-dimensional information of the pallet according to the tilt angle and the position information.

[0013] In some embodiments, the pallet positioning method further includes: performing a second positioning procedure on the rotated first pallet image to obtain the second pallet image.

[0014] In some embodiments, the second positioning process includes comparing the rotated first pallet image with the pallet feature.

[0015] In some embodiments, obtaining the second pallet image includes: determining whether the second pallet image corresponds to the pallet based on the size parameter and the pallet feature; and confirming to obtain the second pallet image if it is determined that the second pallet image corresponds to the pallet.

[0016] In some embodiments, obtaining the second pallet image further includes: comparing an edge distribution of the rotated first pallet shadow with an edge distribution of the pallet feature; and aligning the edge distribution of the rotated first pallet shadow with the edge distribution of the pallet feature.

[0017] In some embodiments, the first positioning procedure includes: positioning the first pallet image in the perspective adjustment image according to the pallet feature.

[0018] The present invention provides another pallet positioning system, comprising: a photographing module, which obtains a scene image; a scene construction module, which is electrically connected to the photographing module, receives the scene image, and comprises: an input unit, which receives a size parameter of a pallet; and a construction unit, which is electrically connected to the input unit, defines a pallet feature of the pallet according to the size parameter, defines a projection reference object on the scene image, and calculates a projection plane coordinate system according to the projection reference object; and a positioning operation module, which is electrically connected to the scene construction module, and comprises: a positioning unit, which converts the scene image into a perspective adjustment image according to the projection plane coordinate system, obtains a first pallet image in the perspective adjustment image according to the pallet feature, calculates a tilt angle of the first pallet image, rotates the first pallet image according to the tilt angle to obtain a second pallet image, and obtains position information of the pallet according to the second pallet image; and a conversion unit, which obtains three-dimensional information of the pallet according to the tilt angle and the position information.

[0019] In some embodiments, the positioning unit compares the rotated first pallet image with the pallet feature to obtain the second pallet image.

[0020] In some embodiments, the positioning unit determines whether the second pallet image corresponds to the pallet based on the size parameter and the pallet feature, and if it is determined that the second pallet image corresponds to the pallet, the second pallet image is confirmed to be obtained.

[0021] In some embodiments, the positioning unit compares an edge distribution of the second pallet shadow with an edge distribution of the pallet feature, and aligns the edge distribution of the second pallet shadow with the edge distribution of the pallet feature to obtain the second pallet image.

[0022] In some embodiments, the conversion unit obtains an inverse conversion matrix according to the tilt angle and the position information, and uses the inverse conversion matrix to convert the position information into the three-dimensional information.

[0023] In summary, the pallet positioning method and pallet positioning system of the present invention obtain a scene image, calculate the projection plane coordinate system in the scene image, and then identify the position and posture of the pallet based on the size parameters and pallet features of the defined pallet in the scene image based on the projection plane coordinate system. Therefore, the pallet positioning method and pallet positioning system of the present invention can automatically position the pallet without human vision. Moreover, the pallet positioning method and pallet positioning system of the present invention can complete the positioning of the pallet only through a photographic device that can obtain a point cloud map or a depth map, which is not only low in cost, but also the equipment can be flexibly set up. Furthermore, the pallet positioning method and pallet positioning system of the present invention use a two-dimensional method to position the pallet, and estimate the tilt angle for each pallet target, and can support multi-pallet positioning, and obtain the position and posture of all pallets. Moreover, the pallet positioning method and pallet positioning system of the present invention can perform multiple positioning on the pallet image to obtain more accurate three-dimensional information of the pallet.

[0024] Furthermore, the pallet positioning method and pallet positioning system of the present invention can accurately obtain the position of the pallet without other additional information, such as two-dimensional images, panoramic maps, etc. In addition, by calibrating the projection reference objects related to the position of the pallet, such as calibration plates, floors, etc., and estimating the coordinate system of the projection reference objects during training, the photography module of the point cloud map or depth map can be set up more flexibly, without being restricted by the pitch and skew angles between the equipment and the pallet. In addition, based on the point cloud projection without pitch and skew angles, pallets at different distances have a uniform size and no rotation angle, which minimizes the complexity of the scene, and any positioning algorithm can be applied for positioning, and the positioning tool can be flexibly selected according to the characteristics of different photography modules. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flow chart of a pallet positioning method according to a first embodiment of the present invention;

[0026] Figure 2 is a schematic diagram of a pallet template of this embodiment;

[0027] Figure 3 is a schematic diagram of a scene image of this embodiment;

[0028] Figure 4A and Figure 4B is a schematic diagram of the process of obtaining the projection plane coordinate system of this embodiment;

[0029] Figure 5A and Figure 5B is a schematic diagram of a process of converting a live image into a perspective-adjusted image according to the present embodiment;

[0030] Figure 6 is a schematic diagram of a first pallet image of this embodiment;

[0031] Figure 7 is a schematic diagram of a second pallet image of this embodiment;

[0032] Figure 8 A schematic diagram of all the pallets in the image of the recognition viewing angle adjustment of the present embodiment;

[0033] Fig. 9 is a schematic diagram of a pallet after identification in this embodiment;

[0034] Fig.10 and Fig.11 is a flow chart of a pallet positioning method according to a second embodiment of the present invention;

[0035] Fig.12 is a flow chart of a pallet positioning method according to a third embodiment of the present invention;

[0036] Fig.13 and Fig.14 is a flow chart of a pallet positioning method according to a fourth embodiment of the present invention;

[0037] Fig.15 Schematic diagram of a pallet positioning system of the present invention.

[0038] Description of Reference Numerals

[0039] 1: Pallet

[0040] 2: Live images

[0041] 21: Projection reference

[0042] 3: Pallet positioning system

[0043] 31: Photography module

[0044] 32: On-site construction module

[0045] 321: Input unit

[0046] 322: Construction Unit

[0047] 33: Positioning calculation module

[0048] 331: Positioning unit

[0049] 332:Conversion unit

[0050] 4: Viewing angle to adjust the image

[0051] 5: First stack image

[0052] 51: inclined part

[0053] 6: Second stack image

[0054] 9: Pallet template

[0055] C: Projection plane coordinate system

[0056] C1: 3D coordinate system

[0057] P1: optical axis projection vector

[0058] P2: Normal vector

[0059] P3, P5: outer product vector

[0060] P4: Reverse vector

[0061] W, BW, PW: Width

[0062] H, PH: height

[0063] S01~S13, S091, S101~S104, S21~S36: Steps DETAILED DESCRIPTION

[0064] As used herein, terms such as "first", "second", etc. describe various elements, components, regions, layers, and / or parts, which should not be limited by these terms. These terms can only be used to distinguish one element, component, region, layer, or part from another. Unless the context clearly indicates, the terms such as "first", "second" used in this article do not imply a sequence or order.

[0065] Figure 1 FIG. 1 is a flow chart of a pallet positioning method according to a first embodiment of the present invention. Figure 1 As shown, the pallet positioning method of this embodiment includes steps S01 to S13. Step S01 is to define the size parameters of the pallet. Step S02 is to define the pallet features of the pallet according to the size parameters. Step S03 is to obtain a scene image. Step S04 is to define a projection reference object in the scene image. Step S05 is to calculate a projection plane coordinate system according to the projection reference object. Step S06 is to convert the scene image into a perspective adjustment image according to the projection plane coordinate system. Step S07 is to obtain a first pallet image in the perspective adjustment image according to the pallet features. Step S08 is to calculate the inclination angle of the first pallet image. Step S09 is to rotate the first pallet image according to the inclination angle. Step S10 is to obtain a second pallet image. Step S11 is to obtain the position information of the pallet according to the second pallet image. Step S12 is to obtain the three-dimensional information of the pallet according to the inclination angle and the position information. Step S13 is to determine whether the three-dimensional information of all pallets has been obtained.

[0066] Figure 2 This is a schematic diagram of the pallet template of this embodiment. Figure 1 and Figure 2As shown, in step S01, the size parameters of the pallet are defined. In step S02, the pallet features of the pallet are defined according to the size parameters. First, the user can input the size parameters of the pallet to be positioned, or directly read the size parameters of the pallet from the database to construct a pallet template with preset pallet features. It should be noted that the size parameters of the pallet in this embodiment only refer to the size parameters of one side of the pallet, such as the side used by the unmanned transport vehicle to identify and insert and move.

[0067] In some embodiments, the user can input the width W and height H of the pallet, the width BW of the spacing area, the width PW and the height PH of the mouth area, etc., and the above parameters are not limiting. Through these parameters, the user can define the pallet features of the pallet, such as the pallet has two mouth areas, etc., to construct the pallet template 9, but the pallet features of the pallet are not limiting. Of course, according to different requirements, the user can flexibly decide which features need to be constructed.

[0068] Figure 3 is a schematic diagram of the on-site image of this embodiment. Figure 1 and Figure 3 As shown, in step S03, a scene image 2 is obtained. In step S04, a projection reference object 21 is defined in the scene image 2. Therefore, the user can use a photography module with an optical axis (for example, a time of flight (ToF) camera, which is not limiting) to be set up at any angle toward the pallet to obtain the scene image 2. The scene image 2 can be, for example, a point cloud image or a depth map, which is not limiting. Then, the user can define the projection reference object 21 in the scene image 2. In other words, the user can calibrate the projection reference object 21 related to the location where the pallet is placed, such as the ground.

[0069] Figure 4A and Figure 4B This is a schematic diagram of the process of obtaining the projection plane coordinate system of this embodiment. Figure 1 and Figure 4A As shown, in step S05, the projection plane coordinate system C is calculated according to the projection reference object 21. After the projection reference object 21 (such as the ground) is defined in the scene image 2, the optical axis 311 of the camera module 31 can be projected to the plane of the projection reference object 21. Then, the three-dimensional coordinate system C1 of the projection reference object 21 can be established through the optical axis projection vector P1, the normal vector P2 of the projection reference object 21 and the outer product vector P3 of the outer product of the two. It is worth mentioning that the outer product vector P3 is the intersection of the optical axis projection plane and the plane of the projection reference object 21. Furthermore, as Figure 4B As shown, the outer product vector P5 can be obtained by taking the outer product of the reverse vector P4 and the outer product vector P3 of the optical axis projection vector P1. The projection plane coordinate system C can be formed by the reverse vector P4, the outer product vector P3 and the outer product vector P5.

[0070] Figure 5A and Figure 5B This is a schematic diagram of the process of converting a live image into a perspective-adjusted image in this embodiment. Figure 1 , Figure 5A and Figure 5B As shown, in step S06, the scene image is converted into the perspective adjustment image 4 according to the projection plane coordinate system. Figure 4B After that, the three-dimensional scene image 2 (as shown Figure 3 ) is projected onto the projection plane coordinate system C. However, the image projected onto the projection plane coordinate system C by the scene image 2 may have a pitch angle or a tilt angle (such as Figure 5A Therefore, according to the projection plane coordinate system C and the three-dimensional coordinate system C1, the pitch angle and the tilt angle can be adjusted to convert the scene image 2 into a perspective-adjusted image 4 without pitch and tilt (such as Figure 5B That is, for example, the outer product vector P5 of the projection plane coordinate system C is parallel to the normal vector P2 of the three-dimensional coordinate system C1 (e.g. Figure 4B As shown), to obtain a perspective-adjusted image 4 without pitch and tilt.

[0071] Figure 6 is a schematic diagram of the first pallet image of this embodiment. Figure 1 , Figure 5A to Figure 5B and Figure 6 As shown, in step S07, in the perspective adjustment image 4, a first pallet image 5 is obtained according to the pallet features. Figure 2 As shown in FIG. 4 , the first stack image 5 can be identified and obtained from the perspective adjustment image 4 (which can also be used as a first positioning procedure). Since the perspective adjustment image 4 is a planar image after the three-dimensional scene image is projected onto the projection plane, the first stack image 5 may include a tilt angle in the horizontal direction (e.g., toward the left or right) (e.g., shown as a tilted portion 51).

[0072] Figure 7 is a schematic diagram of the second pallet image of this embodiment. Figure 1 , Figure 6 and Figure 7 As shown, in step S08, the tilt angle of the first pallet image 5 is calculated. In step S09, the first pallet image 5 is rotated according to the tilt angle. In step S10, the second pallet image 6 is obtained. In step S11, the position information of the pallet is obtained according to the second pallet image 6. In some embodiments, the position information is, for example, two-dimensional information.

[0073] In order to calculate the horizontal tilt angle of the first pallet image 5, an algorithm with a larger error tolerance can be used. For example, a sample comparison method can be used to calculate the tilt angle of the first pallet image 5, but this is not limiting, and other positioning algorithms can be used as required.

[0074] Taking the sample comparison method as an example, the algorithm uses one or more sample images artificially created or obtained from images in the training phase. Then, in the execution phase, the sample image is projected to each position on the first stack image 5, and the similarity between each position of the first stack image 5 and each position of the sample image is compared. The calculation method that can be used in the sample comparison method is, for example, a normalized cross-correlation (NCC) algorithm (such as the following formula), which is non-limiting.

[0075]

[0076] Among them, f i is the pixel value of the first stack image, μ f is the average value of the first stack image in the block, T i is the pixel value of the sample image, μ T is the average value of the sample images.

[0077] After calculation, if the similarity is high, it means that there is a high probability that the position range is a pallet, and the image range can be retained for verification in subsequent procedures. Afterwards, the tilt angle of the pixel points on the first pallet image 5 can be estimated at each position using statistics, fitting or gradient algorithms. Finally, the first pallet image 5 can be rotated according to the tilt angle to straighten the first pallet image 5 to obtain the second pallet image 6. Since the second pallet image 6 is obtained by forward information such as projection, positioning, and tilt angle, the position information of the pallet can be obtained by reversing these steps. It is worth mentioning that Figure 7 The second pallet image 6 in the image is mainly used to calculate the two-dimensional information such as the tilt angle of the pallet, thereby not only obtaining the tilt angle of each pallet, but also the image of the positive angle is also conducive to subsequent more precise positioning or judgment, but the second pallet image 6 does not necessarily need to be displayed. That is, the user can only observe the second pallet image 6. Figure 6 The first stack image 5.

[0078] Figure 8 Schematic diagram of all pallets in the recognition viewing angle adjustment image of this embodiment. Specifically, steps S07 to S11 can be repeated until the recognition viewing angle adjustment image 4 (such as Figure 5A to Figure 5B ) in the first pallet images 5 (which is not limiting, and the second pallet images may also be displayed). It should be noted that the number of pallets is not limiting.

[0079] Fig. 9 This is a schematic diagram of the identified pallet of this embodiment. Figure 1 and Fig. 9 As shown, in step S12, the three-dimensional information of the pallet is obtained according to the tilt angle and the position information. In some embodiments, obtaining the three-dimensional information of the pallet according to the tilt angle and the position information further includes: obtaining an inverse transformation matrix according to the tilt angle and the position information; and using the inverse transformation matrix to convert the position information into three-dimensional information. Finally, in step S13, if it is determined that the three-dimensional information of all pallets has been obtained, the process can be terminated. If it is determined that the three-dimensional information of the pallet has not been obtained, it can return to step S07 for further determination.

[0080] As mentioned above, the second pallet image 6 (eg Figure 7 The position of the second stack 1 is obtained by the forward position information such as projection, positioning and tilt angle, and the three-dimensional information of each stack 1 can be obtained by inversely deducing these position information. In other words, when calculating the second stack image with the projection and tilt angle positive, a 4x4 transformation matrix before and after the transformation will be obtained. As long as the inverse transformation matrix is ​​calculated, the coordinate points on the second stack image can be deduced to the coordinate points on the original coordinate system (such as the coordinate system on the point cloud image).

[0081] In addition, during the positioning process of each pallet 1, the regional coordinates (i.e., the coordinate system of each pallet) are used to replace the global coordinates. Therefore, the final positioning coordinates need to be converted into global coordinates by adding an offset. The following is an explanation of the coordinate conversion matrix. First, the reverse vector P4 (such as Figure 4B As shown), the normal vector P2 of the projection reference object 21 (as shown Figure 4B As shown), and the vector of the outer product of the two vectors, these three vectors can form a plane coordinate system, which is represented by the conversion matrix R between the ToF point cloud coordinate system and the plane coordinate system:

[0082]

[0083] In addition to the rotation relationship, projection also has translation and scaling, which is mainly to convert the millimeter (mm) unit of the point cloud image into the image pixel (pixel) unit. The x and y of the projected image are obtained by subtracting the minimum value (translation) from the point cloud coordinates and then dividing by the resolution (scaling). The z of the image also needs to add the distance between the plane and the camera module. So that z is represented on the plane. The calculation method is as follows.

[0084]

[0085] Combining the above relationships, the conversion matrix between pixels and millimeters can be listed: mm T pixel, then transform the matrix mm T pixel The inverse transformation matrix can be obtained by multiplying the inverse matrix of the transformation matrix R of the rotation relationship on the left. The calculation method is as follows.

[0086]

[0087] Thus, we can Figure 8 The two-dimensional image is transformed according to the tilt angle and position information. Using the inverse transformation matrix, the position information is converted into three-dimensional information to obtain Fig. 9 The three-dimensional information of the pallet 1 is obtained. That is, the information such as the position and posture of the pallet 1 in the three-dimensional space is obtained.

[0088] As mentioned above, the pallet positioning method of the present embodiment obtains the scene image, calculates the projection plane coordinate system in the scene image, and then identifies the position and posture of the pallet based on the size parameters and pallet features of the pallet defined in the scene image based on the projection plane coordinate system. Therefore, the pallet positioning method of the present embodiment can automatically locate the pallet without the need for human vision. Moreover, the pallet positioning method of the present embodiment can complete the positioning of the pallet only through a photographic device that can obtain a point cloud map or a depth map, which is not only low in cost, but also the equipment can be flexibly set up. Furthermore, the pallet positioning method of the present embodiment uses a two-dimensional method to position the pallet, and estimates the tilt angle for each pallet target, which can support multi-pallet positioning and obtain the position and posture of all pallets.

[0089] Furthermore, the pallet positioning method of the present embodiment does not require other additional information, such as two-dimensional images, panoramic maps, etc., to accurately obtain the position of the pallet. Furthermore, by calibrating the projection reference objects related to the position of the pallet, such as calibration plates, floors, etc., and estimating the coordinate system of the projection reference objects during training, the photography module of the point cloud map or depth map can be set up more flexibly and not restricted by the pitch and skew angles between the equipment and the pallet. In addition, based on the point cloud projection without pitch and skew angles, pallets at different distances have a uniform size and no rotation angle, which minimizes the complexity of the scene, and any positioning algorithm can be applied for positioning, and the positioning tool can be flexibly selected according to the characteristics of different photography modules.

[0090] Fig.10 and Fig.11 FIG. 1 is a flow chart of a pallet positioning method according to a second embodiment of the present invention. Figure 1 , Fig.10 and Fig.11As shown, the difference between the pallet positioning method of this embodiment and the pallet positioning method of the first embodiment is that it also includes step S091 and steps S101 to S104. Step S091 is to compare the rotated first pallet image with the pallet feature (which can also be used as a second positioning procedure). Step S101 is to determine whether the second pallet image corresponds to the pallet based on the size parameters and the pallet feature. If it is determined that the second pallet image corresponds to the pallet, step S102 is to confirm that the second pallet image is obtained. Step S103 is to compare the edge distribution of the second pallet image with the edge distribution of the pallet feature. Step S104 is to align the edge distribution of the second pallet image with the edge distribution of the pallet feature.

[0091] Specifically, after the first pallet image is rotated according to the tilt angle in step S09, the pallet can be further positioned (step S091). Since the first pallet image has been rotated to the right angle, in order to make the first pallet image clearer and remove the original uncertainty, a higher similarity standard can be used to judge whether the image range is a pallet for re-positioning.

[0092] For example, a more sophisticated positioning algorithm such as edge comparison can be used to compare the preset pallet features with the first pallet image after the normalization to see if they have similar edge distributions, and align these edge points, but this is not intended to limit the present invention. Since edge distribution is high-frequency information, it has a higher degree of precision than sample comparison.

[0093] Furthermore, after obtaining the second pallet image in step S10, the pallet features of the pallet template (ie, Figure 2 The second pallet image is then compared with the size parameters, edges, connected areas, etc. shown in the figure to see if the size, features, etc. of each area of ​​the second pallet image are consistent with the set pallet features (step S101). If it is determined that the second pallet image corresponds to the pallet, the second pallet image is obtained to perform subsequent procedures, such as comparing and aligning the edge distribution of the second pallet image and the pallet features (steps S102 to S104), thereby effectively eliminating judgment errors (i.e., false positives). On the other hand, if it is determined that the second pallet image does not correspond to the pallet, according to different requirements, the process may return to step S07 to re-acquire the first pallet image, or may return to step S09 to re-rotate the first pallet image to obtain the second pallet image, or other different procedures are not limited here.

[0094] As described above, the pallet positioning method of this embodiment can perform multiple positioning operations on the pallet image, and can apply any positioning algorithm to perform positioning, so as to obtain higher-precision three-dimensional information of the pallet.

[0095] It should be noted that step S091 and steps S101 to S104 are not necessarily all performed. In other words, only step S091 or only steps S101 to S104 may be performed.

[0096] Fig.12 FIG. 1 is a flow chart of a pallet positioning method according to a third embodiment of the present invention. Fig.12 As shown, the pallet positioning method of this embodiment includes steps S21 to S33. Step S21 is to define the size parameters of the pallet. Step S22 is to define the pallet features of the pallet according to the size parameters. Step S23 is to obtain a live image. Step S24 is to define a projection reference object in the live image. Step S25 is to calculate a projection plane coordinate system according to the projection reference object. Step S26 is to convert the live image into a perspective adjustment image according to the projection plane coordinate system. Step S27 is to perform a first positioning procedure and obtain a first pallet image in the perspective adjustment image. Step S28 is to calculate the inclination angle of the first pallet image. Step S29 is to rotate the first pallet image according to the inclination angle. Step S30 is to obtain a second pallet image. Step S31 is to obtain the position information of the pallet according to the second pallet image. Step S32 is to obtain the three-dimensional information of the pallet according to the inclination angle and the position information. Step S33 is to determine whether the three-dimensional information of all pallets has been obtained.

[0097] The pallet positioning method of this embodiment is similar to the pallet positioning method of the first embodiment. The first positioning procedure is included in the perspective adjustment image, and the first pallet image is positioned according to the pallet features. As mentioned above, the specific content has been detailed in the above embodiment and will not be repeated here.

[0098] Fig.13 and Fig.14 FIG. 4 is a flow chart of a pallet positioning method according to a fourth embodiment of the present invention. Figure 12 to Figure 14 As shown, the difference between the pallet positioning method of this embodiment and the pallet positioning method of the third embodiment is that it further includes steps S34, S35 and S36. Step S34 is to perform a second positioning procedure on the rotated first pallet image to obtain a second pallet image. Step S35 is to determine whether the second pallet image corresponds to the pallet based on the size parameter and the pallet characteristics. If it is determined that the second pallet image corresponds to the pallet, step S36 is to confirm that the second pallet image is obtained.

[0099] The pallet positioning method of this embodiment is similar to the pallet positioning method of the second embodiment. The second positioning procedure includes comparing the rotated first pallet image with the pallet features. As mentioned above, the specific content has been detailed in the above embodiment and will not be repeated here.

[0100] It should be noted that step S34, step S35 and step S36 are not necessarily all performed. In other words, only step S34 may be performed, or only steps S35 and S36 may be performed.

[0101] Fig.15Schematic diagram of the pallet positioning system of the present invention. The pallet positioning system 3 includes a photographing module 31, a scene construction module 32 and a positioning calculation module 33. The photographing module 31 is, for example, a photographing device with an optical axis, such as a time-of-flight rangefinder camera, but this is not limiting. The photographing module 31 can obtain the scene image 2 (such as Figure 3 ). The on-site construction module 32 is electrically connected to the camera module 31 and receives the on-site image 2. The on-site construction module 32 may include an input unit 321 and a construction unit 322. The input unit 321 may receive the size parameters of the pallet, which may be, for example, an input interface for a user to input the size parameters of the pallet, or a processing unit for reading the size parameters of the input pallet from a storage. The construction unit 322 is electrically connected to the input unit 321. The construction unit 322 defines the pallet features of the pallet according to the size parameters, defines a projection reference object in the on-site image, and calculates the projection plane coordinate system according to the projection reference object.

[0102] The positioning operation module 33 is electrically connected to the scene construction module 32. The positioning operation module 33 may include a positioning unit 331 and a conversion unit 332. The positioning unit 331 converts the scene image into a perspective adjustment image according to the projection plane coordinate system, obtains a first pallet image in the perspective adjustment image according to the pallet feature, calculates the tilt angle of the first pallet image, rotates the first pallet image according to the tilt angle to obtain a second pallet image, and obtains the position information of the pallet according to the second pallet image. The conversion unit 332 obtains the three-dimensional information of the pallet according to the tilt angle and the position information.

[0103] It is worth mentioning that the on-site construction module 32 and the positioning calculation module 33 can be executed by the same or different processors, for example.

[0104] Furthermore, the pallet positioning system 3 of the present invention can execute the pallet positioning methods of the first to fourth embodiments described above, which will not be described in detail herein. In addition, the positioning unit 331 can execute, for example, a first positioning program, a second positioning program, and other programs for confirming the pallet image (such as Fig.10 Steps S091 to S093, step S101 and step S102 in, or Fig.11 Step S27 and Fig.12 The conversion unit 332 may, for example, execute a procedure such as obtaining an inverse conversion matrix according to the tilt angle and the position information, and converting the position information into three-dimensional information using the inverse conversion matrix.

[0105] In summary, the pallet positioning method and pallet positioning system of the present invention obtain a scene image, calculate the projection plane coordinate system in the scene image, and then identify the position and posture of the pallet based on the size parameters and pallet features of the defined pallet in the scene image based on the projection plane coordinate system. Therefore, the pallet positioning method and pallet positioning system of the present invention can automatically position the pallet without human vision. Moreover, the pallet positioning method and pallet positioning system of the present invention can complete the positioning of the pallet only through a photographic device that can obtain a point cloud map or a depth map, which is not only low in cost, but also the equipment can be flexibly set up. Furthermore, the pallet positioning method and pallet positioning system of the present invention use a two-dimensional method to position the pallet, and estimate the tilt angle for each pallet target, and can support multi-pallet positioning, and obtain the position and posture of all pallets. Moreover, the pallet positioning method and pallet positioning system of the present invention can perform multiple positioning on the pallet image to obtain more accurate three-dimensional information of the pallet.

[0106] Furthermore, the pallet positioning method and pallet positioning system of the present invention can accurately obtain the position of the pallet without other additional information, such as two-dimensional images, panoramic maps, etc. In addition, by calibrating the projection reference objects related to the position of the pallet, such as calibration plates, floors, etc., and estimating the coordinate system of the projection reference objects during training, the photography module of the point cloud map or depth map can be set up more flexibly, without being restricted by the pitch and skew angles between the equipment and the pallet. In addition, based on the point cloud projection without pitch and skew angles, pallets at different distances have a uniform size and no rotation angle, which minimizes the complexity of the scene, and any positioning algorithm can be applied for positioning, and the positioning tool can be flexibly selected according to the characteristics of different photography modules.

[0107] The components of several embodiments are summarized above so that the technical personnel in the technical field to which the present invention belongs can better understand the concept of the embodiments of the present invention. The technical personnel in the technical field to which the present invention belongs should understand that the embodiments of the present invention can be used as a basis to design or modify other processes and structures to achieve the same purpose and / or achieve the same benefits as the embodiments introduced herein. The technical personnel in the technical field to which the present invention belongs should also understand that these equivalent structures do not deviate from the spirit and scope of the present invention, and various changes, substitutions and other options can be made here without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be defined as defined by the attached claims.

Claims

1. A pallet positioning method, comprising: Define the size parameters of the pallet; Defining pallet characteristics of the pallet according to the size parameters; Obtaining on-site images; Defining a projection reference object on the scene image; Calculate the projection plane coordinate system according to the projection reference object; According to the projection plane coordinate system, converting the scene image into a perspective-adjusted image; In the view adjustment image, obtaining a first pallet image according to the pallet feature; Calculating the tilt angle of the first stack image; Rotating the first stack image according to the tilt angle; Get the second stack image; obtaining the position information of the pallet according to the second pallet image; and The three-dimensional information of the pallet is obtained according to the tilt angle and the position information.

2. The pallet positioning method according to claim 1, wherein rotating the first pallet image according to the tilt angle comprises: The rotated first pallet image is compared with the pallet feature.

3. The pallet positioning method according to claim 1, wherein obtaining the second pallet image comprises: determining whether the second pallet image corresponds to the pallet based on the size parameter and the pallet feature; and If it is determined that the second pallet image corresponds to the pallet, it is confirmed that the second pallet image is obtained.

4. The pallet positioning method according to claim 3, wherein obtaining the second pallet image further comprises: comparing the edge distribution of the second pallet image with the edge distribution of the pallet feature; and The edge distribution of the second pallet image is aligned with the edge distribution of the pallet feature.

5. The pallet positioning method according to claim 1, wherein the position information is two-dimensional information.

6. The pallet positioning method according to claim 1, wherein obtaining the three-dimensional information of the pallet according to the tilt angle and the position information comprises: Obtaining an inverse transformation matrix according to the tilt angle and the position information; and The position information is converted into the three-dimensional information using the inverse conversion matrix.

7. A pallet positioning method, comprising: Define the size parameters of the pallet; Defining pallet characteristics of the pallet according to the size parameters; Obtaining on-site images; Defining a projection reference object on the scene image; Calculate the projection plane coordinate system according to the projection reference object; According to the projection plane coordinate system, converting the scene image into a perspective-adjusted image; Performing a first positioning procedure to obtain a first pallet image in the viewing angle adjustment image; Calculating the tilt angle of the first stack image; Rotating the first stack image according to the tilt angle; Get the second stack image; obtaining the position information of the pallet according to the second pallet image; and The three-dimensional information of the pallet is obtained according to the tilt angle and the position information.

8. The pallet positioning method according to claim 7, further comprising: A second positioning procedure is performed on the rotated first pallet image to obtain the second pallet image.

9. The pallet positioning method according to claim 8, wherein the second positioning procedure comprises: The rotated first pallet image is compared with the pallet feature.

10. The pallet positioning method according to claim 7, wherein obtaining the second pallet image further comprises: determining whether the second pallet image corresponds to the pallet based on the size parameter and the pallet feature; and If it is determined that the second pallet image corresponds to the pallet, it is confirmed that the second pallet image is obtained.

11. The pallet positioning method according to claim 10, wherein obtaining the second pallet image further comprises: comparing the edge distribution of the first rotated pallet shadow with the edge distribution of the pallet feature; and The edge distribution of the rotated first pallet shadow is aligned with the edge distribution of the pallet feature.

12. The pallet positioning method according to claim 7, wherein the first positioning procedure comprises: In the view adjustment image, the first pallet image is positioned according to the pallet feature.

13. The pallet positioning method according to claim 7, wherein the position information is two-dimensional information.

14. The pallet positioning method according to claim 7, wherein obtaining the three-dimensional information of the pallet according to the tilt angle and the position information comprises: Obtaining an inverse transformation matrix according to the tilt angle and the position information; and The position information is converted into the three-dimensional information using the inverse conversion matrix.

15. A pallet positioning system, comprising: Photography module, to obtain on-site images; The scene construction module is electrically connected to the photography module, receives the scene image, and includes: An input unit for receiving dimensional parameters of the pallet; and A construction unit, electrically connected to the input unit, defines the pallet characteristics of the pallet according to the size parameters, defines a projection reference object on the scene image, and calculates a projection plane coordinate system according to the projection reference object; and The positioning operation module is electrically connected to the on-site construction module and includes: a positioning unit, which converts the scene image into a perspective adjustment image according to the projection plane coordinate system, obtains a first pallet image in the perspective adjustment image according to the pallet feature, calculates a tilt angle of the first pallet image, rotates the first pallet image according to the tilt angle to obtain a second pallet image, and obtains position information of the pallet according to the second pallet image; and The conversion unit obtains the three-dimensional information of the pallet according to the tilt angle and the position information. 16 . The pallet positioning system according to claim 15 , wherein the positioning unit compares the rotated first pallet image with the pallet feature to obtain the second pallet image.

17. The pallet positioning system according to claim 15, wherein the positioning unit determines whether the second pallet image corresponds to the pallet based on the size parameter and the pallet feature, and if it is determined that the second pallet image corresponds to the pallet, confirms that the second pallet image is obtained.

18. The pallet positioning system according to claim 17, wherein the positioning unit compares the edge distribution of the second pallet shadow with the edge distribution of the pallet feature, and aligns the edge distribution of the second pallet shadow with the edge distribution of the pallet feature to obtain the second pallet image.

19. The pallet positioning system according to claim 15, wherein the position information is two-dimensional information.

20. The pallet positioning system according to claim 15, wherein the conversion unit obtains an inverse conversion matrix according to the tilt angle and the position information, and converts the position information into the three-dimensional information using the inverse conversion matrix.

Images