Object Orientation Estimation Using Centroid Tracking
The maximum grayscale distribution curve is evaluated in the image of the elliptical object through the centroid tracker algorithm, which solves the problem of geometric estimation in perspective distortion and noise scenes of the elliptical object in the prior art, and improves the automatic guide vehicle system Position correction accuracy and decoding rate of MaxiCode tags.
Patent Information
- Application Number
- CN201980035740.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-05-30
- Filing Date
- 2019-05-24
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2039-05-24
AI Technical Summary
Prior art In the case of perspective distortion, defocus shape and crown interruption, it is difficult to accurately estimate the geometry of elliptical objects, especially in weak edge acquisition and noise scenarios, resulting in inaccuracy and decoding rate of target position determination. Low problem.
A method is proposed to manage strong perspective distortion and noise by finding the centroid in the image of an elliptical object, using the maximum grayscale distribution curve in four directions, and by gradually adjusting the position of the centroid, combined with the centroid tracker algorithm Scene, ensuring accurate geometric estimates of elliptical objects.
It realizes more accurate geometric estimation of elliptical objects, improves the position correction accuracy of the automatic guide vehicle system and the decoding rate of the MaxiCode tag, and can handle distortion and fuzzy acquisition, suitable for real-time and time-critical applications.
Smart Images

Figure CN112166437B_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] none. Background Art
[0003] Detecting shapes such as ellipses from real images has great potential. Some examples include detecting elliptical objects from a scene for robotics applications. Ellipse (or circle) geometry estimation is used in real-time and time-critical applications such as symbol recognition such as MaxiCode for automated guided vehicle guidance or 2D barcode decoding.
[0004] Accurate and reliable shape geometry estimation is a critical factor in the performance of automated guided vehicle (AGV) systems.
[0005] See US Patent No. 6,256,560 for an example of position and orientation correction of an AGV using shape. The effects of perspective distortion combined with other geometric distortions (eg, skew due to tilt) on a target can introduce uncertainty in the orientation estimate, reducing the accuracy of the target position determination.
[0006] Many methods have been proposed to detect elliptical shapes and accurately estimate ellipse geometry. See Chapter 2: Methods for Ellipse Detection from EdgeMaps of Real Images in Prasad, DK, Leung, MK, Machine Vision Applications and Systems, 135-162 (2012). Chapter 2 provides a summary of the methods. All methods are "edge-based" and suffer from weak edge acquisition (blurring) and noise, or they are computationally intensive and not suitable for real-time and time-critical applications. In the case of high skew levels, there are inaccuracies in the axis estimation that can produce erroneous results.
[0007] Seiko Corporation's U.S. Patent No. 6,604,682 describes the method: "A first step for comparing the detected black and white pattern of the positioning pattern obtained by scanning in a first scanning direction with a specific black and white reference pattern; a second step for comparing the detected black and white pattern of the positioning pattern obtained by scanning in a second scanning direction with a specific black and white reference pattern. (See, for example, U.S. Patent No. 6,604,682, column 9, lines 18 to 23)".
[0008] What is needed is a solution that allows geometric estimation of elliptical objects even when the original target is distorted or blurred due to acquisition. Summary of the invention
[0009] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
[0010] We present a new method for finding the ellipticity of an object with a concentric crown that is able to manage strong perspective distortion, out-of-focus shapes, and crown interruptions. In particular, an evaluation of the exact center of a circular positioning pattern is performed, which leads to a better method of finding graphic distortions in images such as ellipses or circles. Starting with a preliminary center evaluation, an embodiment of the present invention begins by evaluating the maximum grayscale distribution curve in four (4) directions located at ninety (90) degrees to adjacent directions, moving from the preliminary center to the last white circular crown. The centroid is evaluated as the midpoint. The calculations at the four (4) directions are repeated, moving the axis five (5) degrees before repeating the calculation. If the centroid is correct, a confirmation is performed. An analysis of the major axis of the ellipse is performed. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The present invention is illustrated by way of example and not limitation in the accompanying figures in which like references indicate similar elements, and in which:
[0012] Figure 1 is a schematic diagram of a symbol read by an automated guided vehicle according to an embodiment of the present invention;
[0013] Figure 2 is an exemplary label including a barcode and a MaxiCode implemented according to an embodiment of the present invention;
[0014] Figure 3A is implemented according to an embodiment of the present invention, showing an exemplary elliptical object in four linear directions and four quadrants;
[0015] Figure 3B is an exemplary graph showing a sine curve implemented to illustrate an exemplary calculation of a midpoint;
[0016] Figure 4 is implemented according to an embodiment of the present invention, showing another exemplary elliptical object with a centroid;
[0017] Figure 5 is implemented in accordance with an embodiment of the present invention, showing another exemplary elliptical object in four linear directions after several iterations in which each linear direction is separated by five (5) degrees;
[0018] Figure 6is implemented in accordance with an embodiment of the present invention, showing another exemplary elliptical object in four linear directions after all iterations in which each linear direction or DIST is separated by five (5) degrees;
[0019] Figure 7 is an exemplary graph showing the effect of geometric estimation on an elliptical object implemented in accordance with an embodiment of the present invention;
[0020] Figure 8 is an exemplary process for performing geometric estimation of an elliptical object, implemented according to an embodiment of the present invention;
[0021] Fig. 9 is a schematic block diagram for determining the direction of the major axis according to an embodiment of the present invention;
[0022] Fig.10 and Fig.11 is an exemplary schematic diagram of noisy or distorted labels encountered according to an embodiment of the present invention; and
[0023] Fig.12 is a schematic diagram of an AGV on a track with several symbols for the AGV to encounter, implemented in accordance with an embodiment of the present invention. DETAILED DESCRIPTION
[0024] The subject matter of aspects of the present invention is described herein with specificity to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent.
[0025] Embodiments of the present invention may be implemented to find the ellipticity of an object having concentric crowns. In particular, the center of an elliptical shape having concentric crowns may be identified even if the shape is distorted or blurred at the origin of the shape or after the shape has been captured by an imaging device. Elliptical shapes or objects include ellipses, circles, and variations of ellipses and circles in shaped forms.
[0026] In a first aspect, a system for correcting the position of an automated guided vehicle (AGV) system moving along a path is provided, the system comprising an AGV system having a processor coupled to a memory. The AGV system comprises a scanner having a camera. The AGV system moves in a direction along the path. As the AGV system moves along the path, the scanner scans a symbol on the ground. The AGV system stops in a deviated position. Once the scanner detects a stop symbol, the AGV system stops. The camera captures the stop symbol. The stop symbol is distorted or displayed as a geometric distortion, or the stop symbol is a blurred acquisition after being captured by the camera. The processor performs geometric shape estimation on the geometric distortion or blurred acquisition to correct the deviated position of the AGV system. The processor estimates a geometric shape and determines the position of the geometric shape along an axis in an XY reference system. The AGV system moves or rotates to a correct aligned position along the path based on the position of the geometric shape.
[0027] In a second aspect, a method for correcting the position of an automated guided vehicle (AGV) system moving along a path includes operating an AGV system having at least one processor coupled to a memory. The AGV system includes a scanner having a camera. The AGV system moves along the path in a certain direction. As the AGV system moves along the path, the scanner scans a symbol on the ground. Once the scanner detects a stop symbol, the AGV system stops. The AGV system stops in a deviated position. The camera captures the stop symbol. The stop symbol is distorted or displayed as a geometric distortion, or the stop symbol is a blurred acquisition after being captured by the camera. A geometric shape estimation of the geometric distortion or blurred acquisition is performed by at least one processor to correct the deviated position of the AGV system. At least one processor determines the position of the geometric shape estimation along an axis in an XY reference system. The AGV system moves or rotates along the path to a correct correction position based on the position of the geometry along the axis.
[0028] In a third aspect, a system for correcting the position of an automated guided vehicle (AGV) system moving along a path includes an AGV system having at least one processor coupled to a memory and including a scanner having a camera. The AGV system moves in a direction along the path. As the AGV system moves along the path, the scanner scans a symbol on the ground. The AGV system moves in a deviated position relative to the path. The camera captures the symbol. The symbol is distorted or displayed as geometrically distorted, or the symbol is a blurred acquisition after being captured by the camera. At least one processor performs a geometric shape estimation of the geometrically distorted or blurred acquisition to correct the deviated position of the AGV system while the AGV system moves. At least one processor determines the position of the geometric shape estimation along an axis in an XY reference system. When the AGV system moves along the path, it moves or rotates to a correct correction position. The movement or rotation to the correct correction position is based on the position of the geometry along the axis.
[0029] In a fourth aspect, a method for evaluating distortion of an object having concentric crowns is provided, the method comprising determining a preliminary center position of an ellipse. Determining two perpendicular axes that intersect in the preliminary center position and identifying four line segments of four quadrants. In (a), for each of the four line segments, evaluating the grayscale value of the outer crown. In (b), for each line segment, calculating the center of mass based on the grayscale value. In (c), calculating the distance between each center of mass and the center. Rotating the line segment by a fixed angle. Repeating steps (a), (b) and (c). After the line segment has been rotated a full 360 degrees, analyzing the distance distribution. Selecting the maximum value as the major axis.
[0030] The invention is called centroid tracker and is based on the idea of analyzing the distribution of selected measures (distances of groups of points on the crown from the center) in an elliptical geometry to clearly find the centroid. The advantages of the centroid tracker method are its weak edge acquisition, independence of general shapes and correct management of noisy scenes. Using the centroid tracker method with a reader installed in an automated guided vehicle (such as the Matrix 120 manufactured by Datalogic SpA of Lippo diCalderaradi Reno (Bologna), Italy), a higher accuracy in object detection can be achieved. In addition, higher decoding rates can be achieved even for damaged or distorted MaxiCode tags scanned by an image reader.
[0031] In an embodiment of the present invention, an algorithm may be used to calculate the ellipticity of an elliptical object. First, find the starting point as the maximum value in the grayscale distribution curve, moving from the center in four directions ninety (90) degrees outward from the adjacent directions toward the last white circular crown, resulting in roughly four quadrants. Many methods are described in the literature to find the preliminary center position of a potential circle / ellipse. For example, in an image Figure 3A In the target presented in , the preliminary center position can be roughly estimated as the midpoint of the inner center peak of 5 consecutive signal peaks, such as Figure 3B Find (x2-x1) / 2 in curve 300A.
[0032] exist Figure 3A , an elliptical object 300 is shown with circular crowns 310, 320, and 330. Directions 340, 350, 360, and 370 are shown with a central origin 380. Figure 1 The objects found in are similar.
[0033] exist Figure 1, a symbol 100A, such as a stop sign, is shown with oval objects 110A and 110B and with a barcode 115. Symbol 100A is indicative of the type that is placed on the ground in an environment with an automated guided vehicle (AGV). The AGV moves along a path and is able to scan symbol 100A using an image reader or scanning camera located within the AGV. Symbol 100B is another version of the type that might be found on the ground to guide the AGV. However, symbol 100B shows distortions that may occur based on the symbol being moved or damaged or that may occur after being read by a reader or camera. As seen, the distortions are shown in objects 120A and 120B as well as barcode 125. In the context of this specification, the terms reader or camera may be used interchangeably. Likewise, symbols 100A and 100B may be placed throughout the ground to provide a visual representation of the symbol as described above. Fig.12 The command information for the AGV is shown, which will be discussed later.
[0034] The symbol 110A, the symbol 110B, the symbol 120A, and the symbol 120B may be located in a Figure 2 200. In the embodiment herein, symbol 220 is embedded in MaxiCode 210 in tag 200. Although symbol 220 is not placed on the ground, the concepts are applicable. An image reader or scanning camera can scan MaxiCode 210 with symbol 220. Similar to object 300, the ellipticity of symbol 220 will need to be calculated to obtain information, especially if tag 200 is distorted in some way.
[0035] Continuing with the above discussion, the following formula is used to estimate the centroid:
[0036]
[0037] Where m i = Gray value of sampling position i. x i ,y i = the horizontal and vertical coordinates of the sampling position i. The centroid (x cm ,y cm ) is evaluated from the center, where this distance is called DIST. Figure 4 , where elliptical object 400 shows DIST, which is the distance from the center of the elliptical object to the center of mass. The center of mass is calculated using the above formula for x and y. DIST is also shown by directions 340, 350, 360 and 370. The center of the elliptical object can be updated based on the evaluation of the initial four centers of mass calculated. For example, an average is performed between the midpoints and the center of each of the two axes formed by multiple line segments to obtain an updated center (also called a preliminary center or preliminary center point).
[0038] Now turn to Figure 5 , after the centroid and DIST have been calculated (where DIST is represented by four directions that are ninety (90) degrees from each adjacent direction), the centroid is moved five (5) degrees and the formula discussed above is calculated again. Figure 5 As shown, the elliptical object 500 shows the DIST in the form of the direction in which the centroid is moved five (5) degrees and the DIST is calculated again. At each step, the distance in each quadrant is accumulated based on the measurement from the center to the centroid. Within each quadrant, an assessment of the error is made. The error is defined as: 1) the percentage change in the grayscale value of the centroid from the starting value; or 2) the percentage change in the DIST from the starting distance. If the error is within a fixed range, the centroid is moved five (5) degrees and the formula is calculated again. If the error is not within a fixed range, the calculation is stopped.
[0039] like Figure 6 As shown, when the quadrants in the elliptical object 600 are completely covered, the distance distribution is analyzed to set the major axis position. The minor axis position is set ninety (90) degrees from the major axis. Although the major axis position and the minor axis position are shown in the center of the object 600, in other embodiments, if the object 600 is more elliptical or has an unusual shape, the major axis and the minor axis may be located differently.
[0040] exist Figure 7 In FIG. 7 , an exemplary graph 700 shows the centroid distance distribution. For each step, a comparison is performed such as Figure 6 The distance of the centroid from the center is evaluated at each step. Figure 6 An assessment of the distance from the center of mass is plotted in . On the x-axis, each point (step) represents a five (5) degree displacement from the starting center of mass. [72 steps x 5 degrees = 360 degrees].
[0041] Now go to Figure 8, a process for identifying an object is provided in method 800. In step 805, a maximum value is found in the outer crown in object 300. In step 810, the centroid 410 and the distance are evaluated, where the distance is the length from the center of the object 300 or 400 to the centroid 410. The centroid is calculated using the formula discussed previously. In step 815, a determination is made as to whether the measure of the distance is within the range. In step 820, if the measure is not within the range, the error is accumulated. In steps 825 and 830, if the measure is within the range, the centroid 410 is moved five (5) degrees as shown in object 500, and the calculation is restarted again. However, in step 830, if the measure is not within the range, the method 800 stops. In step 835, a determination is made as to whether the quadrant is complete. If the quadrant is not complete, then in step 810, the centroid 410 and the distance are evaluated again (but at a 5 degree offset). If the quadrant is complete, then in step 840 the quadrant data is stored and the elliptical object 600 will have a virtual distance (DIST) tag covering the quadrant as indicated, the tag typically covering the entire object 600 .
[0042] exist Fig. 9 , block diagram 900 shows first quadrant data, second quadrant data, third quadrant data, and fourth quadrant data being provided to module 910 where the data of the quadrant is analyzed. Based on the analysis, the major axis direction is determined at 920.
[0043] exist Fig.10 and Fig.11 , another embodiment of an implementation of the present invention in a use case for MaxiCode decoding is shown. Fig.10 and Fig.11 Indicates a blurred or distorted tag. As a result, correct estimation of the ellipse (locator pattern) geometry is a key factor in improving decoding performance in the presence of noisy and distorted tags. The present invention can determine the ellipse pattern, which leads to higher performance with respect to decoding the entire tag.
[0044] exist Fig.12, a schematic diagram 1200 is shown with an AGV 1205 moving around a ground surface along a path 1207. As previously described, the AGV 1205 may have a processor with a memory, but also have an image reader or scanning camera for capturing or scanning images or objects located on the ground surface. Start or stop symbols 1210A and 1210B (similar to symbols 100A and 100B) may be placed throughout the ground surface to provide instruction information for the AGV 1205. The AGV 1205 may encounter symbols 1220A to 1220E, each of which provides information for the AGV 1205 to follow, such as determining which way to go at a fork in the path. The AGV 1205 must correctly interpret the start or stop symbol and other symbols even if the symbol is distorted or damaged on the ground surface, or even if the capture of the symbol causes a blurred image. Reliable geometry estimates of the target symbols are required for the correction of the direction and position determination of the path of the AGV 1205. The processor in AGV 1205 calculates the ellipticity, which involves finding the centroid and distance to determine that the distance falls within a certain range indicating the correct calculation of the ellipse. From this, the XY axis is found, enabling AGV 1205 to correct its deflection along path 1207 and assist its performance.
[0045] Many different arrangements of the various components depicted, as well as components not shown, are possible without departing from the spirit and scope of embodiments of the invention. Embodiments of the invention have been described for illustrative, rather than limiting, purposes. Certain features and subcombinations are available and may be employed without reference to other features and subcombinations and are intended to be within the scope of the claims.
Claims
1. A system for correcting the position of an automated guided vehicle (AGV) system moving along a path, the system comprising: an AGV system having at least one processor coupled to a memory and including a scanner having a camera; The AGV system moves in one direction along the path; As the AGV system moves along the path, the scanner scans multiple symbols on the ground; The AGV system stops once the scanner detects the stop sign, wherein the AGV system stops in an offset position, wherein the camera captures the stop sign having an oval shape, wherein the stop sign is distorted or appears geometrically distorted, or the stop sign is a blurred capture after being captured by the camera; The at least one processor performs geometry estimation of geometrically distorted or blurred acquisitions to correct for a deviated position of the AGV system; The at least one processor determines a position of the geometric shape estimate along an axis in an XY reference system; and The AGV system moves or rotates along the path to the correct correction position based on the position of the geometry along the axis. Among them, performing geometric shape estimation for geometrically distorted or blurred acquisition includes: determining a preliminary center position of the ellipse; determining two perpendicular axes intersecting in the preliminary center position and four line segments identifying four quadrants; (a) for each of the four line segments, evaluating the grayscale value of the outer crown; (b) for each line segment, calculating the centroid based on the grayscale value; (c) Calculate the distance between each centroid and the center; Rotate a line segment by a fixed angle; Repeat steps (a), (b), and (c); Analyze the distance distribution after the line segment has rotated a full 360 degrees; and Select the maximum value as the major axis.
2. The system of claim 1, wherein: An off-position is a position in which the AGV system is directed in a direction different from that of the path.
3. The system of claim 1, wherein: The geometric estimation is the estimation of the major and minor axes of the elliptical object.
4. The system of claim 3, wherein: The estimation of the major axis and the minor axis of the elliptical object includes analyzing the distance distribution to set the major axis position and the minor axis position.
5. The system of claim 4, wherein: The distance distribution is an evaluation of the distance between the maximum value in the grayscale distribution curve and the center of the elliptical object at different angular positions, where the distance is calculated moving from the center of the concentric elliptical pattern to the final white circular crown.
6. The system of claim 1, wherein: The stop symbol or symbols are optical codes.
7. The system of claim 6, wherein: The optical code must be at least one of the following types: MaxiCode, AZTEC, PDF417, Datamatrix, QR, MSI Plessy, GS1 Databar, Codabar, Code 93, Interleaved 2 of 5 (ITF), Code 128, Code 39, EAN, or UPC.
8. A method for correcting the position of an automated guided vehicle (AGV) system moving along a path, the method comprising: operating an AGV system having at least one processor coupled to a memory, wherein the AGV system includes a scanner having a camera; Move the AGV system in one direction along the path; As the AGV system moves along the path, a scanner is used to scan multiple symbols on the ground; stopping the AGV system upon detection of the stop sign by the scanner, wherein stopping the AGV system includes stopping in an offset position, wherein the camera captures the stop sign having an oval shape, wherein the stop sign is distorted or appears geometrically distorted, or the stop sign is a blurred capture after being captured by the camera; performing, by the at least one processor, geometric shape estimation of the geometrically distorted or blurred acquisition to correct for a deviated position of the AGV system; determining, by the at least one processor, a position of the geometric shape estimate along an axis in an XY reference system; and Move or rotate the AGV system along the path to the correct corrective position based on the position of the geometry along the axis, Among them, performing geometric shape estimation for geometrically distorted or blurred acquisition includes: determining a preliminary center position of the ellipse; determining two perpendicular axes intersecting in the preliminary center position and four line segments identifying four quadrants; (a) for each of the four line segments, evaluating the grayscale value of the outer crown; (b) for each line segment, calculating the centroid based on the grayscale value; (c) Calculate the distance between each centroid and the center; Rotate a line segment by a fixed angle; Repeat steps (a), (b), and (c); After the line segment rotates a full 360 degrees, analyze the distance distribution; Select the maximum value as the major axis; and The correct skew position is determined based on the position of the major axis.
9. The method of claim 8, wherein: An off-position is a position in which the AGV system is directed in a direction different from that of the path.
10. The method of claim 8, wherein: The geometric estimation is the estimation of the major and minor axes of the elliptical object.
11. The method of claim 10, wherein: The estimation of the major axis and the minor axis of the elliptical object includes analyzing the distance distribution to set the major axis position and the minor axis position.
12. The method of claim 11, wherein: The distance distribution is an evaluation of the distance between the maximum value in the grayscale distribution curve and the center of the elliptical object at different angular positions, where the distance is calculated moving from the center of the concentric elliptical pattern to the final white circular crown.
13. The method of claim 8, wherein: The stop symbol or symbols are optical codes.
14. The method of claim 13, wherein: The optical code must be at least one of the following types: MaxiCode, AZTEC, PDF417, Datamatrix, QR, MSI Plessy, GS1 Databar, Codabar, Code 93, Interleaved 2 of 5 (ITF), Code 128, Code 39, EAN, or UPC.
15. A method for correcting the position of an automated guided vehicle (AGV) system moving along a path, the method comprising: When the AGV system moves along the path, a scanner of the AGV system is used to scan a plurality of symbols on the ground; Once the scanner detects the stop symbol, stopping the AGV system, wherein stopping the AGV system includes stopping in the offset position; capturing a stop sign having an oval shape using a camera of a scanner, wherein the stop sign is distorted or appears geometrically distorted, or the stop sign is a blurred capture after being captured by the camera; Performing, by at least one processor of the AGV system, geometric shape estimation of the geometrically distorted or blurred acquisition to correct the deviated position of the AGV system, including: determining a preliminary center position of the ellipse; determining two perpendicular axes intersecting in the preliminary center position and four line segments identifying four quadrants; (a) for each of the four line segments, evaluating the grayscale value of the outer crown; (b) for each line segment, calculating the centroid based on the grayscale value; (c) Calculate the distance between each centroid and the center; Rotate a line segment by a fixed angle; Repeat steps (a), (b), and (c); After the line segment rotates a full 360 degrees, analyze the distance distribution; Select the maximum value as the major axis; and The correct skew position is determined based on the position of the major axis.
16. The method of claim 15, wherein: Set the minor axis to 90 degrees with the major axis.
17. The method of claim 16, wherein: The centroid is the location on the elliptical object calculated from the set of formulas that derive the abscissa and ordinate.
18. The method of claim 17, wherein: The preliminary center position is revised based at least in part on the results of the evaluation of the initial four center of mass positions.
19. The method of claim 16, wherein: Analyzing the distance distribution includes analyzing the distance distribution to set the major axis position and the minor axis position.
20. The method of claim 19, wherein: The distance distribution is an evaluation of the distance between the maximum value in the grayscale distribution curve and the center of the elliptical object at different angular positions, where the distance is calculated moving from the center of the concentric elliptical pattern to the final white circular crown.
21. The method of claim 15, further comprising determining a preliminary center location of the ellipse in an optical reader having a processor coupled to a memory.
