Machine vision markers, systems and methods for identifying and determining the pose of a target object using machine vision markers, and methods for manufacturing machine vision markers

Abstract

A machine vision marker, and systems and methods for using and manufacturing the machine vision marker, are provided, the marker including a first black target element comprising a black square, a square coding stripe comprised of a plurality of square coding elements extending around and surrounding a perimeter of the black square and collectively coding a binary data bit sequence encoding identification information for a target object, each one of the plurality of square coding elements being either a black square or a white square representing a data bit of 0 or 1 in the binary data bit sequence, a white target element comprising a white square ring extending around and surrounding the perimeter of the square coding stripe, a second black target element comprising a black square ring extending around a perimeter of the white square ring, and an out-of-plane white dot post disposed above or below the plane of the marker and centrally located in the marker.

Description

[Technical field]

[0001] The following relates generally to vision marker systems, and more specifically to visual markers for robotic or autonomous applications, and systems and methods for making and using the same. [Background technology]

[0002] Visual targets or markers are used in space proximity operations to aid in the alignment of mechanical interfaces, for example, during spacecraft rendezvous and docking, satellite capture and mooring, and robotic servicing of space structures. Distinctive and precisely located target features facilitate unambiguous and reliable detection and facilitate accurate estimation of relative position and orientation (attitude) by human operators or automated vision systems. Targets are designed for use by astronauts (using direct observation or camera images) or for automated detection by vision systems. Existing targets used by vision systems typically do not allow discrimination and do not allow multiple targets to be visible within the field of view. Targets are detected in a first image and tracked in successive frames, which introduces potential failure situations. Attitude estimation using existing targets can be difficult due to a lack of features that allow such attitude estimation or accuracy limitations due to the location of features on the target. Summary of the Invention [Problem to be solved by the invention]

[0003] Thus, there is a need for improved vision marker systems, vision marker methods, and vision markers that overcome at least some of the shortcomings of existing systems and methods, such as by providing reliable detection and accurate determination and identification of relative position and attitude (orientation). [Means for solving the problem]

[0004] A machine vision marker for use on a target object is provided, the marker including a first black target element comprising a black square, a square coding stripe extending around and surrounding a perimeter of the black square, the square coding stripe being comprised of a plurality of square coding elements that collectively encode a binary data bit sequence that encodes identification information for the target object, each one of the plurality of square coding elements being either a black square or a white square that respectively represents a 0 or 1 data bit in the binary data bit sequence, a white target element comprising a white square ring extending around and surrounding the perimeter of the square coding stripe, a second black target element comprising a black square ring extending around a perimeter of the white square ring of the white target element, and an out-of-plane white dot post located above or below the plane of the marker and centered in the black square of the first black target element.

[0005] The square coding stripe, the white square ring of the white target element, and the black square ring of the second black target element may each have a width equal to the width of one target coding element.

[0006] The marker may further include a black post for elevating the out-of-plane white dot above the plane of the marker, the black post having a base at a first end for attaching the black post to at least one of the target elements and a post tip at a second end opposite the first end at which the out-of-plane dot is located.

[0007] The post tip may include a black ring around an out-of-plane white dot.

[0008] Each of the white target elements, the white square coding elements, and the out-of-plane white dots may be constructed from a retroreflective material.

[0009] The black square coding element may be integrally formed with the first black target element and disposed around the periphery of the black square of the first black target element to form the target pattern element, and the white target element may be a white square, and the white square ring of the white target element forms the outer periphery of the white square, and the target pattern element may be disposed in the center of the white target element such that the white square coding element of the square coding stripe is formed from the portion of the white square around the periphery of the black square of the target pattern element that is not obscured by the black square coding element.

[0010] 10. The marker of claim 1, further comprising a recess on the visualized side of the marker for lowering the out-of-plane white dot below the plane of the marker, the out-of-plane white dot disposed within the recess.

[0011] The square coding stripes may contain redundant encoding of the target object's identification information that allows for up to three bit inversions in the binary data bit sequence to correctly decode the target object's ID.

[0012] A square coding stripe may include four corner square coding elements, and the data bit values ​​of the four corner coding elements may be predetermined based on a fixed corner square coding element value used across all markers of which the marker is an instance.

[0013] The four corner square coding elements may include a top-left coding element, a top-right coding element, a bottom-right coding element, and a bottom-left coding element, where the top-left and top-right coding elements are black squares and the bottom-left and bottom-right coding elements are white squares, or the top-left and top-right coding elements are white squares and the bottom-left and bottom-right coding elements are black squares.

[0014] A data bit sequence may be encoded clockwise within the square encoding stripe, with a first data bit of the data bit sequence being encoded into a given one of the four corner target encoding elements.

[0015] The first black target element may be at least 10 square coding elements wide, and the square coding stripe, the white square ring of the white target element, and the black square ring of the second black target element may each have a width equal to one square coding element.

[0016] The first black target element may be at least 10 target coding elements wide and the second black target element may be at least 16 target coding elements wide.

[0017] The out-of-plane white dot may have a dot diameter equal to the diagonal length of one square coding element.

[0018] The target pattern element, the white target element, and the second black target element may be three separate pieces of material that are assembled to form the marker.

[0019] The second black target element may be a square piece of material having a square-shaped recess dimensioned to receive the white target element therein and a non-recess extending around the recess forming a black square ring, and the marker may be assembled by mounting the white target element within the square-shaped recess of the second black target element and mounting the first black target element over the white target element.

[0020] According to some embodiments, the target pattern element comprises a beveled edge.

[0021] A system is provided that includes a machine vision marker and a machine vision system. The machine vision marker includes a first black target element that includes a black square, a square coding stripe that extends around and surrounds the perimeter of the black square, the square coding stripe being composed of a plurality of square coding elements that collectively encode a binary data bit sequence that encodes identification information about the target object, each one of the plurality of square coding elements being either a black square or a white square that respectively represents a 0 or 1 data bit in the binary data bit sequence, a white target element that includes a white square ring that extends around and surrounds the perimeter of the square coding stripe, a second black target element that includes a black square ring that extends around the perimeter of the white square ring of the white target element, and an out-of-plane white dot post located above or below the plane of the marker and centered on the black square of the first black target element. The machine vision system includes an imaging device for capturing image data of the target, and a processing device. The processing device is configured to detect a square coding stripe in the image data, decode corner data bit values ​​for each of the four corner square coding elements of the square coding stripe, determine an orientation of the marker in the image data using the corner data bit values ​​by referencing pre-defined corner values ​​for the marker stored in the memory, decode a data bit sequence using the determined orientation, and determine whether the decoded data bit sequence matches a reference data bit sequence stored in a reference database.

[0022] The reference data bit sequence may be linked in the reference database to target object identification data identifying attributes of the target object, and the processing device may be further configured to use the linked target object identification data in subsequent processing operations if the decoded data bit sequence matches the reference data bit sequence.

[0023] The target object identification data may include a target object ID, and the processing device may be configured to identify the target object using the target object ID.

[0024] The processing device may be further configured to detect corners of the white squares in the image data of the marker for use as pose calculation features, and to determine a pose of the target object using the pose calculation features.

[0025] In one embodiment, the processing device may detect the corners of the white square using a region-based detector, such as a maximum stable outer region ("MSER") detector, which finds pixels that belong to light regions surrounded by dark regions. The processing device may then be further configured to decompose the contour into quadrilaterals. The processing device may then be further configured to find sub-pixel equations of four straight lines of the quadrilateral. The processing device may then be further configured to intersect the four straight line equations to find the sub-pixel corners of the white square.

[0026] The processing device may be further configured to refine the pose of the target object using the out-of-plane white dots.

[0027] A method is provided for assembling a machine vision marker for use on a target object, the method including the steps of: providing a square black base plate having a top surface and a bottom surface, the bottom surface for mounting the base plate to the target object, the top surface including a square recess disposed in the center of the top surface and a non-recess forming a black square ring around the square recess; disposing a square white plate having a top surface and a bottom surface within the square recess of the square black base plate such that the bottom surface contacts a bottom surface of the recess; and attaching a black target pattern plate to the top surface of the square white plate, the black target pattern plate comprising an inner square and a plurality of black data bit encoding squares disposed around a perimeter of the inner square, the plurality of black data bit encoding squares collectively constituting an encoding area of ​​the black target pattern plate that is a width of one data bit encoding square, and the attachment of the black target pattern plate to the top surface of the square white plate results in a square recess formed around the perimeter of the inner square of the black target pattern plate. forming a coding stripe, the square coding stripe including a plurality of black data bit coding squares of the black target pattern plate and a plurality of white data bit coding squares formed by square areas of the square white plate not obscured by the coding areas of the black target pattern plate, the plurality of black data bit coding squares and the plurality of white data bit coding squares of the square coding stripe collectively encoding a data bit sequence representing identification information about the target object; and attaching a post to at least one of the black base plate, the white plate, and the target pattern plate such that the post is positioned at the center of the target, the post including a black rod having a base for attaching the post to at least one of the black base plate, the white plate, and the target pattern plate and a tip opposite the base, and a white dot positioned at the center of a top surface of the tip of the rod.

[0028] A method is provided for manufacturing a square machine vision marker with improved roll ambiguity characteristics for determining marker orientation in use, the method comprising the steps of manufacturing a plurality of square machine vision markers each having a square coding stripe including a plurality of square coding elements, each square coding element being black or white, the square coding stripe encoding a binary data bit sequence decodable by a machine vision system to obtain information about a target object with which the marker is displayed, the corner square coding elements being configured such that each square coding stripe of the plurality of square machine vision markers includes the same predefined corner square coding element value, the two non-opposing corner coding elements are black squares and the two non-opposing white corner coding elements are white squares, such that the corner square coding element value is fixed and consistent across the plurality of square machine vision markers.

[0029] The multiple square machine vision markers may include at least a first machine vision marker having a first square coding stripe and a second machine vision marker having a second square coding stripe, where the first and second square coding stripes are different.

[0030] A method for detecting a target pose of a machine vision marker is provided, the method including the steps of determining a target orientation of the machine vision marker, determining a target ID of the machine vision marker, and calculating a target pose of the machine vision marker.

[0031] According to some embodiments, the method further comprises determining a target size using the target ID.

[0032] According to some embodiments, determining the target orientation includes sampling corner data bits of a machine vision marker.

[0033] According to some embodiments, determining the target ID includes sampling data bits of the machine vision marker.

[0034] According to some embodiments, the method further includes capturing an image including at least one machine vision marker.

[0035] According to some embodiments, the method further comprises performing an unwarping operation on the image to obtain an unwarped image, and using the unwarped image to determine the target orientation.

[0036] According to some embodiments, the method further comprises performing blob detection on the unwarped image prior to the step of determining the target orientation.

[0037] According to some embodiments, blob detection includes maximum stable outer area blob detection.

[0038] According to some embodiments, the method further includes performing blob contour tracing on the blobs detected by the blob detection to obtain a contour corresponding to an exterior of the white element of the machine vision marker.

[0039] According to some embodiments, the blob contour tracing includes a vertex tracing algorithm.

[0040] According to some embodiments, the method further comprises the step of computing the convex hull of the blob contour.

[0041] According to some embodiments, the method further comprises determining a closest quadrilateral from the convex hull.

[0042] According to some embodiments, the method further comprises calculating a homography of the quadrilateral to determine the data bit sampling locations.

[0043] According to some embodiments, the method further includes performing a sub-pixel line fit on the image to obtain sub-pixel transitions of the step edges of the white elements of the machine vision marker.

[0044] According to some embodiments, the method further includes determining a position of a post tip of the machine vision marker.

[0045] According to some embodiments, calculating the target pose of the machine vision marker includes applying a quadrilateral to the position of the post tip.

[0046] According to some embodiments, the target ID is any one or more of a marker ID, a target object ID, and an interface type ID.

[0047] Other aspects and features will become apparent to those of ordinary skill in the art upon review of the following description of several exemplary embodiments.

[0048] The drawings included herein are intended to illustrate various examples of the articles, methods, and apparatus herein. [Brief description of the drawings]

[0049] [Figure 1] FIG. 2 is a front view of a machine vision marker showing the side of the machine vision marker that is visualized by a machine vision system (the "visible side"), according to one embodiment. [Diagram 2] FIG. 2 is a front view of an example of the machine marker of FIG. 1 including a coding area made up of black and white target coding elements, according to one embodiment. [Diagram 3]A schematic diagram of a portion of a machine vision marker including an out-of-plane white dot surrounded by a black ring and elevated above or below the plane of the marker, showing the separation of the out-of-plane white dot from the projection of the white portion of the marker behind the out-of-plane white dot, in accordance with one embodiment. [Figure 4] FIG. 2 is a front view of two machine vision markers of the same size, including a first machine vision marker having a configuration described in this disclosure in accordance with one embodiment, and a second machine vision marker having a different configuration, showing the locations of pose calculation features on the first and second machine vision markers. [Diagram 5] 2A-2C are front perspective views of two example machine vision markers of FIG. 1 showing features of the markers used to determine pose, according to an embodiment. [Figure 6] FIG. 2 is an exploded perspective view of an example of the machine vision target of FIG. 1, according to one embodiment. [Figure 7] FIG. 7 is a flow diagram of a method for assembling the machine vision marker of FIG. 6 according to one embodiment. [Figure 8] FIG. 7 is a perspective view of the example machine vision marker of FIG. 6 in an assembled form, according to one embodiment. [Figure 9] FIG. 1 illustrates a front perspective view of a plurality of machine vision markers having different coding regions, according to an embodiment. [Figure 10] FIG. 1 is a front view of a large machine vision marker and a small machine vision marker combined into a dual machine vision marker sharing a single out-of-plane white dot, according to one embodiment. [Figure 11] FIG. 2 is a block diagram of a system for identifying and determining the pose of a target object using the machine vision markers of FIG. 1, according to one embodiment. [Figure 12] FIG. 2 is a flow diagram of a method for detecting and identifying machine vision markers, such as the markers of FIG. 1, and determining the pose of a target object using the machine vision markers, according to one embodiment. [Figure 13]FIG. 2 is a schematic diagram of a machine vision marker such as the marker of FIG. 1 further illustrating the role of various coding elements (data bits) within the square coding stripe, according to one embodiment. [Figure 14] 2 is a schematic diagram of a machine vision marker such as the marker of FIG. 1 showing where sampling points are defined on the marker relative to the white square corners of the marker for initial image processing of the marker image, according to one embodiment. [Figure 15A] FIG. 2 is a first perspective view of a machine vision marker having an out-of-plane white dot recessed below the plane of the square coding stripe, according to one embodiment. [Figure 15B] FIG. 15B is a second perspective view of the machine vision marker of FIG. 15A. [Figure 16] FIG. 2 is a flow diagram of a method for detecting and identifying machine vision markers, such as the markers of FIG. 1, and determining the pose of a target object using the machine vision markers, according to one embodiment. [Figure 17] FIG. 2 is a flow diagram of a method for detecting and identifying machine vision markers, such as the markers of FIG. 1, and determining the pose of a target object using the machine vision markers, according to one embodiment. [Figure 18] FIG. 2 illustrates image warping showing example warped and unwarped images. [Figure 19] FIG. 1 illustrates a contour tracking algorithm, according to one embodiment. [Figure 20] 1 is a front view of a machine vision marker showing a marker with identified bit sampling locations and a marker without identified bit sampling locations according to one embodiment. [Figure 21] 1A-1D are perspective and detailed perspective views of a machine vision marker and marker variations demonstrating the functionality of the marker's chamfered or beveled structure, according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0050] Various devices or processes are described below to provide examples of each claimed embodiment. The embodiments described below do not limit any claimed embodiment, and any claimed embodiment may cover a process or device different from the one described below. A claimed embodiment is not limited to a device or process having all of the features of any device or process described below, or to features common to some or all of the devices described below.

[0051] One or more systems described herein may be implemented in a computer program running on a programmable computer, each of which comprises at least one processor, a data storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. For example, but not limited to, the programmable computer may be a programmable logic unit, a mainframe computer, a server, and a personal computer, a cloud-based program or system, a laptop, a personal digital assistant, a mobile phone, a smartphone, or a tablet device.

[0052] Each program is preferably implemented in a high level procedural or object oriented programming and / or scripting language to communicate with a computer system. However, the programs can be implemented in assembly or machine language, if desired. In either case, the language can be a compiled or interpreted language. Each such computer program is preferably stored on a general purpose or special purpose programmable computer readable storage medium or device for configuring and operating a computer when the storage medium or device is read by a computer to perform the procedures described herein. The computer programs described herein may include or implement machine learning or machine intelligence computer programs, software applications, software modules, models, etc. for performing various computing tasks described herein.

[0053] A description of an embodiment having several components in communication with each other does not imply that all such components are required, rather, a variety of optional components are described to illustrate the wide variety of possible embodiments of the present invention.

[0054] Additionally, although process steps, method steps, algorithms, etc. may be described (in this disclosure and / or claims) in a sequential order, such processes, methods, and algorithms may be configured to operate in another order. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. Steps of processes described herein may be performed in any order that is practical. Additionally, some steps may be performed simultaneously.

[0055] It will be readily apparent that where a single device or article is described herein, two or more devices / articles (whether or not they cooperate) may be used in place of the single device / article. Similarly, it will be readily apparent that where two or more devices or articles are described herein (whether or not they cooperate) a single device / article may be used in place of the two or more devices or articles.

[0056] The following relates generally to vision marker systems, and more specifically to visual markers for robotic or autonomous applications, and systems and methods for manufacturing and using the same. The machine vision markers of the present disclosure can be used to identify a target object on which the machine vision marker is placed, and to determine the position and orientation ("pose") of the target object.

[0057] This disclosure provides a new approach to designing and using visual targets and markers for space operations. The approach incorporates both a redundant design and appropriate computer vision algorithms that allow for reliable detection even with significant data loss in the images. Encoding the marker ID allows the same structure to be used for both object identification and pose determination. In one embodiment, a three-dimensional ("3D") target structure is used to provide more accurate pose estimation. Markers can be detected in every image without relying on processing of previous images (i.e., without tracking).

[0058] The machine vision marker design of the present disclosure provides reliable detection, accurate attitude determination, and identification in space close-proximity operations. The machine vision marker of the present disclosure provides accurate detection from a wide range of distances and viewing angles. The geometric pattern on the machine vision marker may encode information within the data bits, advantageously allowing for unique identification of the marker (e.g., target or marker ID corresponding to the data bits) and resolving roll ambiguity due to roll symmetry of the four sides of the square marker. The data bits can be used by a software application to determine the dimensions of a given marker (such as by using the determined marker ID to obtain marker dimension data), which can indicate how large or small the marker is constructed to be. The marker dimension information is used to determine the marker range, marker orientation, and marker pitch / yaw.

[0059] Furthermore, the redundant design and coding features in the machine vision marker provide reliable operation, including in the case of partial data loss due to shadows, specular reflections, and occlusions. The three-dimensional structure of the machine vision marker, in particular the post protruding from the center of the machine vision marker, can improve accuracy in calculating the pose. The method of detecting machine vision markers described in this disclosure can provide detection of the marker in every frame without frame-to-frame tracking, which can advantageously enable instant recovery from occlusions or image losses.

[0060] Although aspects of the present disclosure describe the use of machine vision markers as space vision markers (i.e., in space-based applications), it should be understood that the machine vision markers, and machine vision systems and methods, may be used in non-space contexts in which machine vision markers are used, providing similar advantages. Thus, the use of the machine vision markers of the present disclosure is not limited to space applications.

[0061] Referring now to FIG. 1, shown therein is a machine vision marker 100 according to one embodiment.

[0062] The depicted view of machine vision marker 100 in FIG. 1 shows the "visible side" of machine vision marker 100, i.e., the side of machine vision marker 100 that is intended to be visualized or imaged by a machine vision system.

[0063] The machine vision marker 100 (also referred to as a "vision marker," "marker," "vision target," or "target") may be used for detection and identification of an object to which the marker 100 is attached. In one embodiment, the object is a space object, and the vision marker 100 may be used to detect and identify the space object during an operation in space. The operation may be, for example, a capture, docking, or rendezvous operation. Similarly, the vision marker 100 may be attached to an object to be used in space, or a replica or emulation thereof, for ground testing of the interaction of a robotic system using the vision marker 100.

[0064] The machine vision marker 100 may be used in a robotic system (space-based or otherwise) to detect and determine the identity and pose of an object where the marker 100 is visible to the machine vision system.

[0065] For example, a robotic system including a robotic arm or manipulator with an end effector or capture tool may be configured to visualize the marker 100 via a camera system. The output of the visualization of the marker 100, such as the ID or pose of the target object, may be used by the robotic system to control the robotic manipulator so that the capture device captures the object. The capture of the object may then allow the object to be manipulated via the robotic manipulator by maneuvering the object from one location to another, actuating the object (e.g., applying torque to the object), or passing data / power to and from the object. In this manner, the machine vision marker 100 may provide important information to the robotic system to enable effective autonomous interaction with the object.

[0066] In general, machine vision markers 100 contain encoded information and characteristics that, when visualized, allow for the determination of the identity of the object to which the vision marker 100 is attached and the pose of the object.

[0067] The machine vision marker 100 may be attached to an exterior surface of the target object or otherwise positioned or attached on the target object such that the machine vision marker 100 can be visualized by an external machine vision system.

[0068] The machine vision marker 100 includes a first black target element 102, a square coding stripe 104, a white target element 106, a second black target element 108, and an out-of-plane white dot 110. In some embodiments, the out-of-plane white dot 110 may not be present.

[0069] The first black target element 102 includes a black square (also referred to as an "inner black square") defined by a square perimeter 112. In some cases, as described herein, the first black target element 102 may include more than just the inner black square (e.g., a portion of the first black target element 102 may extend beyond the perimeter but may be obscured by other marker components when the marker is in the assembled configuration).

[0070] The square coding stripe 104 extends around the black square perimeter 112 of the first black target element 102. For example, the square coding stripe 104 can abut the black square perimeter 112 of the first black target element 102.

[0071] The square coding stripe 104 can generally be defined as the space between the inner black square perimeter 112 of the first black target element 102 and the inner square perimeter 120 of the white square ring of the white target element 106 (the white square ring of the white target element 106 is the portion of the white target element 106 visible in FIG. 1).

[0072] The square coding stripe 104 includes a plurality of square shaped target coding elements 114 .

[0073] The coding elements 114 are generally arranged in a side-by-side configuration extending around the length of the square coding stripe 104 (one side of each coding element 114 abuts the black square perimeter 112 of the first black target element 102 and a second opposing side abuts the inner perimeter 120 of the white square ring of the white target element 106, except in the case of corner coding elements where a corner of the coding element abuts a corner of the black square perimeter 112).

[0074] The square coding stripe 104 has a width 115 of one target coding element. In other embodiments, the width 115 of the square coding stripe 104 may vary. In other embodiments, the square coding stripe 104 may have a width greater than one square ring of data bits (coding elements) to increase the number of target coding elements in the marker 100. Such a configuration may be used to provide more markers (i.e., a greater number of possible data bit sequences) or more redundancy (resistance to more bit flips). Thus, the width of the square coding stripe 104 may vary and may depend on the needs. In a particular embodiment, such as the marker 1300 shown in FIG. 13 (where one square of the grid superimposed on the marker is equal to a coding element in the coding region), the marker may use a coding stripe 104 with a width of one target coding element to provide 8191 unique marker IDs with error correction of three bit flips.

[0075] The target encoding elements 114 collectively encode a binary data bit sequence that encodes the identity of the object on which the marker 100 is placed.

[0076] The identification information, once decoded, can be used to determine the identity of the target object and / or interface associated with the marker or target ID. Generally speaking, the identification information encoded within the binary data bit sequence can be used to distinguish the target object from other objects. This distinction may include distinguishing between classes or types of objects and / or distinguishing between objects within the same class or type. The data bit encoding can be used to "throw out" bright squares in the image that are not markers, so that the system does not detect non-existent markers (false positives). This can be done by determining whether the data bit sequence matches a marker ID in a marker ID database. Thus, the system incorporates a form of noise removal that can remove image artifacts that resemble the target's geometry prior to processing.

[0077] By varying the pattern of target coding elements 114 within the square coding stripe 104 (i.e., which coding elements are black and which coding elements are white), different information can be encoded within the square coding stripe 104. In this manner, the square coding stripe 104 can be varied between different instances of the marker 100 to encode identifying information corresponding to different target objects to enable identification of (and distinction from other) different target objects.

[0078] Each target coding element 114 is either a black square or a white square. A black square coding element encodes or represents a data bit of 0 in the binary data bit sequence encoded in the square coding stripe 104. A white square coding element encodes or represents a data bit of 1 in the data bit sequence encoded in the square coding stripe 104. The use of black and white coding elements may allow intensity values ​​of the coding elements to be determined by suitable computer vision techniques, and the intensity values ​​are used to determine the encoded data bit sequence.

[0079] 1 shows a representative black square target coding element 116 and a representative white square target coding element 118 of a square coding stripe 104. The remainder of the square coding stripe 104 in FIG. 1 is shaded gray. The gray shading of the square coding stripe 104 is intended to represent that the individual target coding elements 114 within the square coding stripe 104 may be black or white, depending on the coding of the particular target 100. Thus, in an actual implementation of the marker 100, the square coding stripe 104 will be filled with some combination of black squares 116 and white squares 118.

[0080] A specific example of a vision marker 100 having a complete coding area is shown in FIG. 2, which shows an example 200 of the machine vision marker 100 of FIG. 1 in which the square coding stripe 104 is made up of black and white coding elements that represent a specific data bit sequence.

[0081] The exemplary marker 200 is shown to include grid lines. The grid lines are included for illustrative purposes. It should be understood that the grid lines may not be present in the actual marker 200.

[0082] In the example 200 of FIG. 2, the square coding stripe 104 encodes a binary data bit pattern of 00010000000010000000011010110010011100011111. The square coding stripe 104 may include data encoded with a redundant coding. The redundant coding may allow recovery from (i.e., the ability to still decode) up to three bit flips. Such bit flips may occur, for example, due to obscuration, noise, or poor lighting. The redundant coding is achieved using a coding redundancy scheme. In one embodiment, the redundant coding may be achieved using BCH coding, which adds a prefix to the actual message. For example, a total of 2 bits encoded within a 27-bit code may be encoded as 0001000000001000000011010110010011100011111. 13 messages are possible. Each code differs from every other code by at least 7 bits. This means that any code can differ from the true code by up to 3 data bits and still be the "closest" to the correct code (i.e., the scheme can handle 3 bit flips and still arrive at the correct code). When a code differs by 4 or more bits, it is actually closer to another code than the correct code.

[0083] In some cases, “intermediate” data bits (intermediate data bits are data bits that are not predefined corner data bits, as described herein) in the data bit sequence may be decoded to obtain the marker ID of marker 200.

[0084] The upper left corner coding element 202 a (ie, the upper left in the diagram shown in FIG. 2) of the square coding stripe 104 represents the start 204 of the data bit pattern encoded by the square coding stripe 104 .

[0085] The starting point 204 is the first data bit in the data bit sequence. The data bit sequence is encoded in the square coding stripes 104 starting from the upper left corner coding element 202a and proceeding clockwise, as indicated by the arrows in FIG.

[0086] The data bit sequence proceeds from the top-left corner coding element 202a to the top-right corner coding element 202b to the bottom-right corner coding element 202c to the bottom-left corner coding element 202d and then to the coding element immediately preceding the top-left corner coding element 202a.

[0087] Corner encoding elements 202 a , 202 b , 202 c , 202 d are collectively referred to herein as corner encoding elements 202 , and generally referred to as corner encoding elements 202 .

[0088] In a variant, the data bit sequence may be encoded within the square coding stripe 104 in any predetermined order that allows the data bit sequence to be obtained from the square coding stripe 104. For example, in another embodiment, the data bit sequence may be encoded counterclockwise from the starting point 204.

[0089] In other embodiments, starting point 204 may change location within square coding stripe 104 such that another coding element is the first coding element in the data bit sequence encoded within square coding stripe 104. For example, starting point 204 may be any of corners 202b, 202c, or 202d.

[0090] In one embodiment, the design and decoding software of the marker 100 may be configured to first determine which corner of the square coded region 104 is the top left corner. To make such a determination, the orientation of the marker 100 must be known. The determination of the marker orientation is resolved using the corner bits of the square coded stripe 104. Once the orientation of the marker is determined (using the corner bits), the decoding software is further configured to determine the marker ID by starting the data bit stream at the correct corner (i.e., the top left corner in this case). Starting at another corner of the square coded region 104 without resolving the role ambiguity would result in obtaining a different (incorrect) marker ID. The order, direction, and what the bits mean are entirely dependent on the marker design and decoding software. In other embodiments, the marker design and decoding software may be configured to implement a different protocol for selecting the starting square of the square coded stripe 104 or for selecting the reading direction of the square coded stripe 104 once the starting square is identified (i.e., using a different starting square and / or a different reading direction).

[0091] As described with reference to FIG. 1, in the marker 200 of FIG. 2 it can be seen that the square coding stripe 104 is positioned between the inner black square (whose outer periphery is indicated by line 206) of the first black target element 102 and the inner periphery of the white square ring of the white target element 106.

[0092] The square coding stripe 104 also includes a predetermined corner value. The term "predetermined corner value" refers to the fact that the value of each corner coding element 202a, 202b, 202c, 202d (i.e., whether the corner element 202 is black or white) is predetermined, known, and consistent across instances of the marker 200. For example, a collection of markers 200 may all be manufactured with the same predetermined corner value. In other words, the predetermined corner values ​​include coding elements (and corresponding data bits) that are at or near the corners of the square coding stripe 104 and are used to determine the marker orientation (four-fold roll ambiguity) and are consistent across all instances of the marker. In contrast, the "intermediate" data bits or "intermediate" coding elements are between the predetermined corner coding elements and include coding elements (and corresponding data bits) that may be used to encode information that can be used to determine the marker ID.

[0093] The pre-defined corner values ​​function as marker orientation features, i.e., the known pre-defined corner values ​​can be used to determine the orientation of the marker 200 (e.g., which way is up) and more specifically the orientation of the square coding stripes 104 in the target marker image. By allowing the orientation of the square coding stripes 104 to be determined, the start point 204 of the data bit sequence can be properly identified and the data bit sequence can be correctly decoded.

[0094] In functioning as a marker orientation feature, the predetermined corner values ​​of the square coding stripe 104 may be advantageously used to resolve the ambiguity of the four-fold roll of a square target such as target 100.

[0095] The predetermined corner values ​​may also provide the advantage of allowing the orientation of the target marker 100 to be determined by both machine vision techniques and human observation.

[0096] The predetermined corner values ​​are configured such that two of the corner coding elements 202 are white (data bit value 1, 202c, 202d) and two of the corner coding elements are black (data bit value 0, 202a, 202b), with two black corner coding elements (202a, 202b) at adjacent corners such that each black corner coding element has one adjacent black corner coding element and one adjacent white corner coding element (i.e., no black corner coding elements are at opposite corners of the square coding stripe 104). For example, corner coding element 202a has one adjacent corner coding element (element 202b) that is black and one adjacent corner coding element (element 202d) that is white.

[0097] In a variant, the predefined corner values ​​may include top corner coding elements 202a, 202b having the same value (e.g., black, 0 or white, 1) and bottom corner coding elements 202c, 202d having the same value (e.g., white, 1 or black, 0), where the top corner element value and the bottom corner element value are different (e.g., top, black, 0; bottom, white, 1).

[0098] In one embodiment, the predefined corner values ​​include an upper-left corner coding element 202a and an upper-right corner coding element 202b, each having a value of 0 (i.e., a black square), and a lower-left corner coding element 202d and a lower-right corner coding element 202c, each having a value of 1 (i.e., a white square).

[0099] In certain embodiments, the predefined corner values ​​may include additional predefined values ​​other than at the corners, in particular at coding elements adjacent to the corner coding element 202 in the square coding stripe 104 (i.e., coding elements immediately to the side of the corner coding element 202 in the coding stripe 104).

[0100] For example, in one embodiment, adjacent black corner coding elements (202a, 202b) each have adjacent black coding elements and adjacent white coding elements (i.e., each black corner element is adjacent to a black element on one side and adjacent to a white element on the other side), and adjacent white corner coding elements (202c, 202d) each have adjacent white coding elements and adjacent black coding elements (i.e., each white corner element is adjacent to a white element on one side and adjacent to a black element on the other side).

[0101] For black corner coding elements, the adjacent black coding elements of each of the two black corner coding elements are located on the same side of the square coding stripe 104 (adjacent white coding elements are located on opposite sides of the square coding stripe).

[0102] For the white corner coding elements, the adjacent black elements of each white corner coding element are located on the same side of the square coding stripe 104 (adjacent white coding elements are located on opposite sides of the square coding stripe).

[0103] In addition to resolving orientation determination and roll ambiguity, such configuration of corner coding element values ​​and adjacent coding element values ​​may advantageously enable manufacturability of the marker 100 in embodiments of the marker 100 in which the black coding elements of the square coding stripe 104 are part of the first black target element 102 (i.e., by being integrally formed with the inner black square), because such a construction ensures that each black corner coding element has at least one adjacent black coding element to support attachment of the black corner element to the inner black square (if there are no adjacent black elements, such as when both elements adjacent to a black corner are white, the only points of contact between the black corner element and the inner black square will be the respective corner points of the black corner element and the inner black square).

[0104] An example of such a configuration is shown in marker 200 of FIG. 2. The black corner coding element includes an upper left coding element 202a and an upper right coding element 202b. The white corner coding element includes a lower left coding element 202d and a lower right coding element 202c. The black corner coding elements 202a, 202b are adjacent corners (i.e., not opposite corners). The upper left corner coding element 202a has an adjacent white coding element 208 and an adjacent black coding element 210. The upper right corner coding element 202b has an adjacent white coding element 212 and an adjacent black coding element 214. The adjacent black coding elements 210 and 214 of the upper left corner element 202a and the upper right corner element 202b are located on the same side 216 of the square coding stripe 104. The lower left corner coding element 202d has an adjacent white coding element 218 and an adjacent black coding element 220. The bottom right corner coding element 202c has an adjacent white coding element 222 and an adjacent black coding element 224. The adjacent black coding elements 220 and 224 of the bottom left corner element 202d and the bottom right corner element 202c are located on the same side 226 of the square coding stripe 104.

[0105] Referring again to FIG. 1, the white target element 106 includes a white square ring. In FIG. 1, the white square ring is seen surrounding the square coding stripe 104. The white square ring may be made of a retroreflective material. The white square ring surrounds the square coding stripe 104. In this sense, the white square ring is disposed outside the coding element that codes the data bit sequence.

[0106] The second black target element 108 includes a black square ring. The second black target element 108 is composed of a black material. The black square ring of the second black target element 108 surrounds the white square ring of the white target element 106.

[0107] The white target element 106 and the second black target element 108 may each include additional white and black material, respectively, beyond their respective square rings. For example, a portion of the white target element 106 may be obscured by other elements of the marker 100, such as the square coding stripe 104 or the inner black square, such that only the white square stripe is visible. Similarly, a portion of the second black target element 108 may be obscured by other elements of the marker 100, such as the white target element 106, the square coding stripe 104, and the inner black square. Such obscuration of a portion of the white target element 106 and a portion of the black target element 108 may occur, for example, when components of the target 100 are placed on top of one another when assembling the target 100.

[0108] The out-of-plane white dot 110 may be elevated or recessed relative to the plane of the marker 100. For example, in situations where a protruding post is undesirable, a recessed version of the marker 100 may be used. The plane of the marker 100 is formed by the visualized surfaces of the target elements 102, 106, 108 and the coding stripe 104 (i.e., the surfaces shown in FIG. 1). The marker 100 may include any suitable structure to elevate the out-of-plane white dot 110 above the plane of the marker 100 or to lower the out-of-plane white dot 110 below the plane of the target 100. Examples of such structures are provided herein. Having the white dot 110 "out-of-plane" from the rest of the marker 100 can provide superior pitch / yaw accuracy.

[0109] The out-of-plane white dot 110 of the marker 100 is located in the center of the marker 100. In other embodiments, the out-of-plane white dot 110 may be located elsewhere on the visualized surface of the marker 100. For example, in a dual marker embodiment including two markers 100 of different sizes combined into a single marker (e.g., FIG. 10 ), the larger marker may include the out-of-plane white dot 110, which may be located outside the smaller marker that is located in the center of the larger marker.

[0110] In embodiments in which the out-of-plane white dot 110 is elevated relative to the plane of the marker 100, the white dot 110 may be placed at the end of a black post.

[0111] Raising (recessing) the white dot 110 above (or below) the surface of the target 100, and thus making the white dot 110 "out of plane", may advantageously reduce errors in attitude calculations. This may be especially true in the pitch and yaw of the target.

[0112] The out-of-plane white dot 110 may include a black ring around the periphery of the white dot 110. The black ring separates the white dot from the protrusion of the white part of the target behind the white dot (i.e., the white square ring, the white coding element). This feature is illustrated in FIG. 3. Preferably, the black ring around the white dot is wide enough to separate the protrusion of the white dot 110 and the white square behind it (i.e., with a few pixels between). In one embodiment, the width of the black ring may be set to 1 / 2 the data bit width. Preferably, the diameter of the white dot 110 is equal to the diagonal of the data bit (or sqrt(2)*data bit size).

[0113] Referring now to FIG. 3 , shown therein is a representation 300 of a portion 304 of a target marker including an out-of-plane white dot 110 with a surrounding black ring 302, an inner black square 306, a square coding stripe 308, a white square ring 310, and a black square ring 312, according to one embodiment.

[0114] Portion 304 of the target marker is part of marker 100, and components 306, 308, 310, and 312 correspond to components 102, 104, 106, 108 of FIG.

[0115] As can be seen, a black ring 302 surrounding the out-of-plane white dot 110 separates the white dot 110 from protruding white portions of the target 304 , such as the coding elements 314 , 316 of the square coding stripe 308 .

[0116] 1, the marker 100 includes regions 122a, 122b, 122c, and 122d. Regions 122a, 122b, 122c, and 122d are collectively referred to herein as regions 122, and collectively referred to as regions 122.

[0117] Region 122 is generally defined as where the outer corner of the white square ring of the white target element 106 meets the inner corner of the black square ring of the second black target element 108 .

[0118] The region 122 is a point defined by the intersection of a line caused by the intensity step between the inner white square ring 106 of the marker 100 and the black outer square ring 108 of the marker 100 (black border). Corners (vertices) are good features to use because their location can be determined in the 2D image and the 3D model. Knowing the location of the 2D image and the location of the 3D model, the pose of the marker 100 in the camera frame can be determined. Adding a fifth pose calculation feature of the out-of-plane white dot 110 can further improve the accuracy of the calculated pose. In one embodiment, the 2D position of the region 122 can be calculated by fitting a straight line equation with sub-pixel resolution to the pixel intensity step on all four sides of the marker's white square ring 106. The straight line equations are intersected to accurately obtain the corner points with sub-pixel resolution. The accuracy of the pose at a given camera-to-marker distance depends on the separation distance between the regions 122. Having regions 122 on the outside of the data bit square ring 104 increases the distance between regions 122 (as opposed to having regions 122 on the inside of the data bit square ring), which can provide improved attitude accuracy.

[0119] Region 122 can be used as a pose calculation feature, i.e., when processing an image of target marker 100 by a machine vision system, region 122 can be easily detected by computer vision algorithms executed by the machine vision system and processed by pose determination algorithms executed by the machine vision system to determine the pose of marker 100.

[0120] The positioning of region 122 on marker 100 may provide certain advantages over other vision marker configurations in calculating the pose of marker 100. Such advantages are illustrated in FIG.

[0121] 4, shown therein is a comparison of pose calculation features on a first vision marker 400 having a configuration described in this disclosure with a second vision marker 450 having a different configuration, according to one embodiment. The first vision marker 400 and second vision marker 450 are the same size.

[0122] The first vision marker 400 is an example of the vision marker 100 of FIG. 1 and includes regions 122a, 122b, 122c, 122d (pose calculation features) that can be used to determine the pose of the marker 100.

[0123] In the marker 400 , the white square ring of white target elements 106 is outside the square coding stripe 104 .

[0124] Marker 450 is constructed similarly to marker 400, with some important differences. Marker 450 includes an inner black square 452, a white square ring 454, a coding area 456, and an outer black square ring 458.

[0125] The marker 450 also includes regions 460 a , 460 b , 460 c , 460 d that can be used to calculate the pose of the marker 450 .

[0126] In marker 450, white square ring 454 is disposed inside coding region 456 (i.e., coding region 456 is outside white square ring 454).

[0127] Advantageously, the regions 122a, 122b, 122c, 122d used to calculate pose are further apart on marker 400 than the regions 460a, 460b, 460c, 460d on marker 450. In other words, the regions 122 on marker 400 are spaced further apart for targets of the same size.

[0128] Pose measurement errors can be reduced by increasing the distance between regions 122 on marker 400 compared to regions 460 on marker 450 (or other similar target configurations where the features used to calculate the pose are not as close to the outer edge of the marker).

[0129] Thus, the markers of the present disclosure, having a white square ring of white target elements 106 positioned outside the data bit coding area (square coding stripes 104) and between the outer black square ring of the second black target elements 108 and the coding area 104, may advantageously improve attitude calculation by reducing attitude measurement errors by increasing the distance between the attitude calculation areas 122 of the marker 100.

[0130] Referring now to FIG. 5, shown therein are first and second exemplary vision markers 500, 550 according to an embodiment.

[0131] Vision markers 500, 550 are example embodiments of vision markers 100 that include a black target rod attached to the center of the marker 100 with an out-of-plane white dot 110 (e.g., a white retroreflective disk) attached to the tip of the black target rod. The black target rod, and therefore the out-of-plane white dot 110, is elevated above the plane of the markers 500, 550.

[0132] The regions 122a, 122b, 122c, 122d of the markers 500, 550 that are used to calculate the pose are labeled.

[0133] Determining the pose of the markers 500, 550 can be performed by calculating an initial pose using the pose calculation features 122a, 122b, 122c, 122d and then refining the initial pose calculation using the position of the out-of-plane white dot 110.

[0134] For example, the distance of each of the pose calculation features 122a, 122b, 122c, 122d from the center of the out-of-plane white dot 110 can be calculated and compared / analyzed by a machine vision system processing images of the markers 500, 550 to provide further information regarding the pose of the markers 500, 550. This is because as the pose of the markers 500, 550 changes, the distance from the pose calculation feature 122 to the out-of-plane white dot 110 changes. Examples of such distances, and how they change based on pose, are shown in FIG. 5 as distances 510a, 510b, 510c, 510d on marker 500 and distances 560a, 560b, 560c, 560d on marker 550. As can be seen from distances 510a-510d compared to distances 560a-560d, markers 500 and 550 have different poses in FIG. 5.

[0135] Thus, the inclusion of an out-of-plane white dot 110 in the markers 500, 550 that is elevated above the plane of the marker may advantageously reduce errors in the attitude calculations of the markers 500, 550. This may be especially true in the pitch and yaw of the target marker.

[0136] The pose of the marker 100 is calculated using vision software (e.g., executed by a processor such as processor 1114 of FIG. 11). The pose of the object to which the marker 100 is attached can be inferred by knowing the rigid body transformation from the marker origin frame to the reference frame of the object of interest (e.g., the grapple fixture to be captured or the tool to be grasped). Thus, for example, in the process of integrating the marker 100 and the grapple fixture to the spacecraft, the transformation vector between the target reference frame and the grapple fixture reference frame can be measured, and also the transformation vector between the grapple fixture reference frame and the captured object's reference frame (sometimes, e.g., the center of mass) can be measured. Furthermore, reference measurements can be performed between the camera imager and the camera housing's reference frame (sometimes, e.g., its mounting bolt position), and reference measurements can be performed between the camera housing's reference frame and the capture reference frame of the end effector. These reference frame transformations are well understood to be necessary for robotic grapple operations and will be apparent to those skilled in the art.

[0137] 1, in some embodiments, the square coding stripe 104 can be a separate component from the white target element 106 and the first black target element 102. That is, the square coding stripe 104, the white target element 106, and the first black target element 102 are physically separate components that can be assembled together in the marker 100 such that the square coding stripe 104 is disposed between the white square ring of the white target element 106 and the inner black square of the first black target element 102.

[0138] In another embodiment, the square coding stripe 104 is composed of a white target element 106 and a first black target element 102, such that the white square coding element 118 of the square coding stripe 104 is provided by the white target element 106 and the black square coding element 116 of the square coding stripe 104 is provided by the first black target element 102.

[0139] For example, the white square coding element 118 may be part of the white target element 106, and the black square coding element 116 may be part of the first black target element 102, and the square coding stripe 104 is formed by the positioning of the white target element 106 and the first black target element 102 relative to one another. For example, the white target element 106 may be a square piece of white material including the square stripe visible in FIG. 1 at its periphery, and the first black target element 102 including the black square coding element 116 around its periphery 112 may be positioned over the white target element 106 such that the portion of the white target element 106 that occupies the square coding stripe 104 is obscured by the black square coding element 116 of the first black target element 102, and the unobscured portion of the white target element 106 in the square coding stripe 104 forms the white square coding element 118 of the square coding stripe 104.

[0140] In a particular embodiment, the black square coding elements 116 are disposed around the perimeter 112 of the inner black square of the first black target element 102 such that the inner black square and any black square coding elements 116 form a continuous piece of material. In such an embodiment, the white target elements 106 may include additional white material beyond what is visible in FIG. 1 (i.e., the white square ring) that extends into the square coding stripe 104. In one embodiment, the first black target element 102 including the black square coding elements 116 may be mounted above the white target element 106 such that a portion of the white target element 106 is obscured by the black square coding elements 116 within the square coding stripe 104 and a portion is not obscured. The unobscured portion of the white target element 106 present within the square coding stripe 104 forms the white square coding element 118.

[0141] The white components of the marker 100, such as the white target element 106, any white target coding elements 118 in the square coding stripe 104, and the off-plane white dots 110, may be constructed from retroreflective materials. For example, white pigments in such white components may be configured to emit light when a monochromatic light beam is incident on the marker 100. The marker 100 may be manufactured using a surface finish that provides high contrast and diffuse reflective properties. Such a marker 100 may be used under any illumination that produces an image suitable for processing. If the marker 100 includes a white portion constructed from a retroreflective material, the retroreflective material may reflect light toward the illumination source for a wide range of surface orientations. In such a case, a system for imaging the marker 100 may include a monochromatic light source (e.g., LED light) located near the camera and a camera lens with a bandpass filter that transmits the source light and attenuates any other illumination. In one embodiment, a filter on the camera lens may be tuned to the LED wavelength of the monochromatic LED attached to the camera to obtain a very bright white color.

[0142] The black components of the marker 100, such as the black squares of the first black target element 102, the second black target element 108, and any black square coding elements 116 in the square coding stripe 104, may be constructed from a light absorbing material to prevent scattering or reflection of the incident light source on the marker 100. The black components of the marker 100 may be constructed from a material that provides sufficient contrast with the white portions of the marker 100. The black material should be sufficiently dark, preferably black within the optical band of interest, and may be matte to avoid any specularity issues due to very intense and constantly changing illumination, such as that present in space operations.

[0143] In one embodiment, the square coding stripe 104 , the white square ring of the white target elements 106 , and the black square ring of the black target elements 108 each have a width equal to the width of one target coding element 114 .

[0144] 6 and 7, shown therein is a machine vision marker 600 and a method 700 of assembling the machine vision marker 600, according to one embodiment.

[0145] Referring first to Figure 6, shown therein is a machine vision marker 600 in an exploded view. Machine vision marker 600 is an embodiment of marker 100 of Figure 1. Thus, any of the following examples may apply to marker 100, and any of the examples discussed above with respect to marker 100 may apply to marker 600.

[0146] The marker 600 includes a first black target plate 602 , a white target plate 604 , a black target pattern plate 606 , and black posts 608 .

[0147] The first black target plate 602 corresponds to the second black target element 108 of Figure 1, the white target plate 604 corresponds to the white target element 106 of Figure 1, and the black target pattern plate 606 corresponds to the first black target element 102 of Figure 1. The square coding stripe 104 of the marker 600 is formed by the black target pattern plate 606 and the white target plate 604 when assembled.

[0148] The first black target plate 602 has a first surface 610 for attaching the first black target plate 602 (and, when assembled, the marker 100) to a surface of a target object (not shown), and a second surface 612 opposite the first surface 610.

[0149] The second surface 612 includes a square shaped recess 614 and a non-recessed outer square ring 616. The square shaped recess 614 is dimensioned to receive the white target plate 604.

[0150] The square recess 614 is defined by a bottom surface 618 and four side surfaces 620. The bottom surface 618 includes a circular opening / aperture 622 for receiving at least a portion of the black post 608 therethrough when the marker 600 is assembled.

[0151] The white target plate 604 is square-shaped and includes a first surface 624, a second surface 626 opposite the first surface 624, and four sides 628. The second surface 626 is composed of a white retroreflective material.

[0152] The first surface 624 is used to mount the white target plate 604 within the recess 612 of the first black target plate 602 , specifically to the bottom surface 618 of the recess 612 .

[0153] The white target plate 604 also includes a circular opening / aperture 630 that traverses from the first surface 624 to the second surface 626 for receiving at least a portion of the black post 608 therethrough when the marker 600 is assembled.

[0154] The black target pattern plate 606 includes a first surface 632 and a second surface 634 opposite the first surface 632. The first surface 632 is used to attach the black target pattern plate 606 to the second surface 626 of the white target plate 604 in the assembled marker 600.

[0155] The black target pattern plate 606 includes a circular opening / aperture 635 that traverses from the first surface 632 to the second surface 634 for receiving at least a portion of the black post 608 therethrough when the marker 600 is assembled.

[0156] The black target pattern plate 606 includes an inner square portion 636 and an outer target pattern portion 638. The outer target pattern portion 638 is comprised of a plurality of black square coding elements (corresponding to the black coding elements 116 of FIG. 1 ) disposed around and extending outwardly from the perimeter of the inner square 636. The combination of the outward protrusions 640 (which may include one black square coding element or multiple consecutive black square coding elements) and recesses 642 (which may be dimensioned to correspond to one white square coding element or multiple consecutive white square coding elements) in the outer target pattern portion 638 contribute to the formation of the square coding stripe 104 of the marker 600 when assembled.

[0157] In some examples, the black target pattern plate 606 may include beveled and / or angled edges. The bevel may be a chamfer. The bevel may be a 45 degree edge. Such angled edges may improve the readability of the marker 600 and / or the data encoded by the marker, especially when viewing the marker at a severe angle. An example of a beveled or angled edge can be seen in marker 1502 of FIGS. 15A-B.

[0158] This advantage is further illustrated in FIG. 21, which shows an oblique view 2102 of a marker 2100 having a chamfer according to an embodiment, and a detailed oblique view 2103 of a marker variant 2100B having a non-beveled edge, where the beveled edges of the coding stripe 2104, as shown in perspective 2110, prevent the coding stripe from obstructing visualization of the white elements 2106 in the marker 2100.

[0159] In contrast, a non-beveled marker 2100B is shown in which coding stripes 2104B comprise right-angled edges instead of the beveled edges of marker 2100. These right-angled edges of coding stripes 2104B obscure white elements 2016B from viewpoint 2110. Such obscuration may reduce the ability of a system or operator to decipher data encoded by coding stripes 2104B from a viewpoint such as viewpoint 2110 due to the thickness of the components that comprise coding stripes 2104B. For example, in the example marker 2100B, from viewpoint 2110, data bits may be counted as 0s (black) when they should be counted as 1s (white). If the coding stripe 2104 is constructed from sufficiently thick components or materials (e.g., as may be required for structural rigidity), it may be advantageous to chamfer, bevel, and / or bevel the edges of the components that comprise the coding stripe 2104B (e.g., as shown by stripe 2104) to improve readability or parsability of the marker at severe angles.

[0160] An example is now described. A machine vision marker may include a black data bit mask element (or black target pattern plate) that is structural and presses down on the retroreflector white target element of the marker. Due to this need for structural rigidity, the thickness of the data bit mask element may need to be increased. One consequence of increasing the thickness of the data bit mask is that when viewed at an angle, the black portion of the data bit mask element may block a portion of the white reflector that should appear white on the camera image. To counter this effect, a chamfer (or other bevel) may be added to the data bit mask element to allow off-axis viewing of the target without the black data mask portion blocking the white reflector portion of the marker. The chamfer ensures that light rays from the camera at a tighter angle to a given white data bit (formed by the white reflector) are not blocked. Without the chamfer (e.g., having a data bit mask element with a right-angled edge), the same light ray may instead hit the top black portion of the data bit mask. This may cause the machine vision software to count that data bit as a 0 (black) when it should be counted as a 1 (white). The bevel or chamfer therefore provides a marker that can be more precisely observed at a particular imaging angle.

[0161] 6, the black post 608 includes a rod 644 having a first end 646 and a second end 648 opposite the first end 646. The first end 646 is used to attach the rod 644 to the remainder of the marker 600 (e.g., the first black target plate 602).

[0162] At the second end 648 is a post tip that includes a generally circular retroreflective white disk 648. The white disk 648 corresponds to the out-of-plane white dot 110 in Figure 1. The post tip includes a black ring that surrounds the white disk 648.

[0163] In other embodiments, the post 608 may have a frusto-conical shape.

[0164] In another embodiment, the post 608 may be replaced with a recess (e.g., a recess below the coding stripe) in which the white disk 648 essentially functions as a "post" with a "negative length" as described herein.

[0165] Referring now to FIG. 7, there is shown a method 700 of assembling the visual marker 600 of FIG. 6 according to one embodiment.

[0166] At 702, the method 700 includes mounting the white target plate 604 within the square recess 612 of the first black target plate 602. This may include, for example, mounting a first surface 624 of the white target plate 604 to a bottom surface 618 of the recess 612 of the first black target element 602. The white target plate 604 may be dimensioned such that when the white target plate 604 is placed within the recess 612, a second surface 626 of the white target plate 604 is approximately level with the outer square ring portion 616 of the second surface 612 of the first black target element 602.

[0167] In one embodiment, the white plate 604 is press-fit and glued to the lower black plate 602 and is redundantly held down by the data bit pattern plate 606 and post 608. The post 608 may be held down by a bolt that passes through the entire lot to the post 608. Dowel pins may be used to align the marker 600 with the object to be tracked. The marker 600 may include mounting "ears" as seen in marker 800 of FIG. 8 that are asymmetrical to the dowel pins so that the marker 600 can only advance to the object in one direction. In another embodiment, the white plate 604 may be attached to the first black target plate 602 using a "picture frame" with a black border around the white reflector plate 604 to hold the white plate 604 and provide a line of contrast.

[0168] At 704, the method 700 includes attaching the black target pattern plate 606 to the white target plate 604. In particular, the first surface 632 of the black target pattern plate 606 is centrally attached to the second surface 626 of the white target plate 604. In some cases, centrally attaching the black target pattern plate 606 may include aligning the circular openings 635, 630 of the respective plates.

[0169] Attaching the black target pattern plate 606 to the white target plate 604 forms the square coding stripe 104 of the marker 600. This square coding stripe 104 is formed from a rectangular protrusion of the outer target pattern portion 638 of the black target pattern plate 606, which provides a black square coding element, and a rectangular portion of the second surface 626 of the white target plate 604 that fills a rectangular recess 642 in the outer target pattern portion (i.e., such portion of the white target plate 604 is visible and not obscured by the outer target pattern portion), which provides a white square coding element.

[0170] The perimeter of the square coding stripe 104 of the marker 600 is formed by the outer edge of a rectangular protrusion 640 .

[0171] Attaching the black target pattern plate 606 to the white target plate 604 in this manner also forms a white square ring on the white target plate 604 which corresponds to the square ring portion of the second surface 626 of the white target plate 604 that extends around (i.e., surrounds) the outside of the square coding stripe 104 (i.e., the portion of the surface 626 that is outside the outer edge of the rectangular protrusion 640).

[0172] At 706, the method 700 includes attaching the black post 608 to the assembled target plates 602, 604, 606. Attaching the black post 608 may include attaching the first end 644 to any one or more of the target plates 602, 604, 606. The black post 608 may be attached to the marker 600 using bolts or the like that enter the post 608 from the bottom (underside 610) of the marker 600 through 622, 630, 635. The bolts hold all of the components of the marker 600 together.

[0173] The black post 608 is mounted such that the black post 608 extends substantially perpendicular to the plane of the assembled target (eg, perpendicular to each of the surfaces 616, 626, 635).

[0174] Steps 702-706 form the assembled marker 600. In variations, steps 702-706 may be performed in any order. For example, the target pattern plate 606 may be attached to the white target plate 604, and then the assembled target pattern plate 606 and white target plate 604 may be attached to the black target plate 602. In embodiments in which the black posts 608 are not present, step 706 is not performed and the assembled marker 600 is formed by performing steps 702 and 704.

[0175] At 708, the method 700 further includes attaching the assembled marker 600 formed from steps 702-706 to a target object. In particular, the first surface 610 of the first black target plate 602 is attached to a visible surface of the target object. A visible surface in this context means a surface that can be visualized by a machine vision system configured to image the marker 600 when used in operation.

[0176] In general, the mounting position of the marker 600 is important for the correct operation of the system since the pose that is measured is that of the marker 600, not that of the object to which the marker 600 is attached. However, if the objective is to grasp or manipulate an object (rather than the marker 600), the transformation from the target reference frame to the object reference frame should be precisely determined and known. In one embodiment, the marker 600 includes two dowel pins and asymmetric mounting "ears" (visible on marker 800 in FIG. 8 and marker 1502 in FIG. 15A, FIG. 15B) to ensure that the marker 600 is mounted in exactly the right orientation. With only two dowel pins, symmetrical with a 180 degree roll, the asymmetric bolt "ears" do not allow for a 180 degree error in mounting.

[0177] Referring now to Figure 8, there is shown a marker 800 in assembled form, according to one embodiment, that is substantially similar to marker 600 and that has been assembled according to method 700 of Figure 7. Marker 800 is substantially similar to marker 600 of Figure 6, and therefore like reference numbers are used for like components.

[0178] The marker 800 also includes washers 802 and 804. The two washers 802, 804 are used with a lubricant between them. The reason for this is that while in space, the black material of the marker 800 absorbs more heat energy from the sun than the object to which the marker 800 is attached. The object to which the marker 800 is attached may be wrapped in reflective foil to prevent the sun's rays from overheating the object, but the marker 800 is not wrapped in this material to keep the marker fully exposed (for visibility). The temperature difference between the marker 800 and the object below the marker 800 (to which the marker 800 is attached) may cause a thermal expansion difference. Using these two washers with a lubricant between them can allow the marker 800 to "slide" as it expands to avoid putting compressive stresses on the marker 800 that could deform or break it if there is a large temperature difference between the marker 800 and the object to which the marker 800 is attached. Additionally, these two washers 802, 804 with a lubricant between them reduces the possibility of damage to the fastener due to thermal stress.

[0179] In variations of markers 600, 800, or any marker of the present disclosure that includes an out-of-plane white dot, the posts may have a "positive length" or a "negative length." A positive length post, such as post 608, may include any structure that elevates an out-of-plane white dot (e.g., white disk 648) above the plane of the square coding stripe (e.g., above the plane of the target pattern plate 606). In some embodiments, a positive length post may have a frustoconical shape, with the wider end of the frustocone attached to the marker and the white disk 648 attached to the narrower end. In contrast, a negative length post may include any structure that lowers an out-of-plane white dot below the plane of the square coding stripe. For example, a negative length post may take the form of a recess in the visualized side of the marker. The out-of-plane white dot (e.g., disk 648) is positioned on the bottom surface of the recess such that the white dot is below the plane of the square coding stripe. 15A and 15B show two perspective views 1500a, 1500b, respectively, of a marker 1502 with "negative length posts" according to one embodiment. Markers with positive length posts and markers with negative length posts can be geometrically and algorithmically designed using the same specifications and can be processed in the same way by the vision system software.

[0180] 9, shown therein are a number of machine vision markers 900 according to one embodiment. Each of the markers 900 is an example of the machine vision marker 800 of FIG.

[0181] Each of the plurality of markers 900 has a different (i.e., unique) data bit sequence encoded within the respective marker's square coding stripes 104. This is achieved by each marker differing only in the configuration of the black target pattern plate 606, and in particular, the configuration of the outer target pattern portion 638 of the black target pattern plate 606. Thus, each marker can be used for a different target object (or class of target objects), and each data bit sequence is linked to a target object ID within the machine vision system.

[0182] The black target pattern plate 606 of each marker 900 also includes predefined corner values ​​to enable orientation of the marker. The predefined corner values ​​are constant across all markers 900.

[0183] In this particular embodiment, the predefined corner values ​​include an upper left corner value of 0 (black square), an upper right corner value of 0 (black square), a lower left corner value of 1 (white square), and a lower right corner value of 1 (white square). The upper left corner has an upper neighbor value of 0 (black square) and a left neighbor value of 0 (white square). The upper right corner has an upper neighbor value of 0 (black square) and a right neighbor value of 1 (white square). The lower left corner has a left neighbor value of 1 (white square) and a lower neighbor value of 1 (black square). The lower right corner value has a right neighbor value of 1 (white square) and a lower neighbor value of 0 (black square). The upper side 902, right side 904, lower side 906, and left side 908 to which the foregoing refers are shown in FIG. 9. Thus, for example, the term "lower neighbor value" refers to the square coding element adjacent to the respective corner coding element on the lower side 906.

[0184] Thus, in this embodiment, the "predetermined corner value" for each corner includes not only the corner coding element value, but also the two coding element values ​​adjacent to the corner coding element, since all three coding element values ​​for that corner across each of the markers 900 (i.e., three for each of the four corners, 12 predetermined values) are predetermined and constant.

[0185] Referring now to FIG. 10, shown therein is a dual machine vision marker 1000 according to one embodiment.

[0186] The dual marker 1000 includes two markers 1002, 1004 of different sizes combined into a single marker 1000. The line 1006 in FIG. 8 that represents the outer edge of the marker 1004 (in particular, the second black target element 108 of the marker 1004) does not actually exist within the marker 1000.

[0187] Each of the markers 1002, 1004 is an example instance of the marker 100 of Figure 1. As such, features described above with respect to marker 100 may be referenced with the same reference numbers (but are not shown in Figure 10).

[0188] The smaller marker 1004 is attached to the center of the larger marker 1002 .

[0189] The markers 1002, 1004 share an out-of-plane white dot 1010. The out-of-plane white dot 1010 is located in an approximately central position on the dual marker 1000 over the smaller marker 1004.

[0190] The markers 1000 can be reliably and accurately detected over a longer range of distances than may be achievable with a single marker. For example, the larger marker 1002 may be detected from a longer distance, while at closer distances, the smaller marker 1004 is detected once the larger marker 1002 is out of view.

[0191] In one embodiment, image processing software may be used (e.g., executed by a processing device such as the processor 1114 of FIG. 11) configured to constantly search the image for all markers that may be present. If the marker is too small, the data bits cannot be resolved correctly and the marker cannot be identified. If any part of the marker is outside the field of view, the marker cannot be identified. In other words, the entire marker must be in the field of view of the camera in order to be identified. At long distances, the smaller inner marker 1004 will be too small to be resolved before the larger outer marker 1002. At close distances, the larger marker 1002 will move out of the field of view of the camera before the smaller inner marker 1004. At intermediate distances, both markers 1002, 1004 are visible. Since both markers 1002, 1004 share the same frame of reference, the pose from both markers 1002, 1004 is the same (ignoring pose measurement noise), so there is no conflict. Since the poses from both markers 1002, 1004 are available, higher level software can decide which pose to use or can fuse them. This process is transparent and poses are calculated whenever a marker is available in the image. In this way, the calculation of the pose of the dual marker 1000 can be performed without further special programming.

[0192] Reference is now made to Figures 11 and 12, which illustrate the systems and methods for detecting machine vision markers of the present disclosure as described above.

[0193] Referring first to FIG. 11, shown therein is a system 1100 for detecting, identifying, and pose determining a machine vision marker, such as the marker 100 of FIG. 1, according to one embodiment.

[0194] In space-based applications, the system 1100 can be used to determine the relative attitude of an approaching spacecraft (e.g., target object 1104) during docking by processing images of the marker 100. Other applications can include, for example, identification and attitude estimation of robotic tools, equipment, containers, and replaceable units for orbital servicing and planetary exploration, as well as vision-guided convoy operations.

[0195] The system 1100 includes a machine vision marker 1102 attached to a target object 1104 and a machine vision system 1106 for acquiring and processing images of the marker 1102. When used in space, the machine vision system 1106 may be located on a spacecraft or the like.

[0196] The target object 1104 may be any object for which identification and / or pose estimation may be desired. For example, in a space-based application, the target object 1104 may be a spacecraft such as a satellite that may be the subject of a capture and docking process. The target object 1104 may be a robotic tool or equipment. The target object 1104 may be a container or replaceable unit for orbital servicing and planetary exploration, etc. The target object 1104 may be a vehicle that is part of a visually guided convoy mission.

[0197] The machine vision system 1106 includes a camera subsystem 1108 for capturing 1110 one or more digital images of the marker 1102. The machine vision system 1106 includes at least one imaging device (e.g., a camera), a light source for illuminating the marker 1102, and image processing software for generating and outputting digital images.

[0198] In some cases, the camera subsystem 1108 includes a monochromatic light source (e.g., an LED light) placed near the camera, and the camera includes a bandpass filter that transmits the source light and attenuates any other illumination.

[0199] The machine vision system 1106 also includes a computer system 1112. During operation, the computer system 1112 is communicatively connected to the camera subsystem 1108 such that the computer system 1112, via the processor 1114, can capture 1110 an image of the marker 1102 and instruct the camera subsystem 1108 to receive digital images from the camera subsystem 1108.

[0200] The processor 1114 of the computer system 1112 is configured to detect, identify, and determine the pose of the marker 1102 using digital images of the marker 1102 captured by the camera subsystem 1108. In some embodiments, the processor 1114 may include machine learning or machine intelligence software components (programs, modules, models, etc.) for performing computing tasks such as those described above using machine learning or machine intelligence techniques.

[0201] The computer system 1112 also includes a reference database 1116. The reference database 1116 may be stored in a memory or other data storage component of the computer system 1112. The reference database 1116 stores a plurality of data bit sequences for identifying target objects. For example, each reference data bit sequence may be unique and linked to an identifier in the database. The identifier may identify the target object 1104 as a member of a target object class (i.e., what type of object the target object is) and / or as a particular instance of the target object class.

[0202] During operation, the camera subsystem 1108 is instructed by the processor 1114 to capture 1110 an image of the marker 1102 on the target object 1104 .

[0203] A digital image of the marker 1102 is output from the camera subsystem 1108 to a computer system 1112 for processing.

[0204] The processor 1114 is configured to first detect the markers 1102 in the digital image.

[0205] Once the marker 1102 is detected in the image, the processor 1114 is further configured to use image processing techniques to detect the square coding stripes 104 of the marker 1102 .

[0206] Using the detected square coding stripes 104, the processor 1114 decodes the data bit sequence from the square coding elements 114 in the square coding stripes 104. This may include, for example, identifying a start point (i.e., the first square coding element) of the detected square coding stripes 104 and proceeding in a predetermined order to determine a bit value for each square coding element 114 in the square coding stripes 104. This may include, for example, determining whether the square coding element 114 is black or white and assigning a value of 0 or 1, respectively, to a data bit in the data bit sequence. The processor 1114 may determine whether the square coding element is black or white by, for example, determining an intensity value for the square coding element 114 by sampling. The intensity sampling may be performed by any suitable intensity sampling algorithm or technique.

[0207] The process of decoding the data bit sequence performed by the processor 1114 may include the processor 1114 first determining the orientation of the marker 1102. Determining the orientation of the marker allows the appropriate data bit sequence to be decoded. For example, the determined orientation of the marker 1102 can be used by the processor 1114 to identify the first coding element 114 in the square coding stripe and thus the first data bit in the data bit sequence.

[0208] In one embodiment, the processor 1114 determines the orientation of the marker 1102 and identifies the first coding element 114 in the data bit sequence using pre-defined corner values ​​as described herein. The pre-defined corner values ​​are stored in a reference database 1116. The processor 1114 is configured to determine the correct orientation of the square coding stripe 104 using the pre-defined corner values. The identification and determination of the pre-defined corner values ​​may be performed using any suitable computer vision or image processing algorithm or technique.

[0209] In one embodiment, the image processing software executed by the processor 1114 begins by finding a white square in the image. Once a white square is found, the software uses the corners of the image and a technique called homography to determine where in the image the software should look for data bits. The location of the data bits is relative to the corners of the found white square. FIG. 14 is an example marker 1400 showing where sampling points are defined on the marker 1400 relative to the corners of the white square ring. Once the marker 1400 is projected onto the image, homography can be used to determine where in the image to sample the pixel intensity and determine if there is a 1 or 0 at that location. The four data bits on each corner can be used to encode one value, a number from 0 to 3, at each corner with redundant data encoding so that one bit flip can be tolerated at each corner and still obtain the correct corner label. Once it is determined which corner is a 0, which corner is a 1, which corner is a 2, and which corner is a 3, it can be determined and known which side of the marker 1400 is "up".

[0210] Once the data bit sequence is determined by the processor 1114, the processor 1114 is configured to use the determined bit sequence to query the reference database 1116 to determine whether the determined bit sequence matches the reference data bit sequence. If the determined data bit sequence matches the reference data bit sequence, the processor 1114 assigns an identifier (or other such data linked to the reference data bit sequence) to the target object 1104. This assigned ID determined by the processor 1114 may then be used in subsequent processing by the computer system 1112 or another computer system. If the determined data bit sequence does not match the reference data bit sequence in the reference database 1116, the processor 1114 may be configured to reject the target object 1104 as a non-target blob.

[0211] In some embodiments, the processor 1114 may be configured to determine the orientation of the marker 1102 using predefined corner values ​​to resolve the ambiguity of the roll of the square marker 1102. The reference database 1116 may store a marker identifier (marker ID) for each unique marker, where the marker identifier is obtainable or determinable from the data bit sequence (or a subset thereof). For example, the processor 1114 may be configured to look up the marker ID in the database using the data bit sequence or a subset thereof (e.g., a subset of redundant data bits in the data bit sequence, such as "middle" data bits). The marker identifier is linked to the marker dimension data in the database such that the marker identifier can be used to obtain the marker dimension data from the database. The marker dimension data includes physical dimension information about the marker. Once the marker orientation and marker dimensions are determined by the processor 1114, the image coordinates of the marker's square vertices and peg tips photogrammetrically define the pose (x, y, z, pitch, yaw, roll) of the target reference frame.

[0212] The processor 1114 may also determine the pose of the marker 1102 in the image. This may include, for example, applying image processing techniques to identify pose calculation features (i.e., regions 122a, 122b, 122c, 122d) in the marker image and use the pose calculation features to determine a pose estimate of the marker 1102. Pose estimation using the pose calculation features may be performed using any suitable pose estimation algorithm or technique.

[0213] In one embodiment, the location (in 2D) of the four corners of the white square ring (region 122) of the marker 1102 in the image is determined. The marker identifier can be determined using the decoded data bit sequence and the marker dimension data determined from a database using the marker identifier to determine the marker dimensions. The system then knows the four corners of the white square ring of the marker (in 3D). The system can store and reference camera property data that describes the properties of the camera used to capture the image of the marker, such as image center, lens warping parameters, local length, etc. To determine the position and orientation of the marker in the camera frame, a "Perspective-N-Point" algorithm is used to match the 2D corner points to the 3D corner points.

[0214] In one embodiment, the pose determination involves detecting the corners of a white square ring (region 122) in the image data of the marker using a region-based detector, such as a Maximum Stable Exterior Region ("MSER") detector configured to find pixels that belong to a bright region surrounded by a dark region. The contour is then decomposed into quadrilaterals. Sub-pixel equations of the four lines of the quadrilateral are then determined. The four line equations are then intersected to find the sub-pixel corners of the white square.

[0215] In embodiments of the marker 100 that include an out-of-plane white dot 110, the pose estimate described above may be an initial pose determination, and the processor 1114 may be further configured to refine the initial pose estimate using the out-of-plane white dot 110. The pose estimation using the out-of-plane white dot 110 may be performed using any suitable pose estimation algorithm or technique.

[0216] The pose estimation may use information about the camera calibration and the size of the markers.

[0217] In some cases, the processor 1114 may be configured to calculate an overall pose of the target object using multiple markers 1102 on the target object 1104 having known relative positions, and determine the pose of each marker using the processor 1114. This may provide pose estimation with improved accuracy.

[0218] In some embodiments, the machine vision system 1106 may not require any initialization and may use marker models (encoded IDs and dimensions) and camera calibration data (for the camera subsystem 1108) stored in a file system on the computer system 1112. Processing time may depend on factors such as image resolution, scene complexity, and computing platform.

[0219] Reliable detection and identification of the marker 1102 by the machine vision system 1106 may require that the marker 1102 be fully visible within the camera's field of view, occupy a minimum number of pixels in the image (e.g., at least 30x30 pixels in the image), and be within a specified range of orientation. The actual maximum distance may be a function of the size of the marker 1102, the field of view of the camera, and the image resolution. The minimum distance may correspond to when the marker 1102 fills the field of view of the camera. The tolerance of the marker orientation may depend on the height of the post (in embodiments where a post is present). The pose estimation accuracy of the computer system 1112 may depend on the size of the marker 1102, the height of the post, the camera resolution and field of view, and the distance.

[0220] In one embodiment, the processor 1114 operates on each image individually (i.e., no frame-to-frame tracking is performed) without relying on the results of processing the previous image. This may require additional computations, but can ensure that the machine vision system 1106 quickly recovers (for the next image) from data loss due to, for example, occlusion.

[0221] Referring now to FIG. 12, shown therein is a method 1200 for detecting, identifying, and determining the pose of a machine vision marker, such as the marker 100 of FIG. 1, according to one embodiment.

[0222] The method 1200 may be encoded in computer-executable instructions for execution by a processor, such as the processor 1114 of FIG.

[0223] The method 1200 may be performed on a digital image of the marker 1102 captured by the camera subsystem 1108 of FIG.

[0224] At 1202, the method 1200 includes detecting edge points.

[0225] Edge points are pixels that have an abrupt intensity change relative to adjacent pixels. Edge points are used to identify locations on the image between the inner white square ring and the outer black square ring. Edge points can be detected using operators such as the Sobel kernel, which typically transforms an image to enhance edges for edge detection algorithms such as those applied herein.

[0226] In another embodiment, a region-based target extraction algorithm that does not rely on edge pixels may be used based on identifying regions.

[0227] At 1204, the method 1200 includes linking the connected detected edges into a chain.

[0228] The step of linking edge pixels into chains includes finding edge pixels adjacent or close to each other with similar orientations with the aim of creating chains of edge pixels that can be divided into lines of edge pixels.

[0229] At 1206, the method 1200 includes segmenting the chain into straight segments.

[0230] At 1208, the method 1200 includes fitting the lines to segments and grouping the lines into quadrilaterals using the proximity of the line endpoints and the segment lengths.

[0231] At 1210, the method 1200 includes calculating the intersection points of the four lines of the quadrilateral from 1208.

[0232] At 1212, the method 1200 includes determining a homography matrix of the projected planar marker using the calculated intersection points from 1210.

[0233] At 1214, the method 1200 includes sampling intensity values ​​from the homography matrix to obtain intensity values ​​for the data bits.

[0234] At 1216, the method 1200 includes decoding the data bits to either identify the target object 1104 (i.e., the decoded data bit sequence matches a reference data bit sequence in the reference database 1116) or reject the quadrangle if the data bit sequence does not match a reference data bit sequence (e.g., stored in the reference database 1116).

[0235] This may include, for example, assigning a 1 or 0 to a given data bit in the sequence based on the intensity value determined in 1214, and then querying a reference database of reference data bit sequences (e.g., database 1116) to see if the determined data bit sequence matches the reference data bit sequence. Each reference data bit sequence is linked in the database to identifying information, such as a unique identifier that specifies the ID of the target object. In this way, when the reference database is queried with a determined data bit sequence that has a matching reference bit sequence, the reference data bit sequence returns a linked identifier that specifies the ID of the target object.

[0236] At 1218, the method 1200 includes calculating an initial pose of the target object using the four corners of the square (region 122 of the marker 100). In particular, the four corners used are the outer corners of a white square ring of white target elements (e.g., white target elements 106 of the marker 100) that surrounds a square coding stripe (e.g., square coding stripe 104 of the marker 100).

[0237] As previously described herein, the four corner points are known in 3D by determining the marker ID and using the marker ID to determine the dimensions of the marker, and the 2D image points onto which they project are known and determined. The "Perspective-N-Point" algorithm can then be used to determine the pose of the marker only from the corner points (by knowing the camera properties (image center, focal length, etc.)). The camera properties can be identified by internal calibration to determine focal length, image center, optical distortion profile, etc. Once this four-point pose is determined, the position of the post on the image (where the post should be) can be predicted. Then, from an initial guess as to where the post should be in the image, the exact location of the post can be determined from the data in the image. The full five-point pose is then calculated (e.g., using a similar "Perspective-N-Point" algorithm).

[0238] At 1220, the method 1200 includes refining the calculated pose using the position of the out-of-plane white dot 110.

[0239] This involves using the pose calculated in 1218 and the marker dimension information (obtained from a database using the determined marker ID) to estimate where the out-of-plane white dot 110 (e.g., a post or recessed dot) should be on the image. The software then looks for the dot 110 at that point in the image. Once the image locations of all five pose calculation features (the four corners of the marker's inner white square ring 106 on the image, and the center of the white dot 110) have been determined in 2D and 3D, and the camera parameters are known (stored in and retrievable by the computer system), the "Perspective-N-Point" algorithm calculates the refined pose. It should be noted that the center of the ellipse created by the projection of the disk may differ from the projected center of the disk, and the computer system may be configured to correct for this.

[0240] In some cases, the four corner points can be used to obtain the pose without refining using out-of-plane white dots (non-planar features), as in the planar embodiments of the markers of the present disclosure.

[0241] In some cases, the pose calculations performed at 1218, 1220 may be performed only if the data bit sequence determined at 1216 matches a reference data bit sequence in the reference database 1116 (i.e., only if the marker is not rejected as a non-target blob).

[0242] The disclosed machine vision markers, methods for assembling machine vision markers, and systems and methods for identifying and determining the pose of a target object using machine vision markers may provide certain advantages.

[0243] The machine vision markers of the present disclosure contain distinctive, precisely located target features that can facilitate unambiguous and reliable detection and accurate estimation of relative position and orientation (pose).

[0244] Markers are reliably detected under a wide range of viewing distances and angles and lighting (including direct sunlight and shadow). Encoded redundant features allow identification even with partial data loss. Marker embodiments with three-dimensional structure (e.g., the out-of-plane white dot 110 of the marker 100) may improve the accuracy of pose estimation.

[0245] A detection algorithm for detecting machine vision markers, such as may be executed by processor 1114 of FIG. 11, may be configured to operate in real time on each image individually, which may not require frame-to-frame tracking and may allow for immediate recovery from failures due, for example, to marker occlusion.

[0246] 13, shown therein is a machine vision marker 1300 according to one embodiment. The machine vision marker 1300 may be the machine vision marker 100 of FIG.

[0247] The marker 1300 includes predefined corner values ​​(or "corner" data bits) within a square coding stripe, as described herein. The "corner" data bits are OT0-OT4, OR0-OR4, OB0-OB4, OL0-OL4. These corner data bits are used to resolve the target roll ambiguity (i.e., which direction is "up"). The corner data bits accomplish this by encoding a 0, 1, 2, 3 for each one of the corners, and are encoded within a 4-bit message, allowing for one bit flip per corner and still obtaining the correct answer. Once it has been determined which direction is up for the marker 1300 using the predefined corner values, the marker ID can be determined from the "middle" data bits D0-D27 (the "middle" data bits are data bits or coding elements between the "corner" data bits that are consistent across all instances of the marker). The "middle" data bits are shown as gray in FIG. 13 to indicate that each bit can be black or white depending on the particular data bit sequence encoded within the marker 1300. A predetermined starting point for decoding the D0-D27 data bits is then identified, and the D0-D27 data bits are decoded. In one embodiment, the starting point for decoding the D0-D27 data bits is OT0 / OL4, and the data bits are read clockwise around the square coding stripe.

[0248] Referring now to Figure 16, shown therein is a method 1600 for calculating the pose of a machine vision marker, according to one embodiment. The machine vision marker can be, for example, any of the markers 100 of Figure 1, 600 of Figure 6, 800 of Figure 8, 1102 of Figure 11, or 1502 of Figures 15A-B.

[0249] The method 1600 may be performed by the computer system 1112 of Figure 11. The method 1600 may be embodied as computer-executable instructions that are stored in a memory and that, when executed by a processor, such as the processor 1114 of Figure 11, cause the computer system 1112 to perform the method 1600.

[0250] At 1602, method 1600 includes determining a target orientation of a machine vision marker. An image including the machine vision marker is captured. The image may be preprocessed (e.g., as described below in method 1700). The orientation of the machine vision marker may be determined by sampling corner bits of the machine vision marker, as described with reference to FIG. 13. For example, the corner bits may comprise a known pattern, and only a unique orientation may be determined depending on the values ​​of the corner bits of the machine vision marker in the image. As described with reference to 1602, the orientation refers to the identification of one of the square edges of the machine vision marker (e.g., the top edge of the square marker in the image is determined to be the "bottom edge" of the marker, such that the marker is upside down).

[0251] At 1604, the method 1600 includes determining a target ID of the machine vision marker. The remaining data bits (e.g., the data bits excluding the corner bits sampled to determine the orientation at 1602), or a subset thereof, may be sampled to read the data encoded by the data bits of the marker. The sampling location may be determined according to the orientation of the target determined at 1602. See, for example, FIG. 20, which shows a depiction 2002 of a target marker with data sampling locations highlighted (see locations D0-D27) and a depiction 2004 of the same target marker without the sampling locations highlighted.

[0252] In some examples, locations D0-D27 (seen in FIG. 20) comprising 28 bits are sampled to determine a 13-bit message. In such examples, each message is separated by at least 7 bits, allowing for error correction of up to 3 bit flips.

[0253] In other examples, other numbers of data bits may be allocated to encoding and determining the target ID, and the message may comprise other numbers of bits.

[0254] In some examples, the message encoded by the data bits may comprise a target ID value. Once the message encoded by the data bits is determined, the corresponding target ID is determined.

[0255] In some examples, the target ID may be referenced against a target database as described above. If the determined target ID is not present in the database, the method 1600 may stop as the image being processed by the method 1600 may be deemed not to contain a valid machine vision marker.

[0256] In some examples, the target ID may be any one or more of a marker ID, a target object ID, and / or an interface type ID. Thus, the target ID, once determined, may provide information about the marker (e.g., size, dimensions), the target object (e.g., the type of object to which the marker is attached), or the robotic interface of which the target object is a component. For example, knowing the type of object or robotic interface may enable (e.g., autonomous) identification of a robotic tool for use on the target object based on the target ID.

[0257] At 1606, the method 1600 includes calculating a target pose of the machine vision marker. The positioning of the machine vision marker may be determined. For example, the positions and / or coordinates of the corners 122 of the marker 100 may be determined. These coordinates, along with camera data associated with the camera used to capture the image of the machine vision marker, may be applied to calculate the target pose. Any suitable pose calculation method or algorithm may be applied to calculate the target pose at 1606. For example, the POSIT algorithm may be applied to calculate the target pose at 1606.

[0258] The IDs determined in 1604 may be applied to determine a target pose of the machine vision markers. For example, a system or environment may include multiple markers. The markers may vary in absolute size. The size of each marker may be known to the computer system 1112 by referencing the target ID of each marker with the marker size associated with the ID stored in a database (e.g., database 1116). The marker sizes may be applied in 1606 to perform a target pose calculation (e.g., using the POSIT algorithm, where the POSIT algorithm is a model-based iterative pose algorithm that uses scaling and orthogonal projection).

[0259] Referring now to Figure 17, shown therein is a method 1700 for calculating the pose of a machine vision marker. The machine vision marker can be, for example, any of the markers 100 of Figure 1, 600 of Figure 6, 800 of Figure 8, 1102 of Figure 11, or 1502 of Figures 15A-B.

[0260] The method 1700 may be performed by the computer system 1112 of Figure 11. The method 1700 may be embodied as computer-executable instructions that are stored in a memory and that, when executed by a processor, such as the processor 1114 of Figure 11, cause the computer system 1112 to perform the method 1700.

[0261] At 1702, the method 1700 includes performing image unwarping on an image to obtain an unwarped image.

[0262] An image of a scene in which one or more machine vision markers (e.g., marker 100) are present or expected to be present may be captured by the camera 1108 of FIG. 11. The image captured by the camera 1108 may include warping or distortion associated with the camera 1108, such as barrel distortion, pincushion distortion, and / or other forms of known distortion. At 1702, unwarping is applied to the captured image to correct for this distortion. Unwarping 1702 may include application of a distortion or warping correction map associated with the camera 1108 to the captured image. Such a map may be generated using a calibration process specific to the camera 1108.

[0263] In one example, unwarping may include calculating pixel intensities of the unwarped image by applying sub-pixel mapping in the x and y directions using bilinear interpolation.

[0264] In other examples of the method 1700, any image unwarping method or algorithm may be applied to the image to generate an unwarped image.

[0265] See Figure 18 for an example of a raw warped image 1802 and an unwarped image 1804 after completion of step 1702. The unwarped image 1804 faithfully reproduces the straight lines of the grid displayed in the image 1804 versus the warped lines of the warped image 1802 due to barrel distortion.

[0266] At 1704, the method 1700 includes performing maximum stable outer region ("MSER") blob detection on the unwarped image.

[0267] MSER is performed to detect blobs or regions characterized by boundaries with sharp boundaries and / or transitions. Such regions are characteristic of elements of a machine vision marker (e.g., marker 100), such as white target element 106.

[0268] In some examples of the method 1700, other suitable image processing methods may be applied to detect regions or blobs associated with the white target elements 106.

[0269] At 1706, blob contour tracing is performed to obtain a contour corresponding to the outside of the square ring (e.g., the outer edge of the white target element 106 of the machine vision marker 100). The contour tracing method performed at 1706 may include a vertex tracking algorithm.

[0270] For example, see Figure 19 where the vertex tracking algorithm is further illustrated. A contour may be drawn around a blob 1902. Starting at the top left most pixel of the blob 1902, a first step may be traced as shown at 1904. Steps may then continue according to the pattern outlined in table 1906 until the contour intersects with the original top left most starting pixel and the contour is completed, resulting in contour 1908.

[0271] In other examples of method 1700, other contour tracking algorithms may be applied to obtain a contour corresponding to the outside of the square ring (e.g., the outer edge of the white element 106 of the machine vision marker 100) at 1706. For example, a pixel tracking type algorithm or any other suitable contour tracking may be applied to obtain a contour corresponding to the outer edge of the white element 106 of the machine vision marker 100.

[0272] At 1708 , the method 1700 includes computing a convex hull of the blob contour obtained at 1706 .

[0273] The contour obtained in 1706 may include multiple points or coordinates. The convex hull may be calculated using, for example, a quickhull algorithm. In other examples, other algorithms may be applied to calculate the convex hull.

[0274] After completion of 1708, a number of convex hull points or coordinates are obtained that can be used for further processing.

[0275] At 1710 , the method 1700 includes determining a closest quadrilateral from the convex hull points obtained at 1708 .

[0276] In some examples, an algorithm may be applied that determines the closest quadrilateral from the convex hull points, with a single point being removed per iteration. Points are removed by combining two consecutive points into a single point. The points selected are selected as the pair that, when combined into a single point, adds the least area to the overall contour. This process may be repeated until only four points remain that define a quadrilateral. The quadrilateral is approximately aligned with the white elements 106 of the machine vision target such that the vertices that define the quadrilateral are approximately aligned with the corners 122.

[0277] In some examples, other algorithms may be applied to determine the closest quadrilateral from the convex hull points obtained in 1708.

[0278] At 1712, the method 1700 includes computing a homography of the quadrilateral obtained at 1710 to identify data bit sampling locations.

[0279] The data bit sampling locations may be defined in a coordinate system (u,v) where the origin is the upper left corner of the white element 106 and the lower right corner of the white element 106 is at coordinate (1,1). The data bit sampling locations may be addressed relative to these coordinates. See, for example, FIG. 14 where the coordinate system is shown and the sampling locations are visible.

[0280] Calculating the homography at 1712 allows the correct data bit sampling locations to be identified when the marker is skewed or misaligned with respect to the plane of the camera sensor. The homography can correct for this skew or misalignment.

[0281] At 1714, the method 1700 includes determining a target orientation of the machine vision marker by sampling the corner data bits.

[0282] A target orientation of a machine vision marker may be determined by sampling corner data bits of the machine vision marker, as described above with reference to FIG. 13, to resolve the ambiguity of the four symmetric rolls of the marker. For example, the corner bit positions may be known, as determined at 1712. The values ​​of the corner bits may be sampled, and depending on the values ​​of the corner bits of the machine vision marker in the image, a unique orientation of the marker may be determined. As described above, orientation refers to identifying one of the square edges of the machine vision marker as a particular square edge. For example, the top edge of the square marker in the image may be determined to be the true "bottom edge" of the marker, such that the marker is upside down. Such a determination may be used to determine a target orientation for sampling of the remaining data bits (e.g., at 1716).

[0283] The corner bits of the machine vision markers may be arranged in a human readable arrangement such that the corner bits may be easily interpreted by visual inspection by a human operator.

[0284] The corner bits may include 2 bits of information encoded into a 4-bit message. In some examples, the corner bits may include a Hamming distance of at least 3 between the corner patterns associated with each of the four orientations so that a 1-bit error can be corrected.

[0285] At 1716, the method 1700 includes determining the target ID by sampling data bits of the machine vision marker.

[0286] The remaining data bits (e.g., data bits excluding the corner bits sampled to determine the orientation in 1714) may be sampled to read the data encoded by the data bits of the marker. The sampling locations may be determined according to the location determined in 1712 and the target orientation determined in 1714. See, for example, FIG. 20, which shows a depiction 2002 of a target with data sampling locations highlighted (see locations D0-D27) and a depiction 2004 of the same target without the sampling locations highlighted.

[0287] In some examples, locations D0-D27 (seen in FIG. 20) comprising 28 bits are sampled to determine a 13-bit message. In such examples, each message is separated by at least 7 bits, allowing for error correction of up to 3 bit flips.

[0288] In other examples, other numbers of data bits may be allocated to encoding and determining the target ID, and the message may comprise other numbers of bits.

[0289] In some examples, the message encoded by the data bits may comprise a target ID value. Once the message encoded by the data bits is determined, the corresponding target ID is determined.

[0290] In some examples, the target ID may be referenced against a target database as described above. If the determined target ID is not present in the database, the method 1700 may stop as the image being processed by the method 1700 may be deemed not to contain a valid machine vision marker.

[0291] Since there may be no benefit in calculating the target pose if no known machine vision markers are detected, it may be advantageous to make a target ID determination before calculating the target pose (i.e., calculated in 1722). Skipping such an operation in instances where no machine vision markers are detected may reduce the processing power requirements of the system performing method 1700.

[0292] At 1718, the method 1700 includes performing a sub-pixel line fit to the machine vision marker to obtain a sub-pixel transition of a step edge (e.g., of the white element 106) of the machine vision marker.

[0293] In some examples, performing a sub-pixel line fit may include applying an optimization method to obtain sub-pixel transitions of step edges of white elements of the machine vision marker.

[0294] By performing a sub-pixel line fit at 1718, the position of the outer boundary of the marker's white element (e.g., 106) may be more accurately located. By precisely locating this boundary, more accurate pose calculations may be calculated later by method 1700. After completion of 1718, the positions of the four points of the quadrilateral may be refined and / or updated to correspond to the more accurate boundary located at 1718 through the sub-pixel line fit operation.

[0295] In some examples of method 1700, a sub-pixel line fit may be performed before 1712, 1714, and / or 1716. It may be advantageous to perform 1718 after 1716 since if a target ID that matches a known target ID is not found, then a machine vision target may not be present. Performing a sub-pixel line fit may be a waste of processing resources if such an operation is performed on an image where no machine vision target is present.

[0296] In some examples of the method 1700, a sub-pixel line fit may be performed after 1720.

[0297] At 1720, the method 1700 includes determining a position of a post tip or an out-of-plane white dot of the machine vision marker.

[0298] Based on the locations and / or positions of the four corners of the quadrilateral detected at 1710 (or refined at 1718), an estimated pose of the marker may be determined. In some examples, the estimated pose may be determined using infinitesimal plane-based pose estimation, or another suitable pose estimation algorithm.

[0299] Once the estimated pose is determined, the approximate location of the post tip may be determined and / or known. A region surrounding the approximate location of the post tip, which may include an ellipse, may be thresholded to find the center of gravity of the post tip seen in the image. After the center of gravity of the post tip is located, the location of the located center of gravity may be corrected to account for projection errors associated with out-of-plane post tips. As previously mentioned, the center of the ellipse created by the projection of the disk is not the projected center of the disk. Such corrections may apply an out-of-plane distance or dimension of the post tip that may be known (e.g., by referencing a previously determined target ID).

[0300] At 1722, the method 1700 includes calculating a pose of the machine vision marker using the points of the quadrilateral and the position of the post tip.

[0301] The pose of the machine vision marker may be calculated using the POSIT algorithm or any other suitable algorithm. These coordinates, along with camera data associated with the camera used to capture the image of the machine vision marker, may be applied to calculate the target pose. Any suitable pose calculation method or algorithm may be applied to calculate the target pose at 1722. For example, the POSIT algorithm may be applied to calculate the target pose at 1722.

[0302] The ID determined at 1716 may be applied to determine a target pose of the machine vision marker. For example, a system or environment may comprise multiple markers. The markers may vary in absolute size. The size of each marker may be known to the computer system 1112 by referencing the target ID of each marker with the marker size associated with the ID stored in a database (e.g., database 1116). The marker size may be applied at 1722 to perform a target pose calculation (e.g., using the POSIT algorithm).

[0303] After completion of 1722, the target pose is known along with the target ID. Such information can be applied to subsequent operations.

[0304] Although the above description provides examples of one or more devices, methods, or systems, it will be understood that other devices, methods, or systems may be within the scope of the claims as interpreted by one of ordinary skill in the art. [Explanation of symbols]

[0305] 100 machine vision marker, marker, vision marker, target, target marker 102 first black target element, target element, component 104 square coding stripe, coding stripe, component, data bit square ring 106 White target element, target element, component 108 second black target element, target element, component, black outer square ring 110 Out-of-plane white dot, white dot 112 Square Periphery, Periphery 114 Square target coding element, coding element, target coding element 115 Width 116 black square target coding element, black square, black square coding element, black coding element 118 White Square Target Coding Element, White Square, White Square Coding Element 120 Inner square periphery, inner circumference 122 Areas, Corners 122a Area, posture calculation features 122b Area, posture calculation features 122c Area, Attitude Calculation Features 122d region, pose calculation features 200 Markers 202 Corner coding elements, corner elements 202a top left coded element, top left corner coded element, corner coded element, top corner coded element, black corner coded element, top left corner element 202b top right corner coding element, corner, corner coding element, top corner coding element, top right coding element, black corner coding element, top right corner element 202c bottom right corner coding element, corner, corner coding element, bottom corner coding element, bottom right coding element 202d bottom left corner coding element, corner, corner coding element, bottom corner coding element, bottom left coding element 204 Starting point 206 line 208 White coding element 210 Black coding element 212 White coding element 214 Black coding element 216 Side 218 White coding element 220 black coding element 222 White coding element 224 Black coded element 226 Side 300 expressions 302 Black Ring 304 Part, Target 306 Inner Black Square, Component 308 square coding stripe, component 310 White Square Ring, Component 312 Black Square Ring, Component 314 encoding element 316 encoding element 400 1st Vision Marker, Marker 450 Second Vision Marker, Marker 452 Inner Black Square 454 White Square Ring 456 Encoding area 458 outer black square ring 460 areas 460a area 460b area 460c area 460d area 500 Vision Marker, Marker 510a distance 510b distance 510c distance 510d distance 550 Vision Marker, Marker 560a distance 560b distance 560c distance 560d distance 600 Machine Vision Marker, Marker 602 first black target plate, first black target element, target plate, black target plate 604 White target plate, White plate, White reflector plate, Target plate 606 Black target pattern plate, data bit pattern plate, target plate 608 Black Post 610 First surface, underside 612 Second surface, recess, square recess 614 Square recess, square recess 616 Non-recessed outer square ring, surface 618 Bottom 620 Side 622 Circular Opening / Aperture 624 First Surface 626 Second Surface, Surface 628 Side 630 Circular Aperture / Aperture, Circular Aperture 632 First Surface 634 Second Surface 635 Circular opening / aperture, circular opening, surface 636 Inner square part, inner square 638 Outer target pattern part 640 Outward protrusion, rectangular protrusion 642 Recess, rectangular recess 644 Rod 646 First End 648 Second end, retroreflective white disk, white disk, disk 800 markers, machine vision markers 802 Washer 804 Washer 900 Machine Vision Marker, Marker 902 Upper side 904 Right side 906 lower side 908 left side 1000 Dual Machine Vision Marker, Marker, Dual Marker 1002 Marker, outer marker 1004 Marker, inner marker 1006 line 1010 Out-of-plane white dots 1100 System 1102 Machine Vision Marker, Marker, Square Marker 1104 Target Object 1106 Machine Vision System 1108 Camera Subsystem 1112 Computer Systems 1114 Processor 1116 Reference Database 1300 Machine Vision Marker 1400 Marker 1502 Marker 1802 raw warped image, warped image 1804 Unwarped Images 1902 Blob 1906 Table 1908 Contour 2002 Description 2004 Description 2100 Marker 2100B Marker variant, non-bevel marker, marker 2104 Coded stripe, stripe 2104B Coded Stripe 2106 White element 2110 Viewpoint

Claims

1. 1. A machine vision marker for use on a target object, said marker comprising: a first black target element comprising a black square; a square coding stripe extending around a perimeter of the black square and surrounding the black square, the square coding stripe being comprised of a plurality of square coding elements which collectively encode a binary data bit sequence encoding identification information for the target object, each one of the plurality of square coding elements being either a black square or a white square representing a 0 or 1 data bit, respectively, in the binary data bit sequence; a white target element comprising a white square ring extending around the periphery of the square coding stripe and surrounding the square coding stripe; a second black target element comprising a black square ring extending around a periphery of the white square ring of the white target element; an out-of-plane white dot post located above or below the plane of the marker and centrally located on the marker; A machine vision marker comprising:

2. 2. The marker of claim 1, wherein the square coding stripe, the white square ring of the white target element, and the black square ring of the second black target element each have a width equal to the width of one target coding element.

3. 2. The marker of claim 1, further comprising a black post for elevating the out-of-plane white dot above the plane of the marker, the black post having a base at a first end for attaching the black post to at least one of the target elements and a post tip at a second end opposite the first end, at which the out-of-plane dot is located.

4. The marker of claim 3 , wherein the post tip includes a black ring around the out-of-plane white dot.

5. 10. The marker of claim 1, wherein each of the white target elements, the white square coding elements, and the out-of-plane white dots are constructed from a retroreflective material.

6. the black square coding element is integrally formed with the first black target element and disposed around the perimeter of the black square of the first black target element to form a target pattern element; the white target element is a white square, the white square ring of the white target element forms a perimeter of the white square; the target pattern element is centered on the white target element such that the white square coding elements of the square coding stripe are formed from the portions of the white squares around the periphery of the black squares of the target pattern element that are not obscured by the black square coding elements, The marker of claim 1.

7. 2. The marker of claim 1, further comprising a recess on the visualized side of the marker for lowering the out-of-plane white dot below the plane of the marker, the out-of-plane white dot being disposed within the recess.

8. 2. The marker of claim 1, wherein the square coding stripe includes redundant coding of the identification information of the target object that allows correct decoding of the target object's ID with up to three bit inversions in the binary data bit sequence.

9. 2. The marker of claim 1, wherein the square coding stripe includes four corner square coding elements, the data bit values ​​of the four corner coding elements being predetermined based on a fixed corner square coding element value used across all markers of which the marker is an instance.

10. 10. The marker of claim 9, wherein the four corner square coding elements include a top left coding element, a top right coding element, a bottom right coding element, and a bottom left coding element, wherein the top left and top right coding elements are black squares and the bottom left and bottom right coding elements are white squares, or the top left and top right coding elements are white squares and the bottom left and bottom right coding elements are black squares.

11. 10. The marker of claim 9, wherein the data bit sequence is encoded clockwise within the square coding stripe, and a first data bit of the data bit sequence is encoded in a given one of the four corner target coding elements.

12. 2. The marker of claim 1, wherein the first black target element is at least 10 square coding elements wide, and the square coding stripe, the white square ring of the white target element, and the black square ring of the second black target element each have a width equal to one square coding element.

13. 10. The marker of claim 1, wherein the first black target element is at least 10 target coding elements wide and the second black target element is at least 16 target coding elements wide.

14. 2. The marker of claim 1, wherein the out-of-plane white dot has a dot diameter equal to the length of the diagonal of one square coding element.

15. The marker of claim 6 , wherein the target pattern element, the white target element, and the second black target element are three separate pieces of material that are assembled to form the marker.

16. 16. The marker of claim 15, wherein the second black target element is a square piece of material having a square shaped recess dimensioned to receive the white target element therein and a non-recess extending around the recess forming the black square ring, and the marker is assembled by mounting the white target element within the square shaped recess of the second black target element and mounting the first black target element over the white target element.

17. The marker of claim 6 , wherein the target pattern element comprises a beveled edge.

18. A system comprising the marker of claim 1 and a machine vision system, the machine vision system comprising: an imaging device for capturing image data of the target; 1. A processing device, comprising: detecting the square coding stripes in the image data; decoding a corner data bit value for each of four corner square coding elements of the square coding stripe; determining an orientation of the marker in the image data using the corner data bit values ​​by referencing predefined corner values ​​for the marker stored in memory; decoding the data bit sequence using the determined orientation; and determining whether the decoded data bit sequence matches a reference data bit sequence stored in a reference database; a processing device configured to A system comprising:

19. 20. The system of claim 18, wherein the reference data bit sequence is linked in the reference database to target object identification data that identifies an attribute of the target object, and the processing device is further configured to use the linked target object identification data in subsequent processing operations if the decoded data bit sequence matches the reference data bit sequence.

20. The system of claim 19 , wherein the target object identification data includes a target object ID, and the processing device is configured to identify the target object using the target object ID.

21. the processing device comprising: Detecting corners of the white square in the image data of the marker for use as pose calculation features; determining a pose of the target object using the pose calculation features; and 20. The system of claim 18, further configured to:

22. 22. The system of claim 21, wherein the processing device is further configured to refine the pose of the target object using the out-of-plane white dots.

23. 1. A method of assembling a machine vision marker, the machine vision marker being for use on a target object, the method comprising: providing a square black base plate having a top surface and a bottom surface, the bottom surface being for mounting the base plate to the target object, the top surface including a square recess located at the center of the top surface and a non-recess forming a black square ring around the square recess; placing a square white plate, having a top surface and a bottom surface, within the square recess of the square black base plate such that the bottom surface contacts a bottom surface of the recess; attaching a black target pattern plate to the top surface of the square white plate, the black target pattern plate comprising an inner square and a plurality of black data bit encoding squares disposed about a periphery of the inner square, the plurality of black data bit encoding squares collectively constituting a coding area of ​​the black target pattern plate that is the width of one data bit encoding square; the attachment of the black target pattern plate to the top surface of the square white plate forms a square coding stripe around the perimeter of the inner square of the black target pattern plate, the square coding stripe including the plurality of black data bit coding squares of the black target pattern plate and a plurality of white data bit coding squares formed by square areas of the square white plate not obscured by the coding areas of the black target pattern plate; the plurality of black data bit coding squares and the plurality of white data bit coding squares of the square coding stripe collectively encode a data bit sequence representative of identification information regarding the target object. Steps and Attaching the post to at least one of the black base plate, the white plate, and the target pattern plate such that the post is centered on the target, the post comprising: a black rod having a base for attaching the post to the at least one of the black base plate, the white plate, and the target pattern plate and a tip portion opposite the base; a white dot disposed at the center of the top surface of the tip of the rod; Steps and A method comprising:

24. 1. A method of manufacturing a square machine vision marker having improved roll ambiguity characteristics for determining marker orientation in use, the method comprising: manufacturing a plurality of square machine vision markers, each having a square coding stripe including a plurality of square coding elements, each square coding element being black or white, the square coding stripe encoding a binary data bit sequence decodable by a machine vision system to obtain information regarding a target object on which the marker is displayed; wherein the square coding stripe of each of the plurality of square machine vision markers includes the same predetermined corner square coding element value such that the corner square coding element value is fixed and consistent across the plurality of square machine vision markers, and the corner square coding elements are configured such that two non-opposing corner coding elements are black squares and two non-opposing white corner coding elements are white squares. method.

25. 25. The method of claim 24, wherein the plurality of square machine vision markers includes at least a first machine vision marker having a first square coding stripe and a second machine vision marker having a second square coding stripe, the first and second square coding stripes being different.

26. 1. A method for detecting a target pose of a machine vision marker, the method comprising: determining a target orientation of the machine vision marker; determining a target ID of the machine vision marker; calculating a target pose of the machine vision marker; A method comprising:

27. The method of claim 26 , further comprising determining a target size using the target ID.

28. 27. The method of claim 26, wherein determining the target orientation comprises sampling corner data bits of the machine vision marker.

29. 27. The method of claim 26, wherein determining the target ID comprises sampling data bits of the machine vision marker.

30. 27. The method of claim 26, further comprising capturing an image including at least one machine vision marker.

31. 31. The method of claim 30, further comprising: performing an unwarping operation on the image to obtain an unwarped image; and using the unwarped image to determine the target orientation.

32. The method of claim 31 , wherein the method further comprises performing blob detection on the unwarped image prior to the step of determining the target orientation.

33. The method of claim 32 , wherein the blob detection comprises maximum stable outer region blob detection.

34. 33. The method of claim 32, further comprising performing blob contour tracing on blobs detected by the blob detection to obtain a contour corresponding to an exterior of a white element of the machine vision marker.

35. The method of claim 34 , wherein the blob contour tracing comprises a vertex following algorithm.

36. The method of claim 34 , wherein the method further comprises the step of computing a convex hull of the blob contour.

37. 37. The method of claim 36, wherein the method further comprises determining a closest quadrilateral from the convex hull.

38. 38. The method of claim 37, further comprising the step of computing a homography of the quadrilateral to determine data bit sampling locations.

39. 38. The method of claim 37, further comprising performing a sub-pixel line fit on the image to obtain a sub-pixel transition of a step edge of the white element of the machine vision marker.

40. 38. The method of claim 37, further comprising determining a position of a post tip of the machine vision marker.

41. 38. The method of claim 37, wherein calculating the target pose of the machine vision marker comprises applying the quadrilateral to a post tip location.

42. The method of claim 26 , wherein the target ID is any one or more of a marker ID, a target object ID, and an interface type ID.

Images