Systems and methods for generating an optical marker and for correcting an image of an optical marker

Abstract

Systems and methods are disclosed for generating an optical marker and for correcting an image of an optical marker, including generating a lattice model including a plurality of isothetic polygons; encoding respective values of the plurality of isothetic polygons using a two-dimensional grammar, wherein the two-dimensional grammar includes a plurality of allowed value combinations for the plurality of isothetic polygons; generating an optical marker using the plurality of isothetic polygons with the encoded values; receiving an image of the optical marker; generating an error syndrome for the image of the optical marker using the two-dimensional grammar; and generating an error-corrected image of the optical marker using the error syndrome.

Description

Attorney Docket No.: 00379-0040-00304 SYSTEMS AND METHODS FOR GENERATING AN OPTICAL MARKER AND FOR CORRECTING AN IMAGE OF AN OPTICAL MARKER CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to U.S. Provisional Application No.63 / 558,931, filed February 28, 2024, which is incorporated by reference herein in its entirety. TECHNICAL FIELD

[0002] Embodiments of this disclosure are directed to systems and methods for generating and reading an optical marker, and more particularly to noise-resistant error correction of an image of optical marker. BACKGROUND

[0003] Computer-readable optical markers are widely used to transmit information based on a captured image of the optical marker. However, due to noise from environmental conditions or image anomalies, for example, an image of an optical marker captured by an image capturing device may include one or more errors. The inclusion of an error in the image may affect the accurate transfer of the information encoded in the optical marker. Some technologies directed to correcting errors require additional data, such as parity bits, to detect errors. However, additional data lengthens the transferred information, which results in a less efficient information transfer.

[0004] The background description provided herein is for the purpose of generally presenting the context of the disclosure. Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art, or suggestions of the prior art, by inclusion in this section. SUMMARY OF THE DISCLOSURE

[0005] According to certain aspects of the disclosure, systems and methods are disclosed for generating an optical marker and for correcting an image of an optical marker.Attorney Docket No.: 00379-0040-00304

[0006] In some aspects, the techniques described herein relate to a computer- implemented method including: generating a lattice model including a plurality of isothetic polygons; encoding respective values of the plurality of isothetic polygons using a two- dimensional grammar, wherein the two-dimensional grammar includes a plurality of allowed value combinations for the plurality of isothetic polygons; generating an optical marker using the plurality of isothetic polygons with the encoded values; receiving an image of the optical marker; generating an error syndrome for the image of the optical marker using the two-dimensional grammar; and generating an error-corrected image of the optical marker using the error syndrome.

[0007] In some aspects, the techniques described herein relate to a computer- implemented method including: generating a lattice model including a plurality of isothetic polygons; encoding respective values of the plurality of isothetic polygons using a two- dimensional grammar, wherein the two-dimensional grammar includes a plurality of allowed value combinations for the plurality of isothetic polygons; and generating an optical marker using the plurality of isothetic polygons with the encoded values.

[0008] In some aspects, the techniques described herein relate to a computer- implemented method including: receiving an image of an optical marker, wherein the image includes a plurality of isothetic polygons with respective encoded values; generating an error syndrome for the image of the optical marker using a two-dimensional grammar, wherein the two-dimensional grammar includes a plurality of allowed value combinations for the plurality of isothetic polygons; and generating an error-corrected image of the optical marker using the error syndrome.

[0009] Additional objects and advantages of the disclosed embodiments will be set forth in part in the description that follows, and in part will be apparent from the description, or may be learned by practice of the disclosed embodiments. The objects and advantages of the disclosed embodiments will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims.

[0010] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments, as claimed.Attorney Docket No.: 00379-0040-00304 BRIEF DESCRIPTION OF THE FIGURES

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various examples and, together with the description, serve to explain the principles of the disclosed examples and embodiments.

[0012] Aspects of the present disclosure may be implemented in connection with embodiments illustrated in the attached drawings. These drawings show different aspects of the present disclosure and, where appropriate, reference numerals illustrating like structures, components, materials, and / or elements in different figures are labeled similarly. It is understood that various combinations of the structures, components, and / or elements, other than those specifically shown, are contemplated and are within the scope of the present disclosure.

[0013] Moreover, there are many embodiments described and illustrated herein. The present disclosure is neither limited to any single aspect or embodiment thereof, nor is it limited to any combinations and / or permutations of such aspects and / or embodiments. Moreover, each of the aspects of the present disclosure, and / or embodiments thereof, may be employed alone or in combination with one or more of the other aspects of the present disclosure and / or embodiments thereof. For the sake of brevity, certain permutations and combinations are not discussed and / or illustrated separately herein. Notably, an embodiment or implementation described herein as “exemplary” is not to be construed as preferred or advantageous, for example, over other embodiments or implementations; rather, it is intended to reflect or indicate the embodiment(s) is / are “example” embodiment(s).

[0014] FIG.1 depicts a system including an aerial vehicle and an optical marker, according to one or more embodiments.

[0015] FIG.2 depicts a flowchart of a method for generating an optical marker, capturing an image of the optical marker, and generating an error-corrected image of the optical marker, according to one or more embodiments.

[0016] FIG.3A, FIG.3B, and FIG.3C depict error correction outcomes for a generated optical marker using a variety of techniques, according to one or more embodiments.

[0017] FIG.4 depicts an example of an optical marker generated using a lattice model comprising a Cartesian grid, according to one or more embodiments.

[0018] FIG.5 depicts an example of an alphabet of a two-dimensional grammar, according to one or more embodiments.Attorney Docket No.: 00379-0040-00304

[0019] FIG.6 depicts an example of an optical marker generated using a lattice model comprising a plurality of concentric circles, according to one or more embodiments.

[0020] FIG.7 depicts an implementation of a computer system that executes techniques presented herein, according to one or more embodiments.

[0021] FIG.8 depicts a flow diagram for training a machine learning model, according to one or more embodiments.

[0022] FIG.9 depicts example two-dimensional lattice models and associated blocks, according to one or more embodiments.

[0023] FIG.10 depicts example non-regular lattice models, according to one or more embodiments.

[0024] As used herein, the terms “comprises,” “comprising,” “includes,” “including,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. The term “exemplary” is used in the sense of “example,” rather than “ideal.” In addition, the terms “first,” “second,” and the like, herein do not denote any order, quantity, or importance, but rather are used to distinguish an element or a structure from another. Moreover, the terms “a” and “an” herein do not denote a limitation of quantity, but rather denote the presence of one or more of the referenced items. DETAILED DESCRIPTION

[0025] Reference will now be made in detail to examples of the present disclosure, which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. In the discussion that follows, relative terms such as “about,” “substantially,” “approximately,” etc. are used to indicate a possible variation of a numerical range in a stated numeric value, as will be designated below.

[0026] Optical markers are widely used for coding and visual reading of information as well as for identification and measurement tasks. In aviation, fiducial markers are used for fast and reliable localization and orientation of aircraft, which is especially important for precisionAttorney Docket No.: 00379-0040-00304 landing tasks. The number of families of such markers is continuously growing, with relevance of localization and identification under challenging visual conditions.

[0027] Embodiments of this disclosure are directed to systems and methods for generating and reading an optical marker, and more particularly to lossless and noise-resistant error correction of an optical marker. One or more embodiments may provide a robust and noise- tolerant method of classifying components of a structure of optical markers. One or more embodiments may provide error detection and correction coding of markers. One or more embodiments may provide a system that is used in addition to existing methods of coding with redundancy. One or more embodiments may provide methods of segmentation, localization, and decoding of existing types of markers. One or more embodiments may increase a robustness of an image recognition system. One or more embodiments may provide a system for synthesis of noise-resistant markers, which may allow identification and / or accurate localization under conditions of high noise, poor visibility, and / or partial occlusion of a marker.

[0028] One or more embodiments may provide a system for computer vision, computational geometry, pattern recognition, error correcting coding, precision landing, or optical navigation, for example. One or more embodiments may provide an advantage of structural pattern recognition for a wide class of geometrically strictly structured objects, such as fiducial markers and QR codes, for example.

[0029] Data structures known to be ordered into lattices carry inherent redundancy and therefore allow noise-resistant coding without additional check symbols. Ordered information checks itself for conformity to a given order and, in case of inconsistency, signals an error and / or corrects errors. Two-dimensional Schlesinger context-free grammars may be used for lossless noise-tolerant coding. Because of the redundancy in the lattices, the error correction coding may be accomplished without lengthening the data with check characters. The vast majority of optical markers may be represented as isothetic polygons. The representation by isothetic polygons may be applied to a wider class of geometrically ordered structures. All such structures admit encoding by the two-dimensional grammars. Thus, optical markers may be intrinsically noise resistant when encoded using the two-dimensional grammars.

[0030] The description of FIG.1 below provides an aerial vehicle as an exemplary system for using the systems and methods described in the disclosure. However, the disclosure is not limited to using the systems and methods in an aerial vehicle. For example, the systems andAttorney Docket No.: 00379-0040-00304 methods described below may be applied to any machine-readable optical image system, such as fiducial markers or quick-response (QR) codes.

[0031] FIG.1 depicts a system 100 including an aerial vehicle 102 and an optical marker 112. System 100 may include aerial vehicle 102, controller 104, network 106, optical marker generator 108, image capturing device 110, and optical marker 112.

[0032] Aerial vehicle 102 may be a vertical takeoff and landing (VTOL) vehicle, for example, and may use one or more of a battery or fuel, for example, to generate power for flight. Aerial vehicle 102 may function to travel short and long distances to provide transportation to one or more passengers, luggage, cargo, and / or other objects. Aerial vehicle 102 may include one or more of a fuselage, front wing, rear wing, tilt-capable rotor, fixed rotor, or control surface. Aerial vehicle 102 may be configured to maneuver in a plurality (e.g., six) of degrees of freedom. For example, aerial vehicle 102 may travel about a vertical axis throughout a flight (e.g., during climbing, descending, takeoff, and / or landing). Aerial vehicle 102 may rotate about the vertical axis for a yaw movement. Aerial vehicle 102 may also move along a longitudinal axis in a generally backward or forward direction. Aerial vehicle 102 may rotate (roll) relative to the longitudinal axis. Aerial vehicle 102 may travel (e.g., translate) along a lateral axis, as well as rotate about the lateral axis to cause a pitch movement.

[0033] Although the present disclosure makes reference to an aerial vehicle, specifically a VTOL aerial vehicle, those of ordinary skill in the art will readily recognize that reference to an aerial vehicle is exemplary, and that the concepts of the present disclosure may be used in conjunction with any suitable or comparable aerial vehicle, e.g., airplanes, helicopters, aerostats, flight simulators, space crafts, commercial airplanes, or electrical vertical takeoff and landing aircrafts (eVTOL aircrafts). The above list does not, in any matter, signify a limited list of what the term “aerial vehicle” defines in terms of structure.

[0034] Aerial vehicle 102 may include a controller 104 and an image capturing device 110. Controller 104 may include one or more controllers, for example, as discussed with reference to computer system 700 of FIG.7. Controller 104 may be configured to control an operation of aerial vehicle 102, such as maneuvering in the plurality of degrees of freedom. Controller 104 may control an operation of image capturing device 110. Image capturing device 110 may be a camera, for example, which may be configured to capture an image, such as an image of optical marker 112, for example.Attorney Docket No.: 00379-0040-00304

[0035] Controller 104 may communicate with optical marker generator 108 using network 106. Network 106 may be a wireless network, for example. Optical marker generator 108 may include one or more controllers configured to generate optical marker 112 and / or to communicate with controller 104 of aerial vehicle 102. Although optical marker generator 108 is depicted as being a separate component of system 100 from aerial vehicle 102, optical marker generator 108 may be an onboard component of aerial vehicle 102.

[0036] Optical marker 112 may provide a target landing area for aerial vehicle 102, or may provide a fixed position marker to optically check an in-flight position of aerial vehicle 102 against Global Positioning System (GPS) coordinates, for example. Controller 104 may control an operation of image capturing device 110 to capture and decode an image of optical marker 112, and may control an operation of aerial vehicle 102 based on the captured image. For example, during a landing operation of aerial vehicle 102, controller 104 may control an operation of image capturing device 110 to periodically capture an image of optical marker 112, and, based on the image, control an operation of one or more of the tilt-capable rotor, fixed rotor, or control surface to adjust a position of aerial vehicle 102 relative to the optical marker 112. Controller 104 computes and adjusts the position of image capturing device 110 in relation to a known position of optical marker 112. For example, where optical marker 112 is an indicator of a vertiport, the geographic coordinates of the vertiport and the optical marker 112 may be transmitted to and / or stored in a memory of controller 104. Controller 104 then computes and adjusts the position of image capturing device 110 in relation to the position of optical marker 112 by accounting for all six degrees of freedom of the image capturing device 110 and aerial vehicle 102, the six degrees of freedom including the degrees of freedom in the x-, y-, and z-axes and three angular degrees of freedom. Optical marker 112 may provide information to aerial vehicle 102 as encoded values in optical marker 112.

[0037] Therefore, the decoded image of optical marker 112 should be error-free. However, due to noise from environmental conditions or image anomalies, such as, for example, motion blur, and / or defocusing blur, and / or image capture noise in low light conditions, generally in low signal to noise ratio of the capturing image, a decoded image of optical marker 112 captured by image capturing device 110 may include one or more errors. The inclusion of an error in the image decoding may affect the operation of controller 104, which may affect an operation of aerial vehicle 102. Accordingly, optical marker generator 108 may generate anAttorney Docket No.: 00379-0040-00304 image to be used as optical marker 112 that may reduce susceptibility to noise relative to some systems. Controller 104 may correct errors in the image of optical marker 112 captured by image capturing device 110.

[0038] The description of FIG.1 above is provided as an exemplary system for using the systems and methods described in the disclosure. However, the disclosure is not limited to using the systems and methods in an aerial vehicle. For example, the systems and methods described below may be applied to any machine-readable optical image system, such as fiducial markers or quick-response (QR) codes, for example.

[0039] FIG.2 depicts a flowchart of a method 200 for generating an optical marker, capturing an image of the optical marker, and generating an error-corrected image of the optical marker by the system 100. Method 200 may include operation 210, operation 220, operation 230, operation 240, operation 250, and operation 260.

[0040] Operation 210 includes generating, at the optical marker generator 108, a lattice model that includes a plurality of isothetic polygons. Isothetic polygons are polygons with alternate sides that belong to two or more parametric families of straight or curved lines that are pencils of lines with centers at two points. In one example, the isothetic polygons may be described by polygons defined by a Cartesian grid, such that each polygon is a square or rectangle with equal dimensions.

[0041] The optical marker depicted in FIG.4 is one such example of a lattice model that includes a plurality of isothetic polygons defined by a Cartesian grid. The isothetic polygons may be described by straight and curved lines. As defined herein, an isothetic polygon is not limited to polygons with strictly straight sides and may include shapes having curved edges. One such example is depicted in the lattice model described in FIG.6, where generating the lattice model includes generating: (i) a plurality of concentric circles, and (ii) a plurality of lines through a center of the plurality of concentric circles to produce isothetic polygons that include two straight lines opposite to each other and two arcs opposite to each other, with the exception of the centermost polygons that share the center of the lattice as a vertex. The centermost polygons define wedges including of a circular arc and two straight sides.

[0042] Lattice models may be defined by tiling triangles with a grid with three families of lines, or hexagons, or a combination of grids including straight and curved features. Lattice models may include curvilinear polygons such as those defined by “barrel” radial distortion orAttorney Docket No.: 00379-0040-00304 “pincushion” radial distortion. Isothetic polygons are generally not limited to two-dimensional grids and may further be generalized for additional dimensions.

[0043] For example, a three-dimensional Cartesian grid may be comprised of bricks in the form of rectangular prisms having a length, width, and height. The bricks may be the “cells” of a grid, such that the lattice model is generated by families of isothetic planes as opposed to the lines described with reference to two-dimensional grids. Here, the lattice model may form isothetic polyhedra rather than the two-dimensional isothetic polygons. For higher dimensions, the lattice model may be generated by families of isothetic hyperplanes forming isothetic hyperpolyhedra. In an example, when an optical marker is provided in an RGB space, the lattice model may include three separate planes for each color component.

[0044] The lattice model may include defined dimensions, and may include higher dimensions. For example, the lattice model depicted in FIG.4 includes 144 squares arranged in a 12 by 12 Cartesian grid. The lattice models may include more or less polygons, and the sizes of the polygons may be changed as well. The lattice model depicted in FIG.6 includes 60 curvilinear polygons defined by six concentric circles and six lines intersecting at the center of the concentric circles. However, by increasing or decreasing the number of concentric circles, increasing or decreasing the number of intersecting lines, or increasing or decreasing the sizes of the concentric circles, the lattice model may be further changed.

[0045] Operation 220 includes encoding respective values of the plurality of isothetic polygons using a two-dimensional grammar. The two-dimensional grammar includes a plurality of allowed value combinations for the plurality of isothetic polygons. In the context of the optical markers discussed herein, the two-dimensional grammar may include allowed combinations of encoded isothetic polygons, and forbidden combinations of encoded isothetic polygons, such that when a captured image of the optical marker includes a forbidden combination, controller 104 may detect an error based on the forbidden combination.

[0046] A two-dimensional grammar is generally a spatial arrangement of symbols that represent concepts. In some examples, such as for a regular Cartesian grid, this may be accomplished according to an “alphabet” of 2x2 blocks of isothetic polygons. The blocks of higher dimensions on Cartesian lattices may also be encoded, but this increases the size of error syndromes, yet does not increase practical error correction abilities.Attorney Docket No.: 00379-0040-00304

[0047] For example, for a Cartesian grid, the following two-dimensional context-free grammar ^^^^^^^^2^^^^^^^^may be used, where ^^^^2^^^^^^^^ stands for binary two-dimensional Cartesian. ^^^^^^^^2^^^^^^^^ ≡ 〈^^^^,^^^^,^^^^ℎ ^^^^^^^^ ,^^^^^^^^, ^^^^0〉:(1) ^^^^ = {0, 1}, |^^^^| = 2

[0048] The two-dimensional generator applied to an ^^^^×^^^^ image may be used in both horizontal and vertical directions: é1 0 0 0 00 1 0 0 0ù ê0 1 0⋯0 0ú ^^^^^^^^^^^^ℎ,^^^^≡ ê0 0 1 0 0ú : ê⋮ ⋱ ⋮ú ê0 0 0 0 1ú �,Attorney Docket No.: 00379-0040-00304 ^^^^^^^^ 0,0 ⋯ ^^^^0,^^^^−1 ^^^^0,0 ⋯ ^^^^^^^^−1,0^^^^ ^^^^^^^^^^^^^^^^ ×�^^^^^^^^^^^^ℎ ×� ⋮ ⋱ ⋮=� ⋮ ⋱ ⋮�^^^^0,0 ⋯ ^^^^0,^^^^−1�⋮ ⋱ ⋮� , which is projected to the applying of horizontal and verticalgenerators. A single application of any the original sequence twice in each direction. Therefore ^^^^^ℎ^^^,^^^^∈ {

[0000] ,

[0001] ,

[0010] ,

[0011] },^^^^ ∈ {0,…,^^^^−1},^^^^∈{0,…,^^^^−1}, with final terminal symbols ^^^^^^^^,^^^^ ∈ {00|00,00|01,00|11,…11|11},^^^^ ∈ {0,…,^^^^−1}, ^^^^ ∈ {0,…,^^^^−1}.

[0050] Similar two-dimensional grammars for triangular isotheticpolygons, i.e., the polygons formed with 3, not 2, sets of lines, where |^^^^|=64, and for hexagonal isothetic polygons, where |^^^^|=8. A generalized higher-dimension grammar for multidimensional data ordered in a ^^^^- dimension Cartesian grid will result in |^^^^|= 2k+1.

[0051] The two-dimensional grammar governs, via allowed and forbidden values, the encoding of the values of the isothetic polygons of the lattice model. In some examples, the encoded values may be binary, such that each isothetic polygon in the lattice is either encoded with “0” or “1,” or “white” or “black.” The binary values may also be the absence or presence of a given pattern, such as a circle or triangle within a cell or tile defined by an isothetic polygon. Two detectors may be used in tandem for two binary encoding systems for added complexity. For example, a first detector may determine if a cell is black or white (or any other binary color scheme), and a second detector may determine if a cell includes a circle or does not (or any other pattern).

[0052] As depicted in FIG.5 and discussed above, an alphabet of 2x2 blocks encoded in binary may have sixteen distinct arrangements, labeled [1] –

[0016] . In some examples, the two- dimensional grammar governs the combinations of 2x2 blocks to which any given polygon within the lattice model may belong. This will be described in more detail with regard to the examples depicted in FIGS.4-6.Attorney Docket No.: 00379-0040-00304

[0053] At operation 230, the optical marker generator 108 generates an optical marker 112 using the plurality of isothetic polygons with the encoded values. For example, the optical marker generator 108 may include a printing device to generate a physical optical marker that can be placed such that an image capturing device 110 may capture an image of optical marker 112. For example, optical marker generator 108 may generate a printed image of optical marker 112 such that, during a landing operation of aerial vehicle 102, optical marker 112 is appropriately sized and positioned such that controller 104 may control an operation of image capturing device 110 to periodically capture an image of optical marker 112, and, based on the image, control an operation of one or more of the tilt-capable rotor, fixed rotor, or control surface to adjust a position of aerial vehicle 102 relative to the optical marker 112 to control the landing operation of aerial vehicle 102.

[0054] At operation 240, image capturing device 110 receives an image of the optical marker 112. Image capturing device 110 may be a camera, for example, which may be configured to capture an image. In some instances, the captured image of the optical marker may include a value, among the encoded values, that has been changed by noise. The noise may be caused, for example, by environmental conditions, such as fog, darkness, smoke, etc., or image anomalies, such as interference or printing errors, etc. Optical marker 112 may provide information to a system, such as aerial vehicle 102 and thus, the image of optical marker 112 should be error-free.

[0055] At operation 250, controller 104 may generate an error syndrome for the image of the optical marker using the two-dimensional grammar. In the example system 100 depicted in FIG.1, the controller 104 may receive the captured image from image capturing device 110 and generate the error syndrome based on the captured image of optical marker 112. Generating the error syndrome includes determining the value or values that have been changed by the noise, if any. Controller 104 receives the captured image and compares the captured image to the allowed and forbidden values in the two-dimensional grammar.

[0056] To generate an error syndrome, controller 104 may compare values of the tiles in the captured image in a pairwise manner and determine whether the values are equal or not. In the example of a regular Cartesian grid, in groups of cells of size 2×2, controller 104 compares the coincidence of neighboring isothetic polygons vertically and horizontally and generates an error syndrome of size 16×16. In at least one example, out of the 256 values, only 16 are allowedAttorney Docket No.: 00379-0040-00304 and the rest are forbidden. Upon detection of a forbidden combination, controller 104 detects an error. Generating the error syndrome may further include using a machine learning model (e.g., see FIG.8), such as a convolutional neural network (CNN), to detect a forbidden combination. A machine learning model may receive as an input an image covering a specific block that contains N tiles as input, where N is an integer. The machine learning model may be described as an N- ary operator, such that given a whole block of N tiles, the model outputs one of the allowed tile values. The machine learning model may be trained on blocks in a manner described with respect to FIG.8, with symmetric training cases generated, as all symmetry groups are also known for each lattice.

[0057] At operation 260, controller 104 generates an error-corrected image of the optical marker using the error syndrome. Each isothetic polygon in the optical marker is included in a plurality of blocks of isothetic polygons. For example, in an optical marker generated based on a lattice model based on a Cartesian grid, such as that depicted and described in FIG.4 below, each isothetic polygon belongs to four 2x2 quadruple blocks. According to the two-dimensional grammar, only defined combinations of blocks are allowed in this arrangement. If the four 2x2 blocks are part of an allowed combination and the value for the tile in all four overlapping blocks is determined to be equal, the algorithm determines that the tile is recognized correctly and does not determine an error at that tile.

[0058] In more detail, neighboring lattice blocks, such as blocks 408A-408D as described in FIG.4, have common tiles. In other words, the blocks overlap. More specifically, each block on the Cartesian lattice model described in FIG.4 has two closest horizontal neighbors: on the left and on the right of the block; and two closest vertical neighbors: above and below the block. Closest blocks have two overlapping tiles.

[0059] For toroidal boundary conditions, i.e., if the left and right column of the decoded marker are the same, and the top, and the same for bottom and top rows of the marker. In terms of error decoding capabilities, it is preferable to generate optical markers satisfying these boundary conditions. This may be analogized to chain codes in a single dimension example. Each tile in this Cartesian example belongs to four blocks based on the four vertices associated with each tile. Each vertex forms the center of a block, and each tile thus belongs to four blocks.

[0060] In an example where a tile has been decoded by controller 104 as a one in one block the tile belongs to, but decoded by controller 104 as a zero in three other blocks that theAttorney Docket No.: 00379-0040-00304 tile belongs to, there is an indication of an error. To correct the error, a two-dimensional grammar is applied. In the example shown in FIG.4, two-dimensional grammar ^^^^^^^^2^^^^^^^^is applied, where ^^^^2^^^^^^^^ stands for binary two-dimensional Cartesian.

[0061] ^^^^ ℎ ^^^^^^^2^^^^^^^^ ≡ 〈^^^^,^^^^,^^^^^^^^ ,^^^^ ^^^^^, ^^^^0〉:

[0062] ^^^^ = {0, 1}, |^^^^| = 2

[0063] ^^^^0 = 0

[0064] ^^^^ =�0000, 0001, 0011, 0100, … , 1011, 1111�, |^^^^| = 16binary image ^^^^:^^^^0,0 ⋯ ^^^^0,^^^^−1^^^^�the grammar constitutes mapping theimage ^^^^ to the 16-bit image ^^^^: 10 0 0 0é ùAttorney Docket No.: 00379-0040-00304

[0073] A simplified version of this determination may be demonstrated on a 2x2 block, the smallest block that generates a meaningful error syndrome in the Cartesian example. ^^^^

[0074] ^^^^ 0,0^^^^0,12^^^^2 ^^^^^^^^^^^^^^^^^^^^ ≡�^^^^ � 1,0^^^^1,1four pixels do not overlap other blocks such that there mayplanar finite basis.

[0076] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^_^^^^^^^^^^^^^^^^^^^^^^^^2^^^^2 ^^^^^^^^^^^^^^^^^^^^ ≡�^^^^0,0�, ^^^^0,0 ∈ ^^^^operation, it is assumed that the first imagegrammar defined above, one modification must be done to reflect this: é0 → 0|00 → 0|1^^^^ ^^^^ 1(4), ^^^^ = 2; thus, the first column may takeany value in ^^^^, and is duplicated in the second column. Therefore,

[0080] ^^^^ℎ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^_^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^0000 11 112^^^^2 ^^^^^^^^^^^^^^^^^^^^ ≡�^^^^0,0�, ^^^^0,0 ∈�00,11,00,11�the 16 possible values are allowed, and the rest are forbidden. Thus, if anything other than the four values are detected and even verified at the first step, an error is identified.

[0082] Similarly, a horizontal cylindrical operation may be applied, in addition to or in lieu of the vertical cylindrical operation. In this case, it is assumed that the first image row is equal to the last image row. 0 0 éì →0 1Attorney Docket No.: 00379-0040-00304

[0084] In the case of the simplest 2 × 2 image (4), ^^^^ = 2; thus, the first row may takeany value in ^^^^, and is duplicated in the second column. Therefore,

[0085] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^_^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ≡�^^^^ �,0001 10 112^^^^2 ^^^^^^^^^^^^^^^^^^^^ 0,0 ^^^^0,0 ∈�00,01,10,11�

[0086] A toroidal operation may also be applied, in addition to or in lieu of the vertical cylindrical operation and horizontal cylindrical operation, as described below. In this case, which is a combination of the vertical and horizontal cylindrical operations described above:

[0087] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^00112^^^^2 ^^^^^^^^^^^^^^^^^^^^ ≡�^^^^0,0�, ^^^^0,0 ∈�00,11�

[0088] The number of boundary conditions is not limited with the three operations described above. Other operations include a Moebius strip operation, or an extension of the size of the block to get either fixed of reflecting outer borders. Generally, if, given ^^^^2^^^^2 ^^^^^^^^^^^^^^^^^^^^and boundary condition ^^^^^^^^, any output other than ^^^^2^^^^^^^^^^^^2^^^^^^^^^^^^^^^^^^^^is regarded as an error. To correct the error, Hamming distances may be applied to determine the shortest allowed block ^^^^0,0in terms of Hamming distance. If the solution is ambiguous using Hamming distances, probabilistic methods may be employed such as Markov chains.

[0089] Similarly, in the general case: ^^^^0,0 ⋯ ^^^^^^^^−1,0^^^^ ⋮ ⋱ ⋮�,the vertices. In the case of 2 × 2 blocks,vertex belongs to 4 blocks. Where the blocks have been computed and self-verified independently, the same symbol ^^^^^^^^,^^^^has been independently computed at each tile in the blocks ^^^^ =^^^^^^^^−1,^^^^−1^^^^^^^^−1,^^^^^^^^^^^^−1,^^^^^^^^^^^^−1,^^^^+1^^^^^^^^,^^^^^^^^^,^^^^^^^^^^^^,^^^^^^^^^^^^ ^^^^, ^^^^ =^^^^ ^^^^, ^^^^^^^−1 ^^^^,^^^^+1^^^^−1,^^^^−1 ^^^^−1,^^^^ ^^^^,^^^^−1=^^^^ ^^^^, ^^^^^^^^,^^^^=^^^^.there is no contradiction and no identified error. Otherwise, ^^^^^^^^,^^^^must be corrected such that a minimum number of the blocks will be affected by the change.

[0093] Thus, overlapping blocks are checked for self-compatibility and mutual compatibility, and corrections are made accordingly to the values that may have been changed by noise. The method depicted in FIG.2 may provide an improvement in error correction over some error correction techniques for optical markers.Attorney Docket No.: 00379-0040-00304

[0094] FIG.3A, FIG.3B, and FIG.3C depict error correction outcomes for a generated optical marker using a variety of techniques. An optical marker may be generated using a lattice model comprising a Cartesian grid, as depicted in FIG.2 and FIG.4, for example. Such an optical marker may be, for example, a black-and-white quick-response (“QR”) code.

[0095] As depicted in graph 300 in FIG.3A, in the absence of noise, the distribution of a black and white image is a first delta function 302 and a second delta function 304 on opposite sides of a threshold line 306, where first delta function 302 represents all of the black cells in the optical marker and second delta function 304 represents all of the white cells in the optical marker. The captured image can therefore be easily processed using a unary operation of automatic thresholding, such as a Naïve-Bayes classifier, based on the threshold line 306, where a unary operation is an operation that maps a polygon with all optically detected pixel values to a single binary value.

[0096] Graph 310 in FIG.3B is an example of an error correction outcome using a unary operation when noise is intentionally added to the image of the optical marker. Zero-mean Gaussian noise, a standard type of noise used for modeling sensor noise, electronic circuit noise etc., of intensity ^^^^ / ^^^^=0.3 is added to the optical marker. The Naïve-Bayes classifier, using automatic thresholding for each polygon in the optical marker, produces a generally bimodal curve with a first peak 312 and a second peak 314 on either side of thresholding line 316. However, the amount of overlap in the distribution is significant, and may result in imperfect error correction.

[0097] Graph 320 in FIG.3C is an example of an error correction outcome using a binary operation for the same amount of noise, ^^^^ / ^^^^=0.3, applied to the optical marker. A binary operation, such as a Wasserstein difference classifier, improves on the outcome of the unary operator, producing more distinct first peak 322 and second peak 324 on either side of boundary 326. A binary operator, as opposed to the thresholding of a unary operator, compares polygons in the optical marker pairwise to determine which polygons are similar and which are dissimilar, and groups polygons in an optical marker together based on perceived similarity or dissimilarity. While this method provides an improvement over the unary operation depicted in graph 310, the error correction may be further improved.

[0098] The method described in FIG.2 using a two-dimensional grammar to create an error syndrome may improve recognition of errors for this exemplary noise intensity. In someAttorney Docket No.: 00379-0040-00304 examples, generating the error syndrome may include using a machine learning model (e.g., see FIG.8), such as a convolutional neural network, to detect forbidden combinations and to assign weights and probabilities to the specific errors identified. Examples of the method described in FIG.2 will now be described in more detail with respect to specific types of lattice models and two-dimensional grammars.

[0099] FIG.4 depicts an example of a Cartesian grid optical marker 400 generated from a lattice model comprising a Cartesian grid. The Cartesian grid optical marker 400 depicted in FIG.4 is based on a 12 by 12 lattice model defined by a Cartesian grid with a first length in a first horizontal direction and a second length in a second vertical direction perpendicular to the first direction. The lattice model includes 144 tiles, such as tile 402 and tile 404. Each of the 144 tiles is encoded with a respective value based on a two-dimensional grammar (e.g., see FIG.5). Tile 402, for example, is encoded to be white, and tile 404, for example, is encoded to be black. Some of the isothetic polygons, such as tile 406, will be part of overlapping blocks of isothetic polygons, such as overlapping and adjacent blocks 408A-408D. The blocks 408A-408D may be subject to the rules of a two-dimensional grammar, which includes an alphabet 500 (see FIG.5) of blocks with allowed and forbidden combinations.

[0100] FIG.5 depicts an example of an alphabet 500 of a two-dimensional grammar which may be applied to the examples of FIG.4 and FIG.6. In this example, the alphabet comprises all quadruples (i.e. group of four tiles read in order from left to right, and top to bottom) possible based on binary coding of white as “0” and black as “1.” Each of the sixteen quadruples may be mapped to a single value, as follows:

[0101] [ 00 | 01 ] = 1

[0102] [ 00 | 10 ] = 2

[0103] [ 10 | 11 ] = 3

[0104] [ 01 | 11 ] = 4

[0105] [ 01 | 00 ] = 5

[0106] [ 10 | 00 ] = 6

[0107] [ 11 | 10 ] = 7

[0108] [ 11 | 01 ] = 8

[0109] [ 10 | 01 ] = 9

[0110] [ 01 | 10 ] = 10Attorney Docket No.: 00379-0040-00304

[0111] [ 00 | 11 ] = 11

[0112] [ 11 | 00 ] = 12

[0113] [ 01 | 01 ] = 13

[0114] [ 10 | 10 ] = 14

[0115] [ 11 | 11 ] = 15

[0116] [ 00 | 00 ] = 16

[0117] As depicted in FIG.4, blocks 408A-408D will be described as an example of how the two-dimensional grammar is applied to determine if there is an error at any of the tiles within the blocks, such as tile 406. Block 408A defined by tile 406 at a lower right corner comprises the quadruple [ 00 | 10 ], corresponding to quadruple 2 in the alphabet 500 described in FIG.5. Block 408B defined by tile 406 at a lower left corner comprises the quadruple [ 00 | 0 0 ], corresponding to quadruple 16 in the alphabet 500. Block 408C defined by tile 406 at an upper right corner comprises the quadruple [ 10 | 10 ], or quadruple 14. Finally, block 408D defined by tile 406 at an upper left corner comprises the quadruple [ 00 | 00 ] as well, or quadruple 16.

[0118] As such, tile 406 is included in the overlapping blocks 408A-408D. The value of tile 406 is determined by processing the part of the image mapped to the tile. One mode of processing is using a thresholding operator, which is a unary operator for mapping the mean value of the image to a discrete scale to get a quantized output. In the binary case shown in FIG. 4, the value of tile 406 is determined to be 1 if the mean of all pixel values is above a given threshold, and to be 0 otherwise.

[0119] A binary operator may also be introduced that takes two operands (V1, V2) such that, in the case of the binary alphabet, two tile values are determined to be the same or different from each other. Such an operator outputs 0 for the pair of tuples (1,1) and (0,0), and 1 for the pair of tuples (1, 0) and (1, 0). Unary operators and binary operators may be computed independently.

[0120] As such, for two tiles belonging to the same block, such as tile 406 and tile 410 in block 408A, the unary operators will output one for both of the tiles and the binary operator will output zero, i.e., the binary operator should indicate that the tiles have the same value. In this case, there is no contradiction: the values 1 and 1 are the same. However, if the unary operators output one for both of the tiles, but the binary operator outputs one rather than zero, there is aAttorney Docket No.: 00379-0040-00304 contradiction. Equal values 1 and 1 are considered to be different. Therefore, a block that contains such a contradiction is determined to be forbidden. In analogous terms of a spellchecker, there is no such "word" in the "vocabulary".

[0121] The dimension of the output set of unary operations is equal to |K| - to the dimension of the alphabet of the formal language. For a Cartesian system as shown in FIG.4, this corresponds to 16, as shown in FIG.5. Similarly, a binary operation on that alphabet is either a difference between two values or as a modulo of this difference. Binary operators for tiles has an additional value of |K|=2.

[0122] Four unary operators over the block binary components generate 16 values:24 = 16. Given the same four-block components, there are up to 64 comparisons between allpossible tuples in the form of binary operators, given by ^^^^(^^^^, 2) = 4! ^^^^(^^^^,2) 6(4−2)!2! = 6; 2 = 2 =64. Generally, for a given regular lattice, with a given dimension, a finite alphabet ^^^^ ={0, … , ^^^^ − 1} with cardinality |^^^^| = ^^^^, ^^^^ ∈ ℕ+, ^^^^ < +∞, and with a given structure, i.e., number oflines, planes, hyperplanes, etc. for a block containing m tiles, there are up to m unary operationswhich will map the block to ^^^^^^^^ discrete values, and up to ^^^^(^^^^, 2) binary operations, which mapthe block to ^^^^^^^^(^^^^,2)discrete values. Therefore, both these operations, being applied together but the block to a matrix of dimensions�^^^^^^^^(^^^^,2), ^^^^^^^^�. This matrix is the errorsyndrome.

[0123] The error syndrome for the 2x2 block on the Cartesian lattice may be represented by a matrix as large as 16x64. This is the maximum dimension accounting for all possible binary operators. Only 16 of the cells in the matrix correspond to the allowed "words" of the alphabet; all other ones are forbidden. Thus, for tile 406, a unary operation is performed and the tile is compared pairwise to each of the tiles directly adjacent to the tile horizontally and vertically. Subsequently, a binary operation is used to compare the tile to the adjacent tiles, and any contradiction will result in the error syndrome indicating whether there is or is not a contradiction at tile 406 based on the alphabet. This process may be repeated for each tile in the Cartesian grid optical marker 400 to perform a “spell check” on a captured image of an optical marker and create an error syndrome of identified potential errors for all of the tiles. Where there is a single error identified, error correction comprises switching the value of the value changed by noise. However, if two or more errors are identified, the error syndrome may be used toAttorney Docket No.: 00379-0040-00304 identify the appropriate error correction to be performed by taking into account the probabilities of falling out of one of the two allowed symbols equidistant from the forbidden one; i.e., using distances with weights proportional to the corresponding probabilities.

[0124] If a forbidden combination is determined for the optical marker, the combination may be translated into a permitted combination based on a minimum Hamming distance, which is a probabilistic method that corrects errors using a minimum number of corrections. If there are two or more possible corrections, a weighted Hamming distance with weights reflecting different probabilities of different errors is used to eliminate ambiguities. Allowed combinations have a zero Hamming distance to themselves. In the case where it is not possible to determine a priori which error is more probable, other methods may be applied, such as applying a Markov chain until the result collapses into a stable, resolved state. The controller 104 may apply a machine learning model (e.g., see FIG.8), such as a convolutional neural network, to detect the errors and probabilities required for error correction.

[0125] Additionally or alternatively, tiles may be compared pairwise to adjacent tiles that share a border. For example, tile 410 may be compared to adjacent tiles 412A, 412B, 412C, and 412D. If the value of tile 410 has been changed by noise, the value may be compared to adjacent tiles 412A, 412B, 412C, and 412D unchanged by noise to determine how alike or different the changed value of tile 410 is to the adjacent isothetic polygons of known values. This may result in a probabilistic estimate of the corrected value for tile 410.

[0126] FIG.6 depicts an example of a curvilinear optical marker 600 generated from a lattice model comprising a plurality of concentric circles and radiant lines. The curvilinear optical marker 600 is based on a lattice model defined by a plurality of concentric circles and a plurality of radiant lines through a center of the plurality of concentric circles to produce isothetic polygons that include two straight lines opposite each other and two arcs opposite each other, with the exception of the centermost polygons that share the center of the lattice. The centermost polygons define wedges including a circular arc and two straight sides.

[0127] The example curvilinear optical marker 600 includes 60 tiles, such as tile 602 and tile 604. Similarly to the Cartesian grid optical marker 400 in FIG.4, each of the 60 isothetic polygons is encoded with a respective value based on a two-dimensional grammar. Tile 602, for example, is encoded to be white, and tile 604, for example, is encoded to be black. Some of the tiles, such as tile 606, will be part of overlapping blocks, such as overlapping and adjacent blocksAttorney Docket No.: 00379-0040-00304 608A-608D. The blocks 608A-608D may be subject to the rules of a two-dimensional grammar, which includes an alphabet 500 (see FIG.5) of blocks with allowed and forbidden combinations.

[0128] Despite the difference in shape, the grammar as applied to the Cartesian grid optical marker 400 also applies to the curvilinear optical marker 600. As depicted in FIG.6, blocks 608A-608D will be described as an example of how the two-dimensional grammar is applied to determine if there is an error at any of the tiles within the blocks, such as tile 606. The value of tile 606 is determined by processing the part of the image mapped to the tile. One mode of processing is using a thresholding operator, which is a unary operator for mapping the mean value of the image to a discrete scale to get a quantized output. In the binary case shown in FIG. 6, the value of tile 606 is determined to be 1 if the mean of all pixel values is above a given threshold, and to be 0 otherwise.

[0129] A binary operator may also be introduced that takes two operands (V1, V2) such that, in the case of the binary alphabet, two tile values are determined to be the same or different from each other. Such an operator outputs 0 for the pair of tuples (1,1) and (0,0), and 1 for the pair of tuples (1, 0) and (1, 0). Unary operators and binary operators may be computed independently.

[0130] As such, for two tiles belonging to the same block, such as tile 406 and tile 410 in block 408A, the unary operators will output one for both of the tiles and the binary operator will output zero, i.e., the binary operator should indicate that the tiles have the same value. In this case, there is no contradiction: the values 1 and 1 are the same. However, if the unary operators output one for both of the tiles, but the binary operator outputs one rather than zero, there is a contradiction. Equal values 1 and 1 are considered to be different. Therefore, a block that contains such a contradiction is determined to be forbidden. In terms of an analogous spellchecker, there is no such "word" in the "vocabulary".

[0131] The dimension of the output set of unary operations is equal to |K| - to the dimension of the alphabet of the formal language. For a Cartesian system as shown in FIG.4, this corresponds to 16, as shown in FIG.5. Similarly, a binary operation on that alphabet is either a difference between two values or as a modulo of this difference. Binary operators for tiles has an additional value of |K|=2.

[0132] Four unary operators over the block binary components generates 16 values: 2^4=16. Given the same four-block components, there are up to 64 comparisons between allAttorney Docket No.: 00379-0040-00304 possible tuples in the form of binary operators, given by C (4,2) = 4! / (4-2)!2!=6; 2^C(4,2) =2^6=64. Generally, for a given regular lattice, with a given dimension, a finite alphabet K={0,…,l-1} with cardinality |K|=l, l ∈ N_+,l<+∞, and with a given structure, i.e., number of lines, planes, hyperplanes, etc. for a block containing m tiles, there are up to m unary operations which will map the block to l^m discrete values, and up to to C (m,2) binary operations, which map the block to l^C(m,2) discrete values. Therefore, both these operations, being applied together but independently, map the block to a matrix of dimensions (l^C(m,2) , l^m). This matrix is the error syndrome.

[0133] The error syndrome for the 2x2 block on the Cartesian lattice may be represented by a matrix as large as 16x64. This is the maximum dimension accounting for all possible binary operators. Only 16 of the cells in the matrix correspond to the allowed "words" of the alphabet; all other ones are forbidden. Thus, for tile 606, a unary operation is performed and the tile is compared pairwise to each of the tiles directly adjacent to the tile horizontally and vertically. Subsequently, a binary operation is used to compare the tile to the adjacent tiles, and any contradiction will result in the error syndrome indicating whether there is or is not a contradiction at tile 606 based on the alphabet. This process may be repeated for each tile in curvilinear optical marker 600 to perform a “spell check” on a captured image of an optical marker and create an error syndrome of identified potential errors for all of the tiles. Where there is a single error identified, error correction comprises switching the value of the value changed by noise. However, if two or more errors are identified, the error syndrome may be used to identify the appropriate error correction to be performed by taking into account the probabilities of falling out of one of the two allowed symbols equidistant from the forbidden one; i.e., using distances with weights proportional to the corresponding probabilities. The controller 104 may apply a machine learning model (e.g., see FIG.8), such as a convolutional neural network, to detect the errors and probabilities required for error correction.

[0134] Additionally or alternatively, tiles may be compared pairwise to adjacent tiles that share a border. For example, tile 610 may be compared to adjacent tiles 612A, 612B, 612C, and 612D. If the value of tile 610 has been changed by noise, the value may be compared to adjacent tiles 612A, 612B, 612C, and 612D unchanged by noise to determine how alike or different the changed value of tile 610 is to the adjacent isothetic polygons of known values. This may result in a probabilistic estimate of the corrected value for tile 610.Attorney Docket No.: 00379-0040-00304

[0135] In general, any process or operation discussed in this disclosure may be computer-implemented, such as the processes illustrated in FIG.2 or described with reference to FIGS.4-6, and may be performed by one or more processors of a computer system. A process or process step performed by one or more processors may also be referred to as an operation. The one or more processors may be configured to perform such processes by having access to instructions (e.g., software or computer-readable code) that, when executed by the one or more processors, cause the one or more processors to perform the processes. The instructions may be stored in a memory of the computer system. A processor may be a central processing unit (CPU), a graphics processing unit (GPU), or any suitable types of processing unit.

[0136] A computer system, such as a system or device implementing a process or operation in the examples above, may include one or more computing devices. One or more processors of a computer system may be included in a single computing device or distributed among a plurality of computing devices. A memory of the computer system may include the respective memory of each computing device of the plurality of computing devices.

[0137] FIG.7 depicts an implementation of a computer system that executes techniques presented herein, according to one or more embodiments. Computer system 700 can include a set of instructions that can be executed to cause the computer system 700 to perform any one or more of the methods or computer-based functions disclosed herein. The computer system 700 operates as a standalone device or is connected, e.g., using a network, to other computer systems or peripheral devices.

[0138] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification, discussions utilizing terms such as "processing," "computing," "calculating," “determining”, “analyzing,” or the like, refer to the action and / or processes of a computer or computing system, or similar electronic computing device, that manipulate and / or transform data represented as physical, such as electronic, quantities into other data similarly represented as physical quantities.

[0139] In a similar manner, the term “"processor" refers to any device or portion of a device that processes electronic data, e.g., from registers and / or memory to transform that electronic data into other electronic data that, e.g., is stored in registers and / or memory. A “computer,” a “computing machine,” a "computing platform," a “computing device,” a “controller,” or a “server” includes one or more processors.Attorney Docket No.: 00379-0040-00304

[0140] In a networked deployment, the computer system 700 operates in the capacity of a server or as a client user computer in a server-client user network environment, or as a peer computer system in a peer-to-peer (or distributed) network environment. The computer system 700 can also be implemented as or incorporated into various devices, such as a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile device, a palmtop computer, a laptop computer, a desktop computer, a communications device, a wireless telephone, a land-line telephone, a control system, a camera, a scanner, a facsimile machine, a printer, a pager, a personal trusted device, a web appliance, a network router, switch or bridge, or any other machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. In a particular implementation, the computer system 700 can be implemented using electronic devices that provide voice, video, or data communication. Further, while the computer system 700 is illustrated as a single system, the term “system” shall also be taken to include any collection of systems or sub-systems that individually or jointly execute a set, or multiple sets, of instructions to perform one or more computer functions.

[0141] As illustrated in FIG.7, the computer system 700 includes a processor 702, e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both. The processor 702 can be a component in a variety of systems. For example, the processor 702 is part of a standard personal computer or a workstation. The processor 702 is one or more processors, digital signal processors, application specific integrated circuits, field programmable gate arrays, servers, networks, digital circuits, analog circuits, combinations thereof, or other now known or later developed devices for analyzing and processing data. The processor 702 implements a software program, such as code generated manually (e.g., programmed).

[0142] The computer system 700 includes a memory 704 that can communicate via a bus 708. The memory 704 is a main memory, a static memory, or a dynamic memory. The memory 704 includes, but is not limited to computer readable storage media such as various types of volatile and non-volatile storage media, including but not limited to random access memory, read-only memory, programmable read-only memory, electrically programmable read- only memory, electrically erasable read-only memory, flash memory, magnetic tape or disk, optical media, and the like. In one implementation, the memory 704 includes a cache or random- access memory for the processor 702. In alternative implementations, the memory 704 isAttorney Docket No.: 00379-0040-00304 separate from the processor 702, such as a cache memory of a processor, the system memory, or other memory. The memory 704 can be an external storage device or database for storing data. Examples include a hard drive, compact disc (“CD”), digital video disc (“DVD”), memory card, memory stick, floppy disc, universal serial bus (“USB”) memory device, or any other device operative to store data. The memory 704 is operable to store instructions executable by the processor 702. The functions, acts or tasks illustrated in the figures or described herein are performed by the processor 702 executing the instructions stored in the memory 704. The functions, acts or tasks are independent of the particular type of instructions set, storage media, processor or processing strategy and are performed by software, hardware, integrated circuits, firm-ware, micro-code and the like, operating alone or in combination. Likewise, processing strategies can include multiprocessing, multitasking, parallel processing, and the like.

[0143] As depicted, the computer system 700 further included a display 710, such as a liquid crystal display (LCD), an organic light emitting diode (OLED), a flat panel display, a solid-state display, a cathode ray tube (CRT), a projector, a printer or other now known or later developed display device for outputting determined information. The display 710 acts as an interface for the user to see the functioning of the processor 702, or specifically as an interface with the software stored in the memory 704 or in a drive unit 706.

[0144] Additionally or alternatively, the computer system 700 includes an input / output device 712 configured to allow a user to interact with any of the components of the computer system 700. The input / output device 712 is a number pad, a keyboard, or a cursor control device, such as a mouse, or a joystick, touch screen display, remote control, or any other device operative to interact with the computer system 700.

[0145] The computer system 700 also or alternatively includes the drive unit 706 implemented as a disk or optical drive. The drive unit 706 includes a computer-readable medium 722 in which one or more sets of instructions 724, e.g., software, can be embedded. Further, the sets of instructions 724 embody one or more of the methods or logic as described herein. The instructions 724 reside completely or partially within the memory 704 and / or within the processor 702 during execution by the computer system 700. The memory 704 and the processor 702 can also include computer-readable media as discussed above.

[0146] In some systems, the computer-readable medium 722 includes the sets of instructions 724 or receives and executes the sets of instructions 724 responsive to a propagatedAttorney Docket No.: 00379-0040-00304 signal so that a device connected to a network 106 can communicate voice, video, audio, images, or any other data over the network 106. Further, the sets of instructions 724 are transmitted or received over the network 106 via a communication port or interface 720, and / or using the bus 708. The communication port or interface 720 is a part of the processor 702 or is a separate component. The communication port or interface 720 is created in software or is a physical connection in hardware. The communication port or interface 720 are configured to connect with the network 106, external media, the display 710, or any other components in the computer system 700, or combinations thereof. The connection with the network 106 is a physical connection, such as a wired Ethernet connection or is established wirelessly as discussed below. Likewise, the additional connections with other components of the computer system 700 are physical connections or are established wirelessly. The network 106 is alternatively directly connected to the bus 708.

[0147] While the computer-readable medium 722 is depicted to be a single medium, the term "computer-readable medium" includes a single medium or multiple media, such as a centralized or distributed database, and / or associated caches and servers that store one or more sets of instructions. The term "computer-readable medium" also includes any medium that is capable of storing, encoding, or carrying a set of instructions for execution by a processor or that cause a computer system to perform any one or more of the methods or operations disclosed herein. In some examples, the computer-readable medium 722 is non-transitory, and is tangible.

[0148] The computer-readable medium 722 can include a solid-state memory such as a memory card or other package that houses one or more non-volatile read-only memories. The computer-readable medium 722 can be a random-access memory or other volatile re-writable memory. Additionally or alternatively, the computer-readable medium 722 can include a magneto-optical or optical medium, such as a disk or tapes or other storage device to capture carrier wave signals such as a signal communicated over a transmission medium. A digital file attachment to an e-mail or other self-contained information archive or set of archives are considered a distribution medium that is a tangible storage medium. Accordingly, the disclosure is considered to include any one or more of a computer-readable medium or a distribution medium and other equivalents and successor media, in which data or instructions are storable.

[0149] In an alternative implementation, dedicated hardware implementations, such as application specific integrated circuits, programmable logic arrays and other hardware devices,Attorney Docket No.: 00379-0040-00304 can be constructed to implement one or more of the methods described herein. Applications that include the apparatus and systems of various implementations can broadly include a variety of electronic and computer systems. One or more implementations described herein implement functions using two or more specific interconnected hardware modules or devices with related control and data signals that can be communicated between and through the modules, or as portions of an application-specific integrated circuit. Accordingly, the present system encompasses software, firmware, and hardware implementations.

[0150] The computer system 700 is connected to the network 106. The network 106 defines one or more networks including wired or wireless networks, such as the network 106 described in FIG.1. The wireless network can be a cellular telephone network, a 702.11, 702.18, 702.20, or WiMAX network. Further, such networks include a public network, such as the Internet, a private network, such as an intranet, or combinations thereof, and utilize a variety of networking protocols now available or later developed including, but not limited to TCP / IP based networking protocols. The network 106 can include wide area networks (WAN), such as the Internet, local area networks (LAN), campus area networks, metropolitan area networks, a direct connection such as through a Universal Serial Bus (USB) port, or any other networks that allow for data communication. The network 106 is configured to couple one computing device to another computing device to enable communication of data between the devices. The network 106 generally is enabled to employ any form of machine-readable media for communicating information from one device to another. The network 106 includes communication methods by which information travels between computing devices. The network 106 can be divided into sub- networks. The sub-networks allow access to all of the other components connected thereto or the sub-networks restrict access between the components. The network 106 can be regarded as a public or private network connection and can include, for example, a virtual private network or an encryption or other security mechanism employed over the public Internet, or the like.

[0151] As disclosed herein, one or more implementations disclosed herein may be applied by using a machine learning model. A machine learning model as disclosed herein may be trained using one or more components or operations of FIGS.1-6. As depicted in flow diagram 800 of FIG.8, training data 812 may include one or more of stage inputs 814 and known outcomes 818 related to a machine learning model to be trained. The stage inputs 814 may be from any applicable source including a component or set depicted in the figures provided herein.Attorney Docket No.: 00379-0040-00304 The known outcomes 818 may be included for machine learning models generated based on supervised or semi-supervised training. An unsupervised machine learning model might not be trained using known outcomes 818. Known outcomes 818 may include known or desired outputs for future inputs similar to or in the same category as stage inputs 814 that do not have corresponding known outputs.

[0152] The training data 812 and a training algorithm 820 may be provided to a training component 830 that may apply the training data 812 to the training algorithm 820 to generate a trained machine learning model 850. According to an implementation, the training component 830 may be provided comparison results 816 that compare a previous output of the corresponding machine learning model to apply the previous result to re-train the machine learning model. The comparison results 816 may be used by the training component 830 to update the corresponding machine learning model. The training algorithm 820 may utilize machine learning networks and / or models including, but not limited to a deep learning network such as Deep Neural Networks (DNN), Convolutional Neural Networks (CNN), Fully Convolutional Networks (FCN) and Recurrent Neural Networks (RCN), probabilistic models such as Bayesian Networks and Graphical Models, and / or discriminative models such as Decision Forests and maximum margin methods, or the like.

[0153] A machine learning model disclosed herein may be trained by adjusting one or more weights, layers, and / or biases during a training phase. During the training phase, historical or simulated data may be provided as inputs to the model. The model may adjust one or more of its weights, layers, and / or biases based on such historical or simulated information. The adjusted weights, layers, and / or biases may be configured in a production version of the machine learning model (e.g., a trained model) based on the training. Once trained, the machine learning model may output machine learning model outputs in accordance with the subject matter disclosed herein. According to an implementation, one or more machine learning models disclosed herein may continuously update based on feedback associated with use or implementation of the machine learning model outputs.

[0154] FIG.9 depicts example two-dimensional lattice models and associated blocks. Lattice model 910 is based on a Cartesian grid model formed by two sets of lines to form tile 912, and each tile 912 is square and part of four blocks, one example of which is shown at block 915 which comprises four tiles. Lattice model 920 is formed by three lines to form tile 922, andAttorney Docket No.: 00379-0040-00304 each tile 922 is triangular and part of three blocks, one example of which is shown at block 925 which comprises six tiles. Lattice model 930 is formed by three sets of lines to form tile 932, and each tile 932 is hexagonal and part of six blocks, one example of which is shown at block 935 which comprises three tiles. The number of blocks that a tile is part of is given by the number of vertices touching the tile. As such, square tiles are part of four blocks, triangular tiles are part of three blocks, and hexagonal tiles are part of six blocks. The larger the dimensionality of the block, the smaller the ratio of allowed combinations to forbidden combinations. As such, the potential for corrective ability increases with the dimensionality of the block.

[0155] In the lattice model 920 with triangular tiles 922, six triangular tiles 922 converge into one node or vertex, such that the dimensionality of the node syndrome will be larger than for the lattice model 910 based on a Cartesian grid (where four rectangular tiles 912 converge into one node or vertex). Thus, the first-stage correction capacity (the first-stage correction described above) of such a node or vertex is higher than that of a Cartesian lattice node or vertex. The methods described in this disclosure may include generating a lattice model including a plurality of isothetic polygons, wherein the lattice model includes a regular triangular grid as described by lattice model 920.

[0156] In the lattice model 930 with hexagon tiles 932, each hexagonal tile 932 in a grid has six neighboring hexagonal tiles 932, such that the corrective power (the second-stage correction described above) will be higher than that of the Cartesian lattice model 910. The methods described in this disclosure may include generating a lattice model including a plurality of isothetic polygons, wherein the lattice model includes a regular hexagonal grid as described by lattice model 930.

[0157] The tile values are considered to be the letters of a given alphabet, the blocks represent “words” to be checked for correctness. When the “word” does not exist in the “vocabulary” of the two-dimensional grammar, it is identified as a self-contradiction and must be replaced. The block 925 may have up to a 64×32768 error syndrome matrix, while the block 935 has a maximum of an 8x6 error syndrome matrix.

[0158] With respect to the lattice model 910, the tiles have natural coordinates [i, j], akin to matrix indices. These coordinates naturally correspond to the centers of tiles. The same coordinate system is also used for raster images such that pixels typically have the same coordinate system. The bounding lines are between tiles such that the bounding lines haveAttorney Docket No.: 00379-0040-00304 coordinates in half-steps (e.g., 0.5, 1.5,…, 12.5). Thus, the lattice vertices, which are the intersections of the lines, have coordinates in half-steps, such as [2.5, 1.5] in one example. A block centered at this vertex is thus mapped to [2.5, 1.5] Cartesian coordinates.

[0159] The simplest blocks for such a structure are 2x2, i.e., given any lattice vertex, a block contains 4 lattice tiles. The coordinates of lattice tiles may be ordered as a matrix with two- dimensional indexing. For example, the value of the upper-left tile may be saved into the matrix as cell (1, 1). Geometrically, the vertices are placed between the tiles, so the geometrical coordinates of the blocks, mapped to the vertices, will have coordinates shifted by (0.5, 0.5).

[0160] As such, one block has four tiles with the coordinates:

[0161] (I, J), (I, J+1);

[0162] (I+1, J), (I+1, J+1).

[0163] The vertex is the intersection of the lines forming the tiles. By this, the vertex is in the very center and touches all four tiles, so the vertex has coordinates (i+0.5, j+0.5).

[0164] The next block to the right has coordinates (i+0.5, j+1.5): the second coordinate incremented by 1. This block contains the tiles:

[0165] (I, J+1), (I, J+2);

[0166] (I+1, J+1), (I+1, J+2).

[0167] Thus, the tiles (I, J+1) and (I+1, J+1) are common to these two neighboring blocks, and the tiles are the overlapping between the blocks.

[0168] FIG.10 depicts example non-regular lattice models, according to one or more embodiments. The non-regular, aperiodic lattice models described in FIG.10 may also be described within the formalism of the grammar described above. As described above, the first- stage correction is correlated with the number of tiles meeting at a vertex. Aperiodic lattice models may have varying numbers of tiles meeting at vertices throughout the model. For example, lattice model 1010 has three tiles 1012 meeting at vertex 1013 and four tiles 1012 meeting at vertex 1015. Lattice model 1010 is defined by one shape of tile 1012 that repeats in an aperiodic manner. Lattice model 1020 includes a first tile shape 1022 and a second tile shape 1024 that repeat in a varying manner to form the aperiodic lattice model 1020. Similarly to the lattice model 1010, the vertices of lattice model 1020 vary in the number of tiles meeting at the node. For example, three tiles meet at vertex 1023, while five tiles meet at vertex 1025. Other vertices have varying numbers of tiles that meet at the vertex, as depicted in FIG.10.Attorney Docket No.: 00379-0040-00304

[0169] Lattice model 1030 is comprised of three different tile shapes, a first tile shape 1032, a second tile shape 1034, and a third tile shape 1036. Yet all the vertices in lattice model 1030 have the same number of tiles that meet at the vertex (i.e, four). For example, four tiles meet at vertex 1033 and four tiles meet at vertex 1035.

[0170] It will be understood that the operations of methods discussed are performed in one embodiment by an appropriate processor (or processors) of a processing (e.g., computer) system executing instructions (computer-readable code) stored in storage. It will also be understood that the disclosure is not limited to any particular implementation or programming technique and that the disclosure is implementable using any appropriate techniques for implementing the functionality described herein. The disclosure is not limited to any particular programming language or operating system.

[0171] It should be appreciated that in the above description of example embodiments of the present disclosure, various features of the present disclosure are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed embodiment requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this disclosure.

[0172] Furthermore, while some embodiments described herein include some but not other features included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the disclosure, and form different embodiments, as would be understood by those skilled in the art. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0173] Furthermore, some of the embodiments are described herein as a method or combination of elements of a method that can be implemented by a processor of a computer system or by other means of carrying out the function. Thus, a processor with the necessary instructions for carrying out such a method or element of a method forms a means for carrying out the method or element of a method. Furthermore, an element described herein of anAttorney Docket No.: 00379-0040-00304 apparatus embodiment is an example of a means for carrying out the function performed by the element for the purpose of carrying out the embodiments.

[0174] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the present disclosure can be practiced without these specific details. In other instances, well-known methods, structures, and techniques have not been shown in detail in order not to obscure an understanding of this description.

[0175] Thus, while there has been described what are believed to be the preferred embodiments of the present disclosure, those skilled in the art will recognize that other and further modifications can be made thereto without departing from the spirit of the embodiments, and it is intended to claim all such changes and modifications as falling within the scope of the embodiments. For example, any formulas given above are merely representative of procedures that can be used. Functionality can be added or deleted from the block diagrams and operations are interchangeable among functional blocks. Steps can be added or deleted to methods described within the scope of the present disclosure.

[0176] The above disclosed subject matter is to be considered illustrative, and not restrictive, and the appended claims are intended to cover all such modifications, enhancements, and other implementations, which fall within the true spirit and scope of the present disclosure. Thus, to the maximum extent allowed by law, the scope of the present disclosure is to be determined by the broadest permissible interpretation of the following claims and their equivalents, and shall not be restricted or limited by the foregoing detailed description. While various implementations of the disclosure have been described, it will be apparent to those of ordinary skill in the art that many more implementations and implementations are possible within the scope of the disclosure. Accordingly, the disclosure is not to be restricted except in light of the attached claims and their equivalents.

Claims

Attorney Docket No.: 00379-0040-00304 What is claimed is:

1. A computer-implemented method comprising: generating a lattice model including a plurality of isothetic polygons; encoding respective values of the plurality of isothetic polygons using a two-dimensional grammar, wherein the two-dimensional grammar includes a plurality of allowed value combinations for the plurality of isothetic polygons; generating an optical marker using the plurality of isothetic polygons with the encoded values; receiving an image of the optical marker; generating an error syndrome for the image of the optical marker using the two- dimensional grammar; and generating an error-corrected image of the optical marker using the error syndrome.

2. The computer-implemented method of claim 1, wherein generating the lattice model including the plurality of isothetic polygons includes generating one or more of a regular Cartesian grid with a first length in a first direction and a second length in a second direction perpendicular to the first direction, a regular triangular grid, or a regular hexagonal grid.

3. The computer-implemented method of claim 1, wherein generating the lattice model includes generating: (i) a plurality of concentric circles, and (ii) a plurality of lines through a center of the plurality of concentric circles.

4. The computer-implemented method of claim 1, wherein the plurality of allowed value combinations of the two-dimensional grammar correspond to combinations of adjacent isothetic polygons of the plurality of isothetic polygons, wherein the adjacent isothetic polygons share a border.

5. The computer-implemented method of claim 1, wherein the plurality of allowed value combinations of the two-dimensional grammar correspond to combinations of adjacent isotheticAttorney Docket No.: 00379-0040-00304 polygons of the plurality of isothetic polygons, wherein the adjacent isothetic polygons share a vertex.

6. The computer-implemented method of claim 1, wherein the two-dimensional grammar further includes a plurality of forbidden value combinations for the plurality of isothetic polygons.

7. The computer-implemented method of claim 1, further comprising: capturing the image of the optical marker using a camera of an aerial vehicle; and receiving the image of the optical marker from the camera.

8. The computer-implemented method of claim 1, wherein generating the error syndrome includes determining a value, among the encoded values, that has been changed by noise.

9. The computer-implemented method of claim 8, wherein generating the error-corrected image of the optical marker includes changing the value that was changed by the noise.

10. The computer-implemented method of claim 1, wherein generating the error syndrome includes using a convolutional neural network.

11. A computer-implemented method comprising: generating a lattice model including a plurality of isothetic polygons; encoding respective values of the plurality of isothetic polygons using a two-dimensional grammar, wherein the two-dimensional grammar includes a plurality of allowed value combinations for the plurality of isothetic polygons; and generating an optical marker using the plurality of isothetic polygons with the encoded values.

12. The computer-implemented method of claim 11, further comprising: receiving an image of the optical marker;Attorney Docket No.: 00379-0040-00304 generating an error syndrome for the image of the optical marker using the two- dimensional grammar; and generating an error-corrected image of the optical marker using the error syndrome.

13. The computer-implemented method of claim 11, wherein generating the lattice model includes generating: (i) a plurality of concentric circles, and (ii) a plurality of lines through a center of the plurality of concentric circles.

14. The computer-implemented method of claim 11, wherein the plurality of allowed value combinations of the two-dimensional grammar correspond to combinations of adjacent isothetic polygons of the plurality of isothetic polygons, wherein the adjacent isothetic polygons share a border.

15. The computer-implemented method of claim 11, wherein the plurality of allowed value combinations of the two-dimensional grammar correspond to combinations of adjacent isothetic polygons of the plurality of isothetic polygons, wherein the adjacent isothetic polygons share a vertex.

16. A computer-implemented method comprising: receiving an image of an optical marker, wherein the image includes a plurality of isothetic polygons with respective encoded values; generating an error syndrome for the image of the optical marker using a two- dimensional grammar, wherein the two-dimensional grammar includes a plurality of allowed value combinations for the plurality of isothetic polygons; and generating an error-corrected image of the optical marker using the error syndrome.

17. The computer-implemented method of claim 16, further comprising: generating a lattice model including the plurality of isothetic polygons; encoding respective values, as the respective encoded values, of the plurality of isothetic polygons using the two-dimensional grammar; andAttorney Docket No.: 00379-0040-00304 generating the optical marker using the plurality of isothetic polygons with the encoded values.

18. The computer-implemented method of claim 16, wherein generating the error syndrome includes determining a value, among the encoded values, that has been changed by noise.

19. The computer-implemented method of claim 18, wherein generating the error-corrected image of the optical marker includes changing the value that was changed by the noise.

20. The computer-implemented method of claim 16, further comprising: capturing the image of the optical marker using a camera of an aerial vehicle; and receiving the image of the optical marker from the camera.

Images