System and method for generating and correcting images of optical marks

Abstract

Systems and methods for generating an optical mark and for correcting an image of the optical mark are disclosed, including: generating a lattice model comprising a plurality of axis-aligned polygons; encoding respective values of the plurality of axis-aligned polygons using a two-dimensional grammar, wherein the two-dimensional grammar comprises a plurality of allowed value combinations for the plurality of axis-aligned polygons; generating an optical mark using the plurality of axis-aligned polygons with encoded values; receiving an image of the optical mark; generating an error syndrome for the image of the optical mark using the two-dimensional grammar; and generating an error-corrected image of the optical mark using the error syndrome.

Description

Cross-reference to related applications

[0001] This application claims priority to U.S. Provisional Application No. 63 / 558,931, filed February 28, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] Embodiments of this disclosure relate to systems and methods for generating and reading optical markers, and more specifically, to noise-resistant error correction of images of optical markers. Background Technology

[0003] Computer-readable optical tags are widely used for transmitting information from captured images. However, images of optical tags captured by image capture devices may contain one or more errors due to noise from environmental conditions or image anomalies. Errors in the image can affect the accurate transmission of information encoded in the optical tag. Some techniques involving error correction require additional data (such as parity bits) to detect the errors. However, this additional data prolongs the transmission time, leading to less efficient information delivery.

[0004] The background description provided herein is for the purpose of generally presenting the background of this disclosure. Unless otherwise stated herein, the material described in this section is not prior art to the claims of this application and is not, by including in this section, an admission that it is prior art or a suggestion of prior art. Summary of the Invention

[0005] According to certain aspects of this disclosure, systems and methods for generating and correcting images of optical markers are disclosed.

[0006] In some aspects, the technology described herein relates to a computer-implemented method comprising: generating a lattice model comprising a plurality of isothetic polygons; encoding corresponding values ​​of the plurality of isothetic polygons using a two-dimensional grammar, wherein the two-dimensional grammar includes a plurality of allowed value combinations for the plurality of isothetic polygons; generating an optical marker using the plurality of isothetic polygons having encoded values; receiving an image of the optical marker; generating an error syndrome for the image of the optical marker using the two-dimensional grammar; and generating an error-corrected image of the optical marker using the error syndrome.

[0007] In some aspects, the technology described herein relates to a computer-implemented method comprising: generating a lattice model comprising a plurality of axis-aligned polygons; encoding corresponding values ​​of the plurality of axis-aligned polygons using a two-dimensional grammar, wherein the two-dimensional grammar includes a plurality of allowed value combinations for the plurality of axis-aligned polygons; and generating optical markers using the plurality of axis-aligned polygons having encoded values.

[0008] In some aspects, the technology described herein relates to a computer-implemented method comprising: receiving an image of an optical marker, wherein the image includes a plurality of axis-aligned polygons having corresponding coded values; generating an error syndrome for the image of the optical marker using a two-dimensional grammar, wherein the two-dimensional grammar includes a plurality of allowed value combinations for the plurality of axis-aligned polygons; and generating an error-corrected image of the optical marker using the error syndrome.

[0009] Other objects and advantages of the disclosed embodiments will be set forth in part in the description which follows, and will be apparent in part from the description, or may be learned by practice of the disclosed embodiments. The objects and advantages of the disclosed embodiments will be realized and achieved by means of the elements and combinations particularly pointed out in the appended claims.

[0010] It should be understood that the above overview and the following detailed description are merely exemplary and illustrative, and do not limit the claimed disclosed embodiments. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various examples and, together with the description, serve to explain the principles of the disclosed examples and embodiments.

[0012] Various aspects of this disclosure can be implemented in conjunction with the embodiments shown in the accompanying drawings. These drawings illustrate different aspects of this disclosure, and reference numerals showing the same structures, components, materials, and / or elements in different drawings are similarly labeled where appropriate. It is understood that, in addition to the structures, components, and / or elements specifically shown, various combinations of structures, components, and / or elements are contemplated, and such combinations are within the scope of this disclosure.

[0013] Furthermore, numerous embodiments exist as described and illustrated herein. This disclosure is not limited to any single aspect or embodiment therein, nor to any combination and / or arrangement of such aspects and / or embodiments. Moreover, each aspect of this disclosure and / or its embodiments may be employed individually or in combination with other aspects of this disclosure and / or its embodiments. For the sake of brevity, certain arrangements and / or combinations are not discussed and / or described separately herein. It is important to note that embodiments or implementations described herein as “exemplary” should not be construed as superior to or advantageous to (e.g.) other embodiments or implementations; rather, they are intended to reflect or indicate that such embodiments are “example” embodiments.

[0014] Figure 1 A system comprising an aircraft and an optical marker, according to one or more embodiments, is described.

[0015] Figure 2 A flowchart is depicted for a method according to one or more embodiments for generating an optical marker, capturing an image of the optical marker, and generating an error-corrected image of the optical marker.

[0016] Figure 3A , Figure 3B and Figure 3C The results of error correction of the generated optical marks using various techniques according to one or more embodiments are described.

[0017] Figure 4 Examples of optical markings generated using a lattice model including a Cartesian grid, according to one or more embodiments, are depicted.

[0018] Figure 5 An example of an alphabet of a two-dimensional grammar according to one or more embodiments is depicted.

[0019] Figure 6 An example of an optical mark generated using a lattice model comprising multiple concentric circles, according to one or more embodiments, is depicted.

[0020] Figure 7 The invention describes implementations of computer systems that perform the techniques described herein, according to one or more embodiments.

[0021] Figure 8 A flowchart for training a machine learning model is depicted according to one or more embodiments.

[0022] Figure 9 An example two-dimensional lattice model and associated blocks according to one or more embodiments are depicted.

[0023] Figure 10 An example irregular lattice model according to one or more embodiments is depicted.

[0024] As used herein, the terms “comprises,” “comprising,” “includes,” “including,” or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but may also include other elements not expressly listed or inherent to such process, method, article, or apparatus. The term “exemplary” is used in the sense of “example” rather than “ideal.” Furthermore, the terms “first,” “second,” etc., used herein do not indicate any order, quantity, or importance, but are used to distinguish one element or structure from another. Additionally, the terms “a” and “an” used herein do not indicate a quantitative limitation but rather the presence of one or more of the items referred to. Detailed Implementation

[0025] Reference will now be made in detail to an example of this disclosure, illustrated in the accompanying drawings. Where possible, the same reference numerals will be used throughout the drawings to refer to the same or similar portions. In the following discussion, relative terms such as “about,” “substantially,” and “approximately” will be used to indicate possible variations in the range of values ​​stated, as will be indicated below.

[0026] Optical markers are widely used for information encoding and visual reading, as well as for identification and measurement tasks. In aviation, reference markers are used for the rapid and reliable location and orientation of aircraft, which is particularly important for precision landing missions. The number of such marker families is constantly growing, which is related to location and identification under challenging visual conditions.

[0027] Embodiments of this disclosure relate to systems and methods for generating and reading optical markers, and more specifically, to non-destructive and noise-resistant error correction of optical markers. One or more embodiments may provide a robust and noise-tolerant method for component classification of the structure of optical markers. One or more embodiments may provide error detection and correction encoding for markers. One or more embodiments may provide a system that can be used as a supplement to existing redundancy encoding methods. One or more embodiments may provide methods for segmenting, locating, and decoding markers of existing types. One or more embodiments may improve the robustness of image recognition systems. One or more embodiments may provide a system for synthesizing noise-resistant markers that allows for identification and / or accurate location under conditions of high noise, poor visibility, and / or partial marker occlusion.

[0028] For example, one or more embodiments may provide systems for computer vision, computational geometry, pattern recognition, error correction coding, precision landing, or optical navigation. For example, one or more embodiments may provide the advantage of structural pattern recognition for a large class of geometrically rigorous objects, such as reference markers and QR codes.

[0029] Data structures arranged in an ordered lattice are known to have inherent redundancy, thus allowing noise-resistant coding without additional check characters. The ordered information checks itself for conformity to a given order, and if inconsistent, signals an error and / or corrects it. Two-dimensional Schlesinger context-free grammars can be used for lossless noise-resistant coding. Due to the redundancy in the lattice, error-correcting coding can be achieved without using check characters to extend the data length. The vast majority of optical markers can be represented as axis-aligned polygons. Representations using axis-aligned polygons can be applied to a larger class of geometrically ordered structures. All of these structures can be encoded using two-dimensional grammars. Therefore, optical markers are likely inherently noise-resistant when encoded using two-dimensional grammars.

[0030] The following is about Figure 1 The description provides an aircraft as an exemplary system for using the systems and methods described in this disclosure. However, this disclosure is not limited to the use of the system and methods in aircraft. For example, the systems and methods described below can be applied to any machine-readable optical imaging system, such as reference markers or quick response (QR) codes.

[0031] Figure 1 System 100 is depicted, which includes an aircraft 102 and an optical marker 112. System 100 may include aircraft 102, controller 104, network 106, optical marker generator 108, image capture device 110, and optical marker 112.

[0032] For example, aircraft 102 may be a vertical takeoff and landing (VTOL) aircraft and may generate flight propulsion, for example, using one or more of batteries or fuel. Aircraft 102 may be used for short and long-distance travel to transport one or more passengers, baggage, cargo, and / or other objects. Aircraft 102 may include one or more of a fuselage, canard, aft wing, tiltrotor, fixed rotor, or control surfaces. Aircraft 102 may be configured to be maneuvered in multiple (e.g., six) degrees of freedom. For example, aircraft 102 may travel about a vertical axis throughout flight (e.g., during climb, descent, takeoff, and / or landing). Aircraft 102 may rotate about a vertical axis for yaw movement. Aircraft 102 may also move along a longitudinal axis in a generally rearward or forward direction. Aircraft 102 may rotate (roll) relative to a longitudinal axis. Aircraft 102 may travel (e.g., translate) along a lateral axis and rotate about a lateral axis to produce pitch movement.

[0033] While this disclosure refers to an aircraft, specifically a VTOL aircraft, it will be readily recognized by those skilled in the art that the reference to an aircraft is merely exemplary, and the concepts of this disclosure can be used in conjunction with any suitable or comparable aircraft, such as fixed-wing aircraft, helicopters, aerostatic aircraft, flight simulators, spacecraft, commercial fixed-wing aircraft, or electric vertical takeoff and landing (eVTOL) aircraft. The above list is by no means a limiting list of the structural definition of the term "aircraft."

[0034] The aircraft 102 may include a controller 104 and an image capture device 110. The controller 104 may include one or more controllers, such as those mentioned in reference 104. Figure 7 The computer system 700 is discussed. Controller 104 can be configured to control the operation of aircraft 102, such as maneuvering in multiple degrees of freedom. Controller 104 can control the operation of image capture device 110. Image capture device 110 can be a camera, for example, and can be configured to capture images, such as images of optical marker 112.

[0035] Controller 104 can communicate with optical tag generator 108 using network 106. For example, network 106 may be a wireless network. Optical tag generator 108 may include one or more controllers configured to generate optical tags 112 and / or communicate with controller 104 of aircraft 102. Although optical tag generator 108 is depicted as a component separate from aircraft 102 in system 100, optical tag generator 108 may also be an onboard component of aircraft 102.

[0036] For example, optical marker 112 can provide a target landing area for aircraft 102, or it can provide a fixed position marker for optically checking the in-flight position of aircraft 102 against Global Positioning System (GPS) coordinates. Controller 104 can control the operation of image acquisition device 110 to capture and decode images of optical marker 112, and can control the operation of aircraft 102 based on the captured images. For example, during the landing operation of aircraft 102, controller 104 can control the operation of image acquisition device 110 to periodically capture images of optical marker 112, and based on these images, control the operation of one or more of the tilting rotor, fixed rotor, or control surfaces to adjust the position of aircraft 102 relative to optical marker 112. Controller 104 calculates and adjusts the position of image acquisition device 110 relative to the known position of optical marker 112. For example, if optical marker 112 is an indicator of a vertical takeoff and landing field (VTOL), the geographic coordinates of the VTOL and optical marker 112 can be transmitted to and / or stored in the memory of controller 104. The controller 104 then calculates and adjusts the position of the image acquisition device 110 relative to the optical marker 112 by considering all six degrees of freedom of the image acquisition device 110 and the aircraft 102, including the x-axis, y-axis, and z-axis degrees of freedom and three angular degrees of freedom. The optical marker 112 can provide the aircraft 102 with information as encoded values ​​in the optical marker 112.

[0037] Therefore, the decoded image of optical marker 112 should be error-free. However, due to noise from environmental conditions or image anomalies such as, for example, motion blur and / or defocus blur and / or image capture noise under low light conditions, the decoded image of optical marker 112 captured by image capture device 110 may include one or more errors, typically at a low signal-to-noise ratio. Errors included in image decoding may affect the operation of controller 104, which in turn may affect the operation of aircraft 102. Therefore, optical marker generator 108 can generate an image for use as optical marker 112 that may be less sensitive to noise relative to some systems. Controller 104 can correct for errors in the image of optical marker 112 captured by image capture device 110.

[0038] Above Figure 1 The description is provided as an exemplary system for using the systems and methods described in this disclosure. However, this disclosure is not limited to the use of the system and methods in aircraft. For example, the systems and methods described below can be applied to any machine-readable optical imaging system, such as, for example, reference markers or quick response (QR) codes.

[0039] Figure 2A flowchart is depicted for a method 200 for generating an optical mark, capturing an image of the optical mark, and generating an error-corrected image of the optical mark using system 100. Method 200 may include operations 210, 220, 230, 240, 250, and 260.

[0040] Operation 210 includes generating a lattice model comprising multiple axis-aligned polygons at the optical marker generator 108. An axis-aligned polygon is a polygon with alternating edges belonging to two or more families of parameters, which are line bundles centered at two points. In one example, the axis-aligned polygon can be described by polygons defined by a Cartesian mesh, such that each polygon is a square or rectangle of equal size.

[0041] Figure 4 The optical markers depicted are an example of such a lattice model, which comprises multiple axis-aligned polygons defined by a Cartesian grid. Axis-aligned polygons can be described by straight lines and curves. As defined herein, axis-aligned polygons are not limited to polygons with strictly straight edges and can include shapes with curved edges. An example of this is... Figure 6 The lattice model described herein depicts generating a lattice model by generating: (i) multiple concentric circles, and (ii) multiple lines passing through the centers of the multiple concentric circles to produce an axis-aligned polygon comprising two lines opposite each other and two arcs opposite each other, except for the central polygon sharing the lattice center as its vertex. The central polygon defines a wedge comprising an arc and two straight edges.

[0042] A lattice model can be defined by tiling triangles, hexagons, or a combination of grids that include both straight and curved features using a grid composed of three families of lines. A lattice model can include curved polygons, such as polygons defined by “barrel” or “pincushion” radial distortions. Axis-aligned polygons are generally not limited to two-dimensional grids and can be further extended to additional dimensions.

[0043] For example, a 3D Cartesian mesh can consist of bricks shaped like rectangular prisms with length, width, and height. These bricks can be “cells” of the mesh, such that the lattice model is generated by a family of axis-aligned planes, rather than by lines as described with reference to a 2D mesh. Here, the lattice model can form axis-aligned polyhedra instead of 2D axis-aligned polygons. For higher dimensions, the lattice model can be generated by a family of axis-aligned hyperpolyhedra. In the example, when providing optical markings in RGB space, the lattice model can include three separate planes for each color component.

[0044] A lattice model can include defined dimensions, and can include higher dimensions. For example, Figure 4 The lattice model depicted consists of 144 squares arranged in a 12x12 Cartesian grid. The lattice model can include more or fewer polygons, and the size of the polygons can also be varied. Figure 6 The lattice model depicted in the image comprises 60 curved polygons defined by six concentric circles and six straight lines intersecting at the centers of the concentric circles. However, the lattice model can be further modified by increasing or decreasing the number of concentric circles, increasing or decreasing the number of intersecting lines, or increasing or decreasing the size of the concentric circles.

[0045] Operation 220 includes encoding corresponding values ​​for a plurality of axis-aligned polygons using a two-dimensional grammar. The two-dimensional grammar includes a plurality of allowed combinations of values ​​for the plurality of axis-aligned polygons. In the context of the optical tag discussed herein, the two-dimensional grammar may include allowed combinations of encoded axis-aligned polygons and prohibited combinations of encoded axis-aligned polygons, such that when the captured image of the optical tag includes prohibited combinations, the controller 104 can detect an error based on the prohibited combinations.

[0046] Two-dimensional grammars are typically spatial arrangements of symbols representing concepts. In some examples, such as for a regular Cartesian grid, this can be achieved using an "alphabet" of 2x2 blocks of axis-aligned polygons. Higher-dimensional blocks on a Cartesian grid can also be encoded, but this increases the size of the error syndrome without improving the actual error correction capability.

[0047] For example, for a Cartesian grid, the following two-dimensional context-free grammar can be used. G B2DC ,in B2DC It represents a two-dimensional Cartesian symbol.

[0048] Applied to m × n The 2D image generator can be used in both the horizontal and vertical directions: Here, the sequence is... m × n Binary image representation: This is achieved by projecting the original sequence into the code space through the sequential application of horizontal and vertical generators. A single application of any of these matrices stretches the original sequence by a factor of two in each direction. Therefore... ∈ {

[00] ,

[01] ,

[10] ,

[11] }, i ∈{0,…,m -1}, j ∈{0,…, n -1}, where the final terminal symbol is x ij ∈ {00|00,00|01,00|11,…11|11}, i ∈ {0,…, m -1}, j ∈ {0,…, n -1}.

[0049] A similar two-dimensional grammar can be used to construct triangle-axis aligned polygons, i.e., polygons formed by 3 sets of lines instead of 2 sets of lines, where | X |=64; and can construct polygons for hexagonal axis alignment, where | X |=8. Used to press k The higher-dimensional grammars generalized to multidimensional data by Vidica's grid sort will be obtained | X |= 2 k+1 .

[0050] Two-dimensional grammars govern the encoding of axis-aligned polygon values ​​in a lattice model using allowed and forbidden values. In some examples, the encoded values ​​may be binary, such that each axis-aligned polygon in the lattice is encoded as either "0" or "1", or as "white" or "black". Binary values ​​can also represent the absence or presence of a given pattern, such as circles or triangles within cells or tiles defined by axis-aligned polygons. To increase complexity, two detectors can be used in conjunction with two binary encoding systems. For example, a first detector might determine whether a cell is black or white (or any other binary color scheme), and a second detector might determine whether a cell contains a circle (or any other pattern).

[0051] like Figure 5 The alphabet for a binary-coded 2x2 block, as depicted in [1] and discussed above, can have sixteen different arrangements, denoted as [1]–

[16] . In some examples, the two-dimensional grammar governs the combination of 2x2 blocks to which any given polygon in the lattice model can belong. This will be about Figures 4-6 The examples depicted in the text will be described in more detail.

[0052] In operation 230, optical mark generator 108 generates optical mark 112 using multiple axis-aligned polygons with encoded values. For example, optical mark generator 108 may include a printing device to generate a physical optical mark, which can be positioned such that image capture device 110 can capture an image of optical mark 112. For example, optical mark generator 108 may generate a printed image of optical mark 112 such that during landing operation of aircraft 102, optical mark 112 is appropriately sized and positioned so that controller 104 can control the operation of image capture device 110 to periodically capture images of optical mark 112 and, based on these images, control the operation of one or more of tiltable rotors, fixed rotors, or control surfaces to adjust the position of aircraft 102 relative to optical mark 112 to control the landing operation of aircraft 102.

[0053] In operation 240, the image capturing device 110 receives an image of the optical tag 112. The image capturing device 110 may be a camera, for example, configured to capture images. In some cases, in addition to the encoded values, the captured image of the optical tag may also include values ​​that have been altered by noise. Noise may be caused, for example, by environmental conditions (such as fog, darkness, smoke, etc.) or image anomalies (such as interference or printing errors). The optical tag 112 may provide information to a system (such as aircraft 102), and therefore, the image of the optical tag 112 should be error-free.

[0054] In operation 250, controller 104 can use a two-dimensional grammar to generate an error syndrome for an image of an optically labeled object. Figure 1 In the example system 100 depicted, controller 104 can receive captured images from image capture device 110 and generate an error syndrome based on the captured images from optical marker 112. The generation error syndrome includes determining one or more values ​​(if any) that have been altered by noise. Controller 104 receives the captured images and compares them with allowed and prohibited values ​​in a two-dimensional grammar.

[0055] To generate the error syndrome, controller 104 can compare the values ​​of tiles in the captured image in pairs and determine whether these values ​​are equal. In the example of a regular Cartesian grid, within a 2×2 cell group, controller 104 compares the overlap of adjacent axis-aligned polygons vertically and horizontally, generating an error syndrome of size 16×16. In at least one example, only 16 of the 256 values ​​are allowed and the rest are prohibited. Controller 104 detects an error when it detects a prohibited combination. Generating the error syndrome can further include using a machine learning model (e.g., see...). Figure 8Machine learning models, such as convolutional neural networks (CNNs), are used to detect prohibited combinations. A machine learning model can take an image covering a specific patch as input, where the patch contains N tiles as input, and N is an integer. The machine learning model can be described as an N-ary operator, such that given a complete patch of N tiles, the model outputs one of the allowed tile values. The machine learning model can be described with respect to... Figure 8 The method described in the text is to train on blocks, where symmetric training cases are generated because all symmetric groups for each cell are also known.

[0056] In operation 260, controller 104 uses an error syndrome to generate an error-corrected image of the optical marker. Each axis-aligned polygon in the optical marker is contained within multiple blocks of axis-aligned polygons. For example, in optical markers generated based on a Cartesian grid-based lattice model (such as the following...) Figure 4 (As depicted and described in the text), each axis-aligned polygon belongs to four 2x2 quadruple blocks. According to the 2D grammar, only defined block combinations are allowed in this arrangement. If the four 2x2 blocks are the allowed combinations and it is determined that the values ​​of the blocks in all four overlapping blocks are equal, the algorithm determines that the block is correctly identified and will not determine an error in that block.

[0057] More specifically, adjacent grid blocks, such as Figure 4 Blocks 408A-408D described in the diagram share common tiles. In other words, these blocks overlap. More specifically, Figure 4 Each block in the Cartesian lattice model described in the diagram has two nearest horizontal neighbors: to its left and right; and two nearest vertical neighbors: above and below. Two adjacent blocks have two overlapping tiles.

[0058] For the ring boundary condition, that is, if the left and right columns of the decoded tag are the same, as is the top, and the top and bottom rows of the tag are the same. In terms of error decoding capability, it is preferable to generate optical tags that satisfy these boundary conditions. This can be compared to chain codes in the one-dimensional example. In this Cartesian example, each tile is based on the four vertices associated with each tile belonging to four blocks. Each vertex forms the center of a block, and therefore each tile belongs to four blocks.

[0059] In an example where controller 104 decodes a block as 1 in one block to which the block belongs, but decodes it as 0 in the other three blocks to which the block belongs, an error is indicated. To correct this error, a two-dimensional grammar is applied. Figure 4 The example shown uses a two-dimensional grammar. G B2DC ,in B2DC Represents a two-dimensional Cartesian symbol.

[0060] Grid by Binary image express: here Applying this grammar is equivalent to transforming a binary image... Mapping to 16-bit image : This deterministic, simplified version can be demonstrated on a 2x2 block (the smallest block that produces a meaningful error syndrome in the Cartesian example).

[0061] For this single block, these four pixels do not overlap with other blocks, making it impossible to perform error correction based on a planar finite basis.

[0062] However, in vertical cylindrical operations, it is assumed that the first image column is equal to the last image column. In the grammar defined above, a modification must be made to reflect this: In the simplest In the case of image (4), Therefore, the first column can be taken as... Any value in the first column, and repeat it in the second column. Therefore, In other words, only four of the 16 possible values ​​are allowed, and the rest are prohibited. Therefore, if any other than these four values ​​are detected and even verified in the first step, an error is identified.

[0063] Similarly, in addition to or instead of vertical cylindrical operations, horizontal cylindrical operations can also be applied. In this case, it is assumed that the first image row is equal to the last image row.

[0064] In the simplest In the case of image (4), Therefore, the first row can be taken. Any value in the first column, and repeat it in the second column. Therefore, In addition to, or in place of, vertical and horizontal cylindrical operations, circular operations can also be applied, as described below. In this case, it is a combination of the aforementioned vertical and horizontal cylindrical operations: The number of boundary conditions is not limited to the three operations mentioned above. Other operations include the Möbius strip operation, or expanding the size of the block to obtain a fixed or reflective outer boundary. In general, if given... and boundary conditions Then except Any output other than this is considered an error. To correct for the error, the Hamming distance can be applied to determine the shortest allowed block in terms of Hamming distance. If the solution obtained using Hamming distance is ambiguous, probabilistic methods, such as Markov chains, can be used.

[0065] Similarly, under normal circumstances: Blocks in It is a 4-tuple surrounding a vertex. In In the case of blocks, each vertex belongs to 4 blocks. In the case of independently computed and self-verified blocks, the same symbols apply. In block , , , Each tile in the graph is calculated independently.

[0066] if If the value is the same for all four blocks of a tile, then there is no contradiction and no identification error. Otherwise, it is necessary to... Make corrections so that the minimum number of blocks affected by this change is minimized.

[0067] Therefore, self-compatibility and mutual compatibility checks are performed on overlapping blocks, and values ​​that may have changed due to noise are corrected accordingly. Figure 2 The method described herein can provide improved error correction compared to some error correction techniques used for optical marking.

[0068] Figure 3A , Figure 3B and Figure 3C The results of error correction for the generated optical marks are depicted using various techniques. For example, optical marks can be generated using a lattice model that includes a Cartesian grid, such as... Figure 2 and Figure 4 As depicted in the image. For example, such optical markings could be black and white fast response (“QR”) codes.

[0069] like Figure 3A As depicted in Figure 300, in the absence of noise, the distribution of the black and white image is represented by a first delta function 302 and a second delta function 304 on either side of the threshold line 306, where the first delta function 302 represents all black cells in the optical markers and the second delta function 304 represents all white cells in the optical markers. Therefore, the captured image can be easily processed based on the threshold line 306 using a unary operation such as a Naive Bayes classifier, where the unary operation is an operation that maps a polygon with all optically detected pixel values ​​to a single binary value.

[0070] Figure 3B Figure 310 in the figure is an example of the error correction result using unary operations when noise is added to an image with intentional optical marking. The intensity is... σ / D Zero-mean Gaussian noise (0.3, a standard type of noise used for modeling sensor noise, electronic circuit noise, etc.) is added to the optical labels. The Naive Bayes classifier uses automatic thresholding for each polygon in the optical labels, producing a roughly bimodal curve with a first peak 312 and a second peak 314 on either side of the thresholding line 316. However, the amount of overlap in the distribution is significant and may lead to imperfect error correction.

[0071] Figure 3C The figure 320 in the image uses binary operations to apply the same amount of noise to the optical mark. σ / D Example of error correction results (=0.3). Binary operations (such as the Wasserstein difference classifier) ​​improve upon the results of unary operators, producing a more pronounced first peak 322 and second peak 324 on either side of boundary 326. Binary operators (as opposed to thresholding in unary operators) compare polygons in pairs in the optical markers to determine which polygons are similar and which are dissimilar, and group the polygons in the optical markers together based on perceived similarity or dissimilarity. While this approach is an improvement over the unary operation depicted in Figure 310, error correction could be further improved.

[0072] Figure 2 The method described uses a two-dimensional grammar to create error syndromes, which can improve the identification of errors for this exemplary noise intensity. In some examples, generating error syndromes may involve using machine learning models (e.g., see...). Figure 8 ), such as convolutional neural networks, are used to detect prohibited combinations and assign weights and probabilities to the specific errors identified. Now, we will describe in more detail the specific types of lattice models and two-dimensional grammars. Figure 2 Examples of the methods described in [the document].

[0073] Figure 4 An example of a Cartesian grid optical marker 400 generated from a lattice model including a Cartesian grid is depicted. Figure 4 The Cartesian grid optical marker 400 depicted is based on a 12x12 lattice model defined by a Cartesian grid, which has a first length in a first horizontal direction and a second length in a second vertical direction perpendicular to the first direction. The lattice model comprises 144 tiles, such as tile 402 and tile 404. Each of the 144 tiles is encoded with a corresponding value based on a two-dimensional grammar (see, for example, [reference needed]). Figure 5 Patch 402 is encoded as white, for example, while patch 404 is encoded as black, for example. Some of the axis-aligned polygons, such as patch 406, will become part of overlapping blocks of the axis-aligned polygons (such as overlapping and adjacent blocks 408A-408D). Blocks 408A-408D may be subject to the rules of a two-dimensional grammar, which includes an alphabet 500 with allowed and prohibited combinations of blocks (see...). Figure 5 ).

[0074] Figure 5 An example of an alphabet 500 depicting a two-dimensional grammar, which can be applied to... Figure 4 and Figure 6 Example. In this example, the alphabet comprises all possible quadruples (i.e., groups of four tiles read sequentially from left to right and top to bottom) based on a binary code where white is "0" and black is "1". Each of these sixteen quadruples can be mapped to a single value, as follows: [ 0 0 | 0 1 ] = 1 [ 0 0 | 1 0 ] = 2 [ 1 0 | 1 1 ] = 3 [ 0 1 | 1 1 ] = 4 [ 0 1 | 0 0 ] = 5 [ 1 0 | 0 0 ] = 6 [ 1 1 | 1 0 ] = 7 [ 1 1 | 0 1 ] = 8 [ 1 0 | 0 1 ] = 9 [ 0 1 | 1 0 ] = 10 [ 0 0 | 1 1 ]= 11 [ 1 1 | 0 0 ] = 12 [ 0 1 | 0 1 ] = 13 [ 1 0 | 1 0 ] = 14 [ 1 1 | 1 1 ] = 15 [ 0 0 | 0 0 ] = 16 like Figure 4 The text describes blocks 408A-408D as an example of how a two-dimensional grammar can be applied to determine if an error exists in any of the blocks within a block (such as block 406). Block 408A, defined by block 406 in the lower right corner, includes the quadruple [00 | 1 0], corresponding to... Figure 5 The quadruple 2 in alphabet 500 described in the diagram. Block 408B, defined by the lower left corner tile 406, includes the quadruple [0 0 | 0 0], corresponding to quadruple 16 in alphabet 500. Block 408C, defined by the upper right corner tile 406, includes the quadruple [1 0 | 1 0], or quadruple 14. Finally, block 408D, defined by the upper left corner tile 406, also includes the quadruple [0 0 | 0 0], or quadruple 16.

[0075] Therefore, patch 406 is included in the overlapping patches 408A-408D. The value of patch 406 is determined by processing the portion of the image mapped to that patch. One processing mode is to use a thresholding operator, a unary operator used to map the average value of the image to a discrete scale to obtain a quantized output. Figure 4 In the binary case shown, if the mean of all pixel values ​​is higher than a given threshold, the value of patch 406 is determined to be 1; otherwise, it is determined to be 0.

[0076] A binary operator can also be introduced, which accepts two operands (V1, V2) such that, in the case of a binary alphabet, the values ​​of two tiles are determined to be either the same or different. This operator outputs 0 for the tuple pair (1,1) and (0,0), and 1 for the tuple pair (1, 0) and (1, 0). Unary and binary operators can be evaluated independently.

[0077] Therefore, for two blocks belonging to the same block, such as blocks 406 and 410 in block 408A, the unary operator will output 1 for both blocks, while the binary operator will output zero; that is, the binary operator should indicate that the blocks have the same value. In this case, there is no contradiction: the values ​​1 and 1 are the same. However, if the unary operator outputs 1 for both blocks, while the binary operator outputs 1 instead of zero, a contradiction exists. The equal values ​​1 and 1 are considered different. Therefore, blocks containing such contradictions are determined to be prohibited. Analogous to a spell checker, there is no such "word" in the vocabulary.

[0078] The dimension of the output set of a unary operation is equal to |K| — the dimension of the alphabet in the formal language. For example, Figure 4The Cartesian coordinate system shown corresponds to 16, as... Figure 5 As shown. Similarly, binary operations on this alphabet are either the difference between two values ​​or the modulus of that difference. Binary operators used for tiles have an additional value |K|=2.

[0079] The four unary operators on the block's binary components generate 16 values: Given the same four components, there are at most 64 comparisons between all possible tuples in the form of binary operators, by ; Given. Generally, for a given regular cell, with a given dimension and a cardinality of , Finite alphabet Given a structure (i.e., the number of lines, faces, hyperplanes, etc., for a block containing m tiles), there are at most m unary operations that map the block to... There are at most discrete values, and at most A binary operation maps the block to There are discrete values. Therefore, these two operations are applied together but independently to map the block to a dimension of . The matrix. This matrix is ​​the error syndrome.

[0080] The error syndrome for a 2x2 block on a Cartesian grid can be represented by a matrix as large as 16x64. This is the largest dimension to consider all possible binary operators. Only 16 cells in the matrix correspond to the allowed “words” in the alphabet; all other cells are forbidden. Therefore, for block 406, a unary operation is performed, and the block is compared pairwise with each of the blocks directly adjacent to it horizontally and vertically. Subsequently, a binary operation is used to compare the block with its neighboring blocks, and any inconsistencies will result in an error syndrome based on the alphabetical indication of whether an inconsistency exists in block 406. This process can be repeated for each block in the Cartesian grid optical marker 400 to perform a “spell check” on the captured image of the optical marker and create an error syndrome for potential errors in the recognition of all blocks. In the case of a single error in recognition, error correction involves switching the values ​​of the values ​​altered by noise. However, if two or more errors are identified, an error syndrome can be used to identify the appropriate error correction to be performed by considering the probability of falling from one of the two permitted symbols equidistant from the prohibited symbol, i.e., using distances with weights proportional to the corresponding probabilities.

[0081] If prohibited combinations are determined for an optical marker, these combinations can be translated into permitted combinations based on the minimum Hamming distance, a probabilistic method that uses the fewest number of corrections to correct errors. If two or more possible corrections exist, a weighted Hamming distance with weights reflecting different probabilities of different errors is used to eliminate ambiguity. Permitted combinations have a Hamming distance of zero from each other. In cases where it is not possible to determine in advance which error is more likely, other methods can be applied, such as applying a Markov chain, until the result collapses into a stable, resolved state. Controller 104 can apply machine learning models (e.g., see...). Figure 8 ), such as convolutional neural networks, to detect errors and the probabilities required for error correction.

[0082] Alternatively, a tile can be compared pairwise with adjacent tiles that share a boundary. For example, tile 410 can be compared with adjacent tiles 412A, 412B, 412C, and 412D. If the value of tile 410 has been altered by noise, that value can be compared with adjacent tiles 412A, 412B, 412C, and 412D that have not been altered by noise to determine how similar or different the altered value of tile 410 is to adjacent axis-aligned polygons with known values. This may yield a probabilistic estimate of the corrected value of tile 410.

[0083] Figure 6 An example of a curved optical marker 600 generated from a lattice model comprising multiple concentric circles and radial lines is depicted. The curved optical marker 600 is based on a lattice model defined by multiple concentric circles and multiple radial lines passing through the centers of the concentric circles to produce an axis-aligned polygon comprising two straight lines and two arcs opposite each other, except for the central polygon sharing the lattice center. The central polygon defines a wedge comprising an arc and two straight edges.

[0084] Example curve optical marker 600 includes 60 tiles, such as tile 602 and tile 604. (Compared to...) Figure 4 Similar to the Cartesian grid optical marker 400, each of the 60 axis-aligned polygons is encoded with a corresponding value based on a two-dimensional grammar. Patch 602, for example, is encoded as white, while patch 604, for example, is encoded as black. Some of the patches, such as patch 606, will become part of overlapping blocks (such as overlapping and adjacent blocks 608A-608D). Blocks 608A-608D may be subject to the rules of the two-dimensional grammar, which includes an alphabet 500 with allowed and prohibited combinations of blocks (see...). Figure 5 ).

[0085] Despite the different shapes, the grammar applied to Cartesian grid optical markings 400 is also applicable to curved optical markings 600. For example... Figure 6 The text describes how blocks 608A-608D are illustrated as an example of how a two-dimensional grammar can be applied to determine if any of the tiles within a block (such as tile 606) contains an error. The value of tile 606 is determined by processing the portion of the image mapped to that tile. One processing mode is to use a thresholding operator, a unary operator used to map the average of the image to a discrete scale to obtain a quantized output. Figure 6 In the binary case shown, if the mean of all pixel values ​​is higher than a given threshold, the value of patch 606 is determined to be 1; otherwise, it is determined to be 0.

[0086] A binary operator can also be introduced, which accepts two operands (V1, V2) such that, in the case of a binary alphabet, the values ​​of two tiles are determined to be either the same or different. This operator outputs 0 for the tuple pair (1,1) and (0,0), and 1 for the tuple pair (1, 0) and (1, 0). Unary and binary operators can be evaluated independently.

[0087] Therefore, for two blocks belonging to the same block, such as blocks 406 and 410 in block 408A, the unary operator will output 1 for both blocks, while the binary operator will output zero; that is, the binary operator should indicate that the blocks have the same value. In this case, there is no contradiction: the values ​​1 and 1 are the same. However, if the unary operator outputs 1 for both blocks, while the binary operator outputs 1 instead of zero, a contradiction exists. The equal values ​​1 and 1 are considered different. Therefore, blocks containing such contradictions are determined to be prohibited. As far as the analogy spell checker is concerned, there is no such "word" in the vocabulary.

[0088] The dimension of the output set of a unary operation is equal to |K| — the dimension of the alphabet in the formal language. For example, Figure 4 The Cartesian coordinate system shown corresponds to 16, as... Figure 5 As shown. Similarly, binary operations on this alphabet are either the difference between two values ​​or the modulus of that difference. Binary operators used for tiles have an additional value |K|=2.

[0089] Four unary operators on the binary components of a block generate 16 values: 2^4 = 16. Given the same four block components, there are at most 64 comparisons between all possible tuples in the form of binary operators, given by C(4,2) = 4! / (4-2)!2! = 6; 2^C(4,2) = 2^6 = 64. In general, for a given regular lattice, a given dimension, a finite alphabet K = {0,…,l-1} with cardinality |K| = l, l∈N_+, l<+∞, and a given structure (i.e., the number of lines, faces, hyperplanes, etc. for a block containing m tiles), there are at most m unary operations that map the block to l^m discrete values, and at most C(m,2) binary operations that map the block to l^C(m,2) discrete values. Thus, these two operations, applied together but independently, map the block to a matrix of dimension (l^C(m,2), l^m). This matrix represents the error syndrome.

[0090] The error syndrome for a 2x2 block on a Cartesian lattice can be represented by a matrix as large as 16x64. This is the largest dimension to consider all possible binary operators. Only 16 cells in the matrix correspond to the allowed “words” in the alphabet; all other cells are forbidden. Therefore, for block 606, a unary operation is performed, and the block is compared pairwise with each of the blocks directly adjacent to it horizontally and vertically. Subsequently, a binary operation is used to compare the block with its neighboring blocks, and any inconsistencies will result in an error syndrome based on the alphabetical indication of whether an inconsistency exists in block 606. This process can be repeated for each block in the curved optical markers 600 to perform a “spell check” on the captured image of the optical markers and create an error syndrome for all blocks with potential recognition errors. In the case of a single recognition error, error correction involves switching the values ​​of the values ​​altered by noise. However, if two or more errors are identified, an error syndrome can be used to identify the appropriate error correction to be performed by considering the probability of falling from one of the two allowed symbols equidistant from the prohibited symbol; that is, using distances with weights proportional to the corresponding probabilities. The controller 104 can apply machine learning models (e.g., see...). Figure 8 ), such as convolutional neural networks, to detect errors and the probabilities required for error correction.

[0091] Alternatively, a tile can be compared pairwise with adjacent tiles that share a boundary. For example, tile 610 can be compared with adjacent tiles 612A, 612B, 612C, and 612D. If the value of tile 610 has been altered by noise, it can be compared with adjacent tiles 612A, 612B, 612C, and 612D that have not been altered by noise to determine how similar or different the altered value of tile 610 is from adjacent axis-aligned polygons with known values. This may yield a probabilistic estimate of the corrected value of tile 610.

[0092] Generally speaking, any process or operation discussed in this disclosure can be implemented by a computer, such as... Figure 2 The process or reference shown Figures 4-6 The process described herein can be executed by one or more processors of a computer system. A process or process step executed by one or more processors may also be called an operation. One or more processors can be configured to execute such processes by accessing instructions (e.g., software or computer-readable code) that, when executed by one or more processors, cause one or more processors to perform these processes. These instructions may be stored in the memory of the computer system. A processor may be a central processing unit (CPU), a graphics processing unit (GPU), or any suitable type of processing unit.

[0093] A computer system (such as a system or apparatus that implements a process or operation as described in the examples above) may include one or more computing devices. One or more processors of a computer system may be located in a single computing device or may be distributed across multiple computing devices. The memory of a computer system may include the respective memory of each of the multiple computing devices.

[0094] Figure 7 Implementations of a computer system performing the techniques described herein, according to one or more embodiments, are depicted. Computer system 700 may include a set of instructions executable to cause computer system 700 to perform any or more of the methods or computer-based functions disclosed herein. Computer system 700 operates as a standalone device or, for example, is connected to other computer systems or peripheral devices using a network.

[0095] Unless otherwise expressly stated, as will be apparent from the discussion below, it should be understood that in this specification, the use of terms such as “processing,” “computing,” “calculating,” “determining,” “analyzing,” etc., refers to the actions and / or processes of a computer or computing system or similar electronic computing device that manipulate and / or transform data represented as physical quantities (such as electronic quantities) into other data similarly represented as physical quantities.

[0096] In a similar manner, the term "processor" refers to any means or part of a means of processing electronic data (e.g., electronic data from registers and / or memory) to transform that electronic data into other electronic data, such as data stored in registers and / or memory. "Computer," "computing machine," "computing platform," "computing device," "controller," or "server" includes one or more processors.

[0097] In a networked deployment, computer system 700 operates as a server or as a client user computer in a server-client user network environment, or as a peer-to-peer (or distributed) computer system in a peer-to-peer (or distributed) network environment. Computer system 700 can also be implemented as or integrated into various devices, such as personal computers (PCs), tablet PCs, set-top boxes (STBs), personal digital assistants (PDAs), mobile devices, handheld computers, notebook computers, desktop computers, communication devices, wireless telephones, landline telephones, control systems, cameras, scanners, fax machines, printers, pagers, personal trusted devices, network equipment, network routers, switches or bridges, or any other machine capable of executing a set of instructions (sequentially or otherwise) specifying the actions to be performed by that machine. In certain implementations, computer system 700 may be implemented using electronic devices that provide voice, video, or data communication. Furthermore, although computer system 700 is shown as a single system, the term "system" should also be considered as any collection including systems or subsystems that individually or jointly execute one or more sets of instructions to perform one or more computer functions.

[0098] like Figure 7 As shown, computer system 700 includes a processor 702, such as a central processing unit (CPU), a graphics processing unit (GPU), or both. Processor 702 can be a component in various systems. For example, processor 702 is part of a standard personal computer or workstation. Processor 702 is one or more processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), servers, networks, digital circuits, analog circuits, combinations thereof, or other means now known or later developed for analyzing and processing data. Processor 702 implements software programs, such as manually generated code (e.g., programmed code).

[0099] Computer system 700 includes memory 704, which can communicate via bus 708. Memory 704 is main memory, static memory, or dynamic memory. Memory 704 includes, but is not limited to, computer-readable storage media, such as various types of volatile and non-volatile storage media, including but not limited to random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, magnetic tape or disk, optical media, etc. In one implementation, memory 704 includes cache or random access memory for processor 702. In alternative implementations, memory 704 is separate from processor 702, such as processor cache memory, system memory, or other memory. Memory 704 may be an external storage device or database for storing data. Examples include hard disks, optical discs (“CDs”), digital video discs (“DVDs”), memory cards, memory sticks, floppy disks, universal serial bus (“USB”) storage devices, or any other device that operates for storing data. Memory 704 is operable to store instructions executable by processor 702. The functions, behaviors, or tasks shown in the figures or described herein are executed by processor 702 executing instructions stored in memory 704. These functions, behaviors, or tasks are independent of a specific type of instruction set, storage medium, processor, or processing strategy, but are executed individually or in combination by software, hardware, integrated circuits, firmware, microcode, etc. Similarly, processing strategies may include multiprocessing, multitasking, parallel processing, etc.

[0100] As depicted, the computer system 700 further includes a display 710, such as a liquid crystal display (LCD), an organic light-emitting diode (OLED), a flat panel display, a solid-state display, a cathode ray tube (CRT), a projector, a printer, or other display devices now known or developed in the future for outputting specific information. The display 710 serves as an interface for a user to view the operation of the processor 702, or more specifically, as an interface for interacting with software stored in the memory 704 or the drive unit 706.

[0101] Alternatively or concurrently, the computer system 700 includes an input / output device 712 configured to allow a user to interact with any of the components of the computer system 700. The input / output device 712 is a numeric keypad, keyboard, or cursor control device, such as a mouse, joystick, touchscreen display, remote control, or any other device operated to interact with the computer system 700.

[0102] Computer system 700 also includes, or alternatively includes, a drive unit 706, implemented as a disk drive or optical drive. Drive unit 706 includes a computer-readable medium 722 in which one or more sets of instructions 724, such as software, can be embedded. Further, the set of instructions 724 embodies one or more of the methods or logic described herein. Instructions 724 reside wholly or partially within memory 704 and / or processor 702 during execution of computer system 700. As discussed above, memory 704 and processor 702 may also include computer-readable media.

[0103] In some systems, computer-readable medium 722 includes a set of instructions 724, or receives and executes a set of instructions 724 in response to a propagated signal, enabling devices connected to network 106 to transmit voice, video, audio, images, or any other data through network 106. Further, the set of instructions 724 is transmitted or received through network 106 via a communication port or interface 720, and / or via bus 708. The communication port or interface 720 is part of processor 702 or a separate component. The communication port or interface 720 is created in software or is a physical connection in hardware. The communication port or interface 720 is configured to connect to network 106, external media, display 710, or any other component or combination thereof in computer system 700. The connection to network 106 is a physical connection, such as a wired Ethernet connection, or wirelessly established as discussed below. Similarly, additional connections to other components of computer system 700 are either physical connections or wirelessly established. Network 106 is alternatively directly connected to bus 708.

[0104] Although computer-readable medium 722 is described as a single medium, the term "computer-readable medium" includes single or multiple media, such as centralized or distributed databases, and / or associated caches and servers storing one or more sets of instructions. The term "computer-readable medium" also includes any medium capable of storing, encoding, or carrying a set of instructions for execution by a processor or to cause a computer system to perform any or more of the methods or operations disclosed herein. In some examples, computer-readable medium 722 is non-transitory and tangible.

[0105] Computer-readable medium 722 may include solid-state memory, such as a memory card, or other package housing one or more non-volatile read-only memories. Computer-readable medium 722 may be random access memory or other volatile rewritable memory. Alternatively or additionally, computer-readable medium 722 may include magneto-optical or optical media, such as magnetic disks or magnetic tapes, or other storage devices, to capture carrier signals, such as signals transmitted via a transmission medium. Digital file attachments to emails or other self-contained archives or sets of information are considered as distribution media as tangible storage media. Therefore, this disclosure is considered to include one or more of computer-readable media or distribution media, as well as other equivalent media and subsequent media, on which data or instructions may be stored.

[0106] In alternative implementations, specialized hardware implementations, such as application-specific integrated circuits (ASICs), programmable logic arrays (PLA), and other hardware devices, can be constructed to implement one or more of the methods described herein. Applications of devices and systems incorporating various implementations can broadly encompass a wide range of electronic and computer systems. One or more implementations described herein utilize two or more specific interconnected hardware modules or devices to implement functionality. These modules or devices have associated control and data signals that can communicate between and through modules, or function as part of an ASIC. Therefore, this system includes software, firmware, and hardware implementations.

[0107] Computer system 700 is connected to network 106. Network 106 defines one or more networks, including wired or wireless networks, such as... Figure 1 Network 106 is described herein. Wireless networks may be cellular telephone networks, 702.11, 702.18, 702.20, or WiMAX networks. Further, such networks include public networks (such as the Internet), private networks (such as intranets), or combinations thereof, and utilize a variety of existing or later-developed networking protocols, including but not limited to TCP / IP-based networking protocols. Network 106 may include a wide area network (WAN), such as the Internet, a local area network (LAN), a campus network, a metropolitan area network, a direct connection (such as via a universal serial bus (USB) port), or any other network that allows data communication. Network 106 is configured to couple one computing device to another to enable data communication between the devices. Network 106 is generally capable of using any form of machine-readable medium for transferring information from one device to another. Network 106 includes communication methods for the travel of information between computing devices. Network 106 may be divided into subnets. Subnets may allow access to all other components connected to them, or subnets may restrict access between components. Network 106 can be considered a public or private network connection and may include, for example, a virtual private network or encryption or other security mechanisms employed over the public Internet.

[0108] As disclosed herein, one or more implementations disclosed herein can be applied using machine learning models. The machine learning models disclosed herein can be used... Figures 1-6 One or more components or operations in the process are trained. For example... Figure 8 As depicted in flowchart 800, training data 812 may include one or more of stage inputs 814 and known results 818 associated with the machine learning model to be trained. Stage inputs 814 may come from any applicable source, including the components or sets of components depicted in the figures provided herein. For machine learning models generated based on supervised or semi-supervised training, known results 818 may be included. Unsupervised machine learning models may not be trained using known results 818. Known results 818 may include known or expected outputs for future inputs similar to or belonging to the same category as stage inputs 814, but for which there are no corresponding known outputs.

[0109] Training data 812 and training algorithm 820 can be provided to training component 830, which can apply training data 812 to training algorithm 820 to generate a trained machine learning model 850. Depending on the implementation, training component 830 can be provided with a comparison result 816, which is compared with the previous output of the corresponding machine learning model, so as to apply the previous result to retrain the machine learning model. Training component 830 can use comparison result 816 to update the corresponding machine learning model. Training algorithm 820 can utilize machine learning networks and / or models, including but not limited to deep learning networks (such as deep neural networks (DNN), convolutional neural networks (CNN), fully convolutional networks (FCN), and recurrent neural networks (RCN)), probabilistic models (such as Bayesian networks and graph models), and / or discriminative models (such as decision forests and maximum margin methods).

[0110] The machine learning models disclosed herein can be trained by adjusting one or more weights, layers, and / or biases during the training phase. During the training phase, historical or simulated data can be provided as input to the model. The model can adjust one or more of its weights, layers, and / or biases based on such historical or simulated information. Based on the training, the adjusted weights, layers, and / or biases can be configured in a production version of the machine learning model (e.g., a trained model). Once trained, the machine learning model can output machine learning model outputs according to the topics disclosed herein. Depending on the implementation, one or more machine learning models disclosed herein can be continuously updated based on feedback associated with the use or implementation of the machine learning model outputs.

[0111] Figure 9Examples of two-dimensional lattice models and their associated blocks are depicted. Lattice model 910 is based on a Cartesian grid model formed by two sets of lines to form blocks 912, and each block 912 is a square and part of four blocks. An example of four blocks is shown in block 915, which comprises four blocks. Lattice model 920 is formed by three sets of lines to form blocks 922, and each block 922 is a triangle and part of three blocks. An example of three blocks is shown in block 925, which comprises six blocks. Lattice model 930 is formed by three sets of lines to form blocks 932, and each block 932 is a hexagon and part of six blocks. An example of six blocks is shown in block 935, which comprises three blocks. The number of blocks to which a block is a part is given by the number of vertices connected to that block. Thus, square blocks are part of four blocks, triangular blocks are part of three blocks, and hexagonal blocks are part of six blocks. The larger the dimension of the block, the smaller the ratio of allowed to prohibited combinations. Therefore, the potential for correction capability increases with the dimensionality of the block.

[0112] In the lattice model 920 with triangular tiles 922, six triangular tiles 922 converge to form a node or vertex, such that the dimension of the node syndrome will be larger than that of the Cartesian mesh-based lattice model 910 (where four rectangular tiles 912 converge to form a node or vertex). Therefore, the first-stage correction capability of this node or vertex (as described above) is higher than that of the Cartesian lattice node or vertex. The method described in this disclosure may include generating a lattice model comprising a plurality of axis-aligned polygons, wherein the lattice model comprises a regular triangular mesh as described in lattice model 920.

[0113] In a lattice model 930 with hexagonal tiles 932, each hexagonal tile 932 in the grid has six adjacent hexagonal tiles 932, such that the correction capability (second-stage correction as described above) will be higher than that of the Cartesian lattice model 910. The method described in this disclosure may include generating a lattice model comprising a plurality of axis-aligned polygons, wherein the lattice model comprises a regular hexagonal grid as described in lattice model 930.

[0114] Piece values ​​are treated as letters in a given alphabet, and blocks represent “words” to be checked for correctness. When a “word” is not present in the “vocabulary” of a two-dimensional grammar, it is identified as a contradiction and must be replaced. Block 925 may have an error syndrome matrix of up to 64×32768, while block 935 has a maximum error syndrome matrix of 8x6.

[0115] For the lattice model 910, tiles have natural coordinates [i, j], similar to matrix indices. These coordinates naturally correspond to the center of the tile. The same coordinate system is used for raster images, such that pixels generally have the same coordinate system. Boundary lines lie between tiles, such that boundary lines have coordinates in half-step increments (e.g., 0.5, 1.5, ..., 12.5). Therefore, lattice vertices (i.e., the intersections of lines) have coordinates in half-step increments, such as [2.5, 1.5] in one example. Thus, a tile centered at that vertex is mapped to Cartesian coordinates of [2.5, 1.5].

[0116] The simplest block for this structure is 2x2, meaning that given any vertices, a block contains 4 vertices. The coordinates of the vertices can be sorted into a matrix using two-dimensional indices. For example, the value of the top-left vertices can be stored in the matrix as cell (1, 1). Geometrically, vertices are placed between the vertices, so the geometric coordinates of the blocks mapped to vertices will have a shift of (0.5, 0.5).

[0117] Therefore, a block has four tiles with the following coordinates: (I, J), (I, J+1); (I+1, J), (I+1, J+1).

[0118] A vertex is the intersection of the lines that form a tile. Therefore, the vertex is located at the very center and in contact with all four tiles, thus the vertex has coordinates (i+0.5, j+0.5).

[0119] The next block to the right has coordinates (i+0.5, j+1.5): the second coordinate plus 1. This block contains the following tiles: (I, J+1), (I, J+2); (I+1, J+1), (I+1, J+2).

[0120] Therefore, tiles (I, J+1) and (I+1, J+1) are shared tiles between these two adjacent tiles, and the tiles are overlaps between the tiles.

[0121] Figure 10 An example irregular lattice model according to one or more embodiments is depicted. Figure 10The irregular, aperiodic character submodels described above can also be described within the formal framework of the grammar mentioned above. As mentioned above, the first-stage correction is related to the number of tiles that meet at vertices. An aperiodic character submodel can have a varying number of tiles that meet at vertices throughout the model. For example, lattice model 1010 has three tiles 1012 meeting at vertex 1013 and four tiles 1012 meeting at vertex 1015. Lattice model 1010 is defined by tiles 1012 of a shape that repeats aperiodically. Lattice model 1020 includes a first tile shape 1022 and a second tile shape 1024 that repeat in a varying manner to form aperiodic character submodel 1020. Similar to lattice model 1010, the number of tiles that meet at nodes varies at the vertices of lattice model 1020. For example, three tiles meet at vertex 1023, while five tiles meet at vertex 1025. Figure 10 The depicted features a varying number of tiles at other vertices that meet at that vertex.

[0122] The lattice model 1030 consists of three different tile shapes: a first tile shape 1032, a second tile shape 1034, and a third tile shape 1036. However, all vertices in the lattice model 1030 have the same number of tiles meeting at that vertex (i.e., four). For example, four tiles meet at vertex 1033, and four tiles meet at vertex 1035.

[0123] It should be understood that, in one embodiment, the operation of the discussed method is performed by a suitable processor (or processors) of a processing (e.g., a computer) system executing instructions (computer-readable code) stored in a storage device. It should also be understood that this disclosure is not limited to any particular implementation or programming technique, and that any suitable technique may be used to implement the functionality described herein. This disclosure is not limited to any particular programming language or operating system.

[0124] It should be understood that in the above description of exemplary embodiments of this disclosure, various features of this disclosure are sometimes grouped together in a single embodiment, drawing, or description thereof in order to simplify the disclosure and aid in understanding one or more of the various inventive aspects. However, this approach of the disclosure should not be construed as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as reflected in the appended claims, inventive aspects lie in fewer features than all the features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are expressly incorporated herein by reference, wherein each claim itself is a separate embodiment of this disclosure.

[0125] Furthermore, while some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this disclosure and to form different embodiments that should be understood by those skilled in the art. For example, any of the claimed embodiments in the following claims may be used in any combination.

[0126] Furthermore, this document describes some of the embodiments as methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing functions. Therefore, a processor having instructions required to perform such methods or method elements forms a means for performing such methods or method elements. Furthermore, the elements of the device embodiments described herein are examples of means for performing the functions performed by those elements in carrying out the embodiments.

[0127] Many specific details are set forth in the description provided herein. However, it should be understood that embodiments of this disclosure can be practiced without these specific details. In other instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.

[0128] Therefore, while preferred embodiments considered to be those of this disclosure have been described, those skilled in the art will recognize that other and further modifications may be made thereto without departing from the spirit of the embodiments, and all such variations and modifications falling within the scope of the embodiments are intended to be claimed. For example, any formulas given above merely represent usable procedures. Functions may be added or removed from the block diagrams, and operations may be interchanged within functional blocks. Steps may be added or removed from the described methods within the scope of this disclosure.

[0129] The subject matter disclosed above should be considered illustrative rather than restrictive, and the appended claims are intended to cover all such modifications, enhancements, and other implementations that fall within the true spirit and scope of this disclosure. Therefore, to the maximum extent permitted by law, the scope of this disclosure is determined by the broadest possible interpretation of the appended claims and their equivalents, and should not be construed as limited by the foregoing detailed descriptions. While various implementations of this disclosure have been described, it will be apparent to those skilled in the art that many implementations and variations are possible within the scope of this disclosure. Therefore, this disclosure is not limited except as provided in the appended claims and their equivalents.

Claims

1. A computer-implemented method, comprising: Generate a grid model that includes multiple axis-aligned polygons; The corresponding values ​​of the plurality of axis-aligned polygons are encoded using a two-dimensional grammar, wherein the two-dimensional grammar includes a plurality of allowed value combinations for the plurality of axis-aligned polygons; Optical markers are generated using the plurality of axis-aligned polygons with encoded values; Receive the image of the optical marker; An error syndrome is generated using the two-dimensional grammar for the image of the optical marker; and The error syndrome is used to generate an error-corrected image of the optical marker.

2. The computer-implemented method of claim 1, wherein generating the lattice model comprising the plurality of axis-aligned polygons comprises generating one or more of the following: a regular Cartesian grid having a first length along a first direction and a second length along a second direction perpendicular to the first direction; a regular triangular grid; or a regular hexagonal grid.

3. The computer-implemented method of claim 1, wherein generating the lattice model comprises generating: (i) a plurality of concentric circles, and (ii) a plurality of lines passing through the centers of the plurality of concentric circles.

4. The computer-implemented method of claim 1, wherein the plurality of allowed value combinations of the two-dimensional grammar corresponds to the combination of adjacent axis-aligned polygons of the plurality of axis-aligned polygons, wherein the adjacent axis-aligned polygons share a boundary.

5. The computer-implemented method of claim 1, wherein the plurality of allowed value combinations of the two-dimensional grammar corresponds to the combination of adjacent axis-aligned polygons of the plurality of axis-aligned polygons, wherein the adjacent axis-aligned polygons share vertices.

6. The computer-implemented method of claim 1, wherein the two-dimensional grammar further includes a plurality of forbidden value combinations for the plurality of axis-aligned polygons.

7. The computer-implemented method of claim 1, further comprising: The image of the optical marker is captured using the aircraft's camera; The image of the optical marker is received from the camera.

8. The computer-implemented method of claim 1, wherein generating the error syndrome includes determining values ​​among the encoded values ​​that have been altered by noise.

9. The computer-implemented method of claim 8, wherein generating the error-corrected image of the optical marker includes changing the value altered by the noise.

10. The computer-implemented method of claim 1, wherein generating the error syndrome includes using a convolutional neural network.

11. A computer-implemented method, comprising: Generate a grid model that includes multiple axis-aligned polygons; The values ​​of the plurality of axis-aligned polygons are encoded using a two-dimensional grammar, wherein the two-dimensional grammar includes a plurality of allowed value combinations for the plurality of axis-aligned polygons; and Optical markers are generated using the plurality of axis-aligned polygons with encoded values.

12. The computer-implemented method of claim 11, further comprising: Receive the image of the optical marker; An error syndrome for generating the image of the optical marker using the two-dimensional grammar; as well as The error syndrome is used to generate an error-corrected image of the optical marker.

13. The computer-implemented method of claim 11, wherein generating the lattice model comprises generating: (i) a plurality of concentric circles, and (ii) a plurality of lines passing through the centers of the plurality of concentric circles.

14. The computer-implemented method of claim 11, wherein the plurality of allowed value combinations of the two-dimensional grammar corresponds to a combination of adjacent axis-aligned polygons of the plurality of axis-aligned polygons, wherein the adjacent axis-aligned polygons share a boundary.

15. The computer-implemented method of claim 11, wherein the plurality of allowed value combinations of the two-dimensional grammar corresponds to a combination of adjacent axis-aligned polygons of the plurality of axis-aligned polygons, wherein the adjacent axis-aligned polygons share vertices.

16. A computer-implemented method, comprising: Receive an image of an optical marker, wherein the image comprises a plurality of axis-aligned polygons having corresponding encoded values; An error syndrome is generated for the image of the optical marker using a two-dimensional grammar, wherein the two-dimensional grammar includes multiple allowed value combinations for the plurality of axis-aligned polygons; as well as The error syndrome is used to generate an error-corrected image of the optical marker.

17. The computer-implemented method of claim 16, further comprising: Generate a lattice model that includes the plurality of axis-aligned polygons; The corresponding values ​​of the plurality of axis-aligned polygons are encoded using the two-dimensional grammar as the corresponding encoded values; as well as The optical mark is generated using the plurality of axis-aligned polygons having the encoded values.

18. The computer-implemented method of claim 16, wherein generating the error syndrome includes determining values ​​among the encoded values ​​that have been altered by noise.

19. The computer-implemented method of claim 18, wherein generating the error-corrected image of the optical marker includes changing the value altered by the noise.

20. The computer-implemented method of claim 16, further comprising: The image of the optical marker is captured using the aircraft's camera; The image of the optical marker is received from the camera.

Images