A method for identifying at least one degree of freedom of a camera, a computer program, a machine-readable storage medium, and an electronic control unit or automation device.

Abstract

The present invention relates to a method for identifying at least one degree of freedom of camera (1) relative to a code array (2) from a camera image of camera (1), wherein the code array (2) has a point grid with a plurality of basic symbols (20), the coarse positions of the basic symbols (20) are encoded in the code array (2), a start field (28) having basic symbols (20) is identified, the start field (28) has at least three basic symbols (20), two independent principal directions along the point grid in the camera image are estimated via the basic symbols (20) of the start field (2), and the basic symbols (20) of the start field are valid basic symbols The present invention relates to a method for forming a code array, in which, in a search step, further basic symbols (20) are searched along at least one of the principal directions, starting from at least one valid basic symbol (20), and if the search is successful, the further basic symbols (20) are characterized as valid basic symbols (20), the search step is performed multiple times, and based on the valid basic symbols (20) or a subset thereof, the coarse position of at least one of the basic symbols (20) in the code array is identified, and / or the coarse position of the coordinate system of the point grid is identified as two degrees of freedom of the camera relative to the code array (2).

Description

[Technical Field]

[0001] Conventional technology The present invention relates to a method for identifying at least one degree of freedom of a camera relative to a code sequence in a camera image of the camera, as described in claim 1. The present invention also relates to a computer program, a machine-readable storage medium equipped with the computer program, and an electronic control unit or an automated device equipped with an electronic control unit. [Background technology]

[0002] In modern manufacturing and other applications, it is necessary to detect the relative position of a moving object to its surroundings. For this purpose, numerous sensors exist, such as displacement sensors, and encoded position sensors. All existing sensors have unique advantages that can be based on their accuracy, simplicity, low-cost integration, robustness, or other characteristics, depending on the application.

[0003] In the present applicant's German Patent Application Publication No. 102016216221, which forms the closest prior art, a method is proposed for positioning and / or locating objects in space and / or on a surface using a two-dimensional code array. The two-dimensional code array has a plurality of basic symbols, which are arranged on a surface, and these basic symbols on the surface form a two-dimensional periodic grid. Based on the code array, the position of a camera relative to the code array can be determined.

[0004] The present applicant's German Patent Application Publication No. 102016216196 discloses a sensor system which utilizes a method for positioning and / or locating a prior publication. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] German Patent Application Publication No. 102016216221 [Patent Document 2] German Patent Application Publication No. 102016216196 Specification [Overview of the project] [Means for solving the problem]

[0006] Disclosure of the invention A method for determining at least one degree of freedom of a camera relative to a code sequence from a camera image of a camera, having the features of claim 1, is proposed, along with an electronic control unit or an automated device equipped with an electronic control unit, a computer program and a machine-readable storage medium, each having the features of an independent claim. Preferred or advantageous embodiments of the present invention will become apparent from the dependent claims, the following description and the accompanying drawings.

[0007] The method according to the present invention is used to identify at least one degree of freedom of a camera relative to a code array from a camera image of the camera.

[0008] This method is based on the method described in German Patent Application Publication No. 102016216221 and / or German Patent Application Publication No. 102016216196, and the disclosures of these specifications are incorporated herein by reference.

[0009] Definition of a camera: The camera includes an optical sensor unit (also referred to as an imaging sensor element, camera chip, or imaging device) for capturing camera images, and an imaging optical system. In particular, the camera is configured as a color camera or a monochrome camera. The optical sensor unit is configured to provide a single camera image of one segment of a code sequence, and / or to provide a sequence of multiple camera images, for example, in the form of video. The camera has an imaging device as an optical sensor unit. The camera may include an objective lens, which is preferably configured as a wide-angle objective lens. In particular, the camera employs vanishing point perspective and / or one-point perspective. In this case, parallel edges, in particular, are not displayed parallel to the image when viewed obliquely, but are optically focused at a single virtual point, a so-called vanishing point. In particular, cameras that include an objective lens implement desired perspective distortion resulting from use.

[0010] Definition of optical axis 5: An imaging optical system is ideally a rotationally symmetric optical system, where the axis of symmetry is the optical axis. An imaging optical system is characterized by the fact that light rays do not experience deflection along the optical axis as they pass through the optical system.

[0011] Definition of Image Center 7: The intersection point of the optical axis 5 and the optical sensor unit 12 is referred to as the image center 7. This image center 7 may coincide with the geometric center of the optical sensor unit 12, but it does not necessarily have to coincide with it.

[0012] Definition of a camera image: The camera image from a camera is specifically constructed as a matrix, and this matrix has pixels at its matrix points, such as 8-bit grayscale points or color points.

[0013] Definition of code array: The code sequence can be imaged by a camera, and based on the camera image, it is possible to measure, for example, the absolute position of a machine or machine part with one to six degrees of freedom. In particular, the code sequence is configured to be detected by non-contact reading from a camera, and non-contact detection by the camera makes it possible to determine the actual position in multiple dimensions. Preferably, a two-dimensional code sequence can be used in production equipment and / or inspection equipment where the positioning of workpieces and / or inspection means and / or working means must be performed.

[0014] Definitions of basic symbols: A two-dimensional code array includes a plurality of basic symbols, which are arranged on a surface, and these basic symbols on the surface form a two-dimensional periodic grid, i.e., a dot grid. The basic symbols are preferably geometric figures such as circles, squares, triangles, or lines. Particularly preferably, the basic symbols are configured as circles. Basic symbols are also referred to as dots. The basic symbols preferably represent digits in a number system. In particular, a two-dimensional code array includes at least two different basic symbols. In a possible embodiment of the present invention, the two-dimensional periodic grid includes empty grid points not occupied by basic symbols in the grid space and / or grid space as parcel symbols.

[0015] The basic symbols are arranged on a surface, which may be curved or not. For example, the surface may be the floor of production equipment and / or inspection equipment. The basic symbols are arranged in a two-dimensional grid within the surface, preferably with the area centroid of the basic symbols forming a grid point within the point grid.

[0016] Definition of a point grid in a code array: Grid points are also referred to as grid space and / or lattice space below. In particular, a two-dimensional periodic grid of points forms a two-dimensional lattice.

[0017] Definition of a parcel: The surface is preferably divided into a plurality of parcels of the same kind arranged regularly, and these parcels have a basic shape, for example, a square, rectangle, triangle, or hexagon. A parcel of the same kind is understood to be a parcel of particularly similar size and / or similar shape. Preferably, each parcel contains n basic symbols. A parcel contains an integer number of basic symbols, and especially an even number of basic symbols. Preferably, a parcel contains more than 10 basic symbols, especially more than 20 basic symbols, and especially more than 40 basic symbols. Furthermore, the number of basic symbols in a single parcel is preferably less than 100. The parcels formed by the basic symbols may be optically displayed within the code array, for example by a border, or they may not be optically displayed and simply form a conceptual and / or logical unit.

[0018] Definition of parcel area: A parcel has at least one first parcel region and a second parcel region. The X parcel region includes the first parcel region, and the Y parcel region includes the second parcel region. In particular, each parcel region occupies one connected surface within the parcel or occupies multiple distributed, unconnected subsurfaces.

[0019] Each parcel region contains multiple basic symbols. In particular, the X parcel region and the Y parcel region contain the same number of basic symbols. Specifically, at least two parcel regions are arranged within the parcel such that they have p rotational symmetries with respect to the center of the parcel as a point of rotation, where p rotational symmetries are, for example, 2, 3, or 4 rotational symmetries.

[0020] Definition of Parcel Symbols: Each parcel has at least one parcel symbol, which represents a fixed reference point within each parcel. The parcel symbols enable reading and / or decoding of the basic symbols in a defined order. The parcel symbols are regularly and / or periodically arranged, in particular, within a two-dimensional periodic grid of the basic symbols. The parcel symbols can be located at or adjacent to grid points. The parcel symbols are each located in the same position within the parcel, for example, at the center of the parcel. The parcel symbols are graphic elements distinct from the basic symbols, such as triangles, hexagons, or lines. Alternatively and / or supplementally, the parcel symbols are represented by omitting one or more basic symbols within the parcel. In one possible embodiment, the parcel symbols form points of symmetry with p rotational symmetry of the parcel. In particular, the reading direction and / or decoding order, i.e., the order in which the basic symbols within the X parcel region and / or Y parcel region must be read, is defined. Preferably, the reading direction and / or decoding order corresponds to the setting of which basic symbols should be read and / or decoded in sequence.

[0021] Definition of coding: Within the X-parcel region, the X-coordinate values ​​are encoded by the basic symbols, and within the Y-parcel region, the Y-coordinate values ​​are encoded by the basic symbols. In particular, the X and Y coordinate values ​​are the coordinates of the basic symbols within the parcel, and the coordinates are shown in the Cartesian coordinate system of the surface stretched by the basic symbols. Alternatively and / or supplementarily, the X and Y coordinate values ​​can also indicate the position within the surface as coordinates in other coordinate systems, such as oblique coordinates, cylindrical coordinates, or spherical coordinates.

[0022] Definition of a preferred embodiment of a point grid: In a particularly preferred embodiment of the present invention, the two-dimensional periodic grid is a rectangular grid, and the parcels are also rectangular. In particular, the rectangular grid and rectangular parcels are square grids and / or square parcels. Preferably, the distances between basic symbols are of equal magnitude along the length axis and width axis of the rectangular grid. For example, in the case of a square grid with square parcels, the number of basic symbols in the X and Y directions of the planar two-dimensional periodic grid is the same. In particular, the X parcel region and the Y parcel region each consist of two spatially separated rectangular sub-faces within the parcel, and the sub-faces have longitudinal extensions. The longitudinal extension of the sub-faces of the X parcel region is preferably perpendicular to the longitudinal extension of the sub-faces of the Y parcel region. Preferably, the surface occupied by the X parcel region can be transitioned to the surface occupied by the Y parcel region by a 90° rotation.

[0023] In a particularly preferred embodiment of the present invention, the periodic grid has a grid length and a grid width. The grid length extends in the X direction of the Cartesian coordinate system, and the grid width extends in the Y direction. The coordinate system thus formed associates each point on the coding surface with a uniquely identifiable position vector (X,Y).

[0024] g consecutive basic symbols form in particular one overall sequence. The overall sequence is entered in the X parcel area, in particular in a plurality of adjacent parcels in the X direction. The entry of the basic symbols is carried out, in particular, within each respective parcel, preferably in ascending order in the X direction of the parcels, in a predefined reading order and / or decoding order within the X parcel area, according to the order of those basic symbols in the overall sequence. The number g is a number large enough to completely fill the X parcel area of all adjacent parcels in the X direction. In particular, g is greater than 50, in particular greater than 1000, in particular greater than one million. The content of the X parcel area of adjacent parcels in the Y direction is the same. It is particularly preferred that one segment of t consecutive basic symbols forms one partial sequence in the overall sequence. In particular, each segment of t consecutive basic symbols of the overall sequence forms one partial sequence, and the consecutive basic symbols are consecutive in decoding order and / or reading order. In particular, the overall sequence is configured such that each partial sequence of t consecutive basic symbols is included only once in the overall sequence when read in the forward direction, and each partial sequence read in the backward direction is not included in the overall sequence when read in the forward direction. Preferably, the partial sequence includes at least 5 consecutive basic symbols, in particular at least 20 consecutive basic symbols, in particular at least 30 basic symbols. Furthermore, the partial sequence preferably includes less than 50 basic symbols, in particular less than 30 basic symbols. In particular, t < g holds.

[0025] In a particularly preferred embodiment of the present invention, the basic symbol is configured to encode digits in base b. Base b is preferably the base of a positional numeral system. Preferably, base b = 2, i.e., the base of the binary system, where the binary digits include 0 and 1. Furthermore, base b = 10 can be used to form the base of a decimal system, where the digits include 0, 1, 2, 3, 4, 5, 6, 7, 8 and 9. Alternatively, base b = 16, i.e., the base of a hexadecimal system, where the digits include 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E and F. In particular, base b is arbitrarily selectable, and the digits of the base include the elements 0, 1, ..., b-1. Alternatively and / or supplementarily, base b is the base of an additive number system, such as Roman numerals.

[0026] In a particularly preferred embodiment of the present invention, the base b=2 is selected. In particular, the selected base b=2 forms a binary system. The binary system and / or the number system using base b=2 includes two digits, in particular the digit 0 and the digit 1. Preferably, two different basic symbols encode the digits 0 and 1 of the binary system. The two different basic symbols are formed by two circles, namely, a first circle with radius R1 and a second circle with radius R2. In particular, radius R1 is selected to be smaller than radius R2. In particular, the ratio of radius R2 to radius R1 is greater than the square root of 2. For example, radius R2 is selected to be smaller than half the grid dimension of the code grid, so that two adjacent circles in a periodic grid do not touch R2. This embodiment is based on the consideration that, on the one hand, particularly high information density is achieved, and on the other hand, reliable readability is achieved using standard image processing methods such as segmentation with bloba analysis.

[0027] In possible embodiments of the present invention, the whole sequence is configured such that the sum of the digits of one subsequence is less than half the maximum possible sum of the digits of one subsequence. In particular, the sum of the digits of each arbitrary subsequence taken from the whole sequence is less than half the maximum possible sum of the digits of one subsequence. For example, the sum of the digits of each subsequence of length t for a number system with base b is less than q = t·(b-1) / 2.

[0028] In a particularly preferred embodiment of the present invention, the two-dimensional code sequence includes an inverted overall sequence. The inverted overall sequence preferably represents an inversion of the overall sequence. Preferably, the inverted overall sequence is entered into the Y-parcel regions of multiple adjacent parcels in the Y-direction. The entry of basic symbols is performed within each parcel, preferably in ascending order in the Y-direction of the parcel, depending on the order of those basic symbols in the inverted overall sequence, in a predetermined reading and / or decoding order within the Y-parcel region. The number g is large enough to completely fill the Y-parcel regions of all adjacent parcels in the Y-direction. In particular, the contents of the Y-parcel regions of adjacent parcels in the X-direction are identical.

[0029] In possible embodiments of the present invention, the base symbol at the k-th position of the inverted global sequence is an encoded digit m(k), and this digit m(k), together with the digit n(k) encoded by the base symbol at the k-th position of the global sequence, satisfies the relation: m(k) = bn(k) - 1. For example, the inverted global sequence and the global sequence when binary with base b = 2 is chosen form the relation: m(k) = 1 - n(k).

[0030] Definition of reference point: Particularly preferably, the section includes a reference point. The reference point is, for example, the intersection of the optical axis of the camera and the plane on which the code array is located. Preferably, the method is configured to determine the position of the camera relative to the work area, based on an image captured by the sensor unit, such as the X coordinate (rx) and Y coordinate (ry) of the reference point in a coordinate system stretched by the work area and / or basic symbols and / or code array.

[0031] For the time being, within the framework of this method, only the coarse position of the encoded X and Y coordinates is identified as two degrees of freedom. Even with just the coarse position, a reliable and usable position for determining the camera's coarse position can be obtained.

[0032] By identifying the X and Y coordinates within the code sequence as coarse positions, the relative position of the camera relative to the code sequence can be estimated. For example, if the coarse position is identified in the immediate vicinity of a reference point, the camera's position can be estimated through knowledge of the camera's optical axis extension. The camera's near-precise position can be obtained if the reference point is equal to the encoded position and the angle of the optical axis relative to the code sequence is known. The simplest example is when the optical axis is oriented perpendicular to the code sequence.

[0033] Definition of degrees of freedom: In a particularly preferred embodiment of the present invention, the method is configured to specify the position and / or orientation of the work module relative to the work area with up to six degrees of freedom. These degrees of freedom are, in particular, as follows: rx: Location coordinates in the code array's coordinate system (as coarse and / or precise positions). ry: Location coordinates in the coordinate system of the code array (as coarse and / or precise positions). In particular, rx and ry relate to the reference point, especially the point where the optical axis penetrates through the code array. rz: The distance from the reference point along the optical axis to the camera. phiz: The Z-axis rotation angle of the camera around the optical axis. phix: The first pitch angle of the optical axis with respect to the code array, particularly with respect to the first principal direction. phiy: The second pitch angle of the optical axis with respect to the code array, particularly with respect to the second principal direction.

[0034] Definitions of X and Y coordinates (rx, ry): For example, the six degrees of freedom of the camera's position and / or orientation relative to the work area are particularly related to the coordinate system (X) of the code array within the plane of the code array. C ,Y C ) includes coordinates X and Y with respect to the Cartesian coordinate system established by the basic symbols.

[0035] Definition of the distance (rz) between the reference point and the optical center of the camera: Furthermore, the six degrees of freedom are determined by the distance Z or r from the camera to the reference point along the optical axis. z This includes the distance between, for example, a reference point and, for example, the optical center of a camera's objective lens.

[0036] Definition of rotation angle: Furthermore, three independent rotation angles of the camera in the camera coordinate system are identified. Two of these rotation angles are defined as intermediate angles between the optical axis and the code array (phix, phiy). The third rotation angle (phiz) describes the rotation of the camera about the optical axis.

[0037] Definition of the coordinate system for code arrays: Coordinate system of code array (X C ,Y C ) is located within the plane of the code array.

[0038] Definition of the coordinate system of the camera's imaging device and / or the coordinate system within the camera image: Coordinate system of the imaging device (X I ,Y I ) is located within the plane of the imaging device or within the camera image.

[0039] Definition of the camera coordinate system: The camera's coordinate system (X,Y,Z) has its origin at the optical center of the camera's objective lens, and the negative Z direction coincides with the camera's optical axis.

[0040] Generalization: In this specification, coordinate systems are exemplified with respect to their origin and orientation, although equivalent coordinate systems can also be used.

[0041] Definition of the method: In several modifications, this method is characterized in that the X and Y coordinate values ​​are decoded and / or identified based on a camera image captured by the camera. In one possible evolution of this method, the orientation of a segment with respect to three coordinate axes is identified based on the segment captured in the image within the code sequence. In particular, the orientation of the segment captured in the image is identified with respect to the X and Y axes of the Cartesian coordinate system stretched by the basic symbols. Alternatively and / or supplementarily, the coordinates of the position of a reference point within the code sequence are identified by decoding the segment contained in the image within the code sequence. In a particularly preferred embodiment of the present invention, the orientation of the work module with respect to the work area is identified in up to six degrees of freedom. In particular, in this case, the orientation of the camera within the work area is identified in three Cartesian coordinates X, Y, and Z, where X and Y are coordinates of the coordinate system stretched by the basic symbols, and Z is the distance from the work module to the work area. Furthermore, in this method, for example, three Euler angles of the camera are identified.

[0042] Alternatively and / or supplementally, two-dimensional code sequences are used in fields other than automation technology, for example, in nature and the environment, biology and medicine, architecture, entertainment appliances, and / or sensors, to monitor motion flow.

[0043] The general advantages of this invention depend on modifications: The central objective of this invention is to provide a high-performance, highly reliable, and inexpensive method for locating and identifying one or more objects in a workspace with up to six degrees of freedom (6D positioning = simultaneous position detection in six degrees of motion).

[0044] This method processes image data provided by an imaging measurement system, such as a camera. A smartphone, for example, can be used to capture images and to determine the location using the method according to the present invention, and this method is implemented by the smartphone's embedded computer or as an algorithm within an application running in the cloud.

[0045] To determine the object's position with up to six degrees of freedom, the camera detects a two-dimensional code array associated with the object to be located.

[0046] The method according to the present invention provides the following characteristics depending on the modification: • Up to complete 6D position information can be obtained: translation (X, Y, Z) and rotation (φ_X, φ_Y, φ_Z). • Absolute position information is obtained (not incremental), meaning that the reference drive required in the case of incremental sensors is omitted. • The measurement range is very wide: • In X,Y: It is "almost infinite" and limited only by the code sequence. For example, the area that can be uniquely encoded using the code and 2.5 mm dot grid described in [B] is as follows: For a parcel size of 5x5 dots: 2m x 2m For a parcel size of 7x7 dots: 8km x 8km For a parcel size of 9x9 dots: 550,000 km x 550,000 km At z: It is a single dimension with limitations. It can be adapted to the application via camera resolution, objective lens, and the grid dimensions of the code array. In φ_X and φ_Y, the actual values ​​are + / -60°. (As shown in Figures 5 and 6) If an object has point codes on all sides, it is infinite. In φ_Z: Infinite (0° to 360°) • The measurement rate is high, for example, 100Hz to 10000Hz. • The delay time (the time from when an image is captured until the 6D position measurement is output) is short, for example, 10ms to 100μs. • High measurement accuracy in all six degrees of freedom: If the code grid dimension is 2.5 mm, the actual repeatability is 1 μm and 0.01° (in the case of 3 sigma). • Maximum readability is achieved even when localized noise occurs in the image or when image brightness fluctuates. • Both the sensor system and the code arrangement can be made inexpensive. • It is possible to read additional data simultaneously with location determination. For example, an ID code embedded in a code array can be read to identify an object. By changing the objective lens, image sensor resolution, code array grid dimensions, reading field size, computational performance, etc., accuracy, measurement range in Z, redundancy, and measurement rate can be scaled. • For example, as a smartphone application, it is easy to integrate into existing systems.

[0047] This makes it possible to easily implement numerous automation technologies that were previously impossible or economically unfeasible, such as the following: • To achieve real-time attitude control in up to six dimensions, precisely position the axes or rotors in the axis system or robot system. • To build an absolute positioning system and robot that eliminates the need for reference driving. • Simplify the positioning system because multiple sensors that measure in one dimension can be replaced by a single sensor that measures in multiple dimensions. For example, visual servo processing to make a robot's gripper follow a moving target. • Monitoring and tracking of multiple objects in space through periodic position measurements. • Capture and analysis of the operation flow • Vibration analysis of machines in 6 dimensions. • Identifying the relative orientation of two objects. For example, a camera in a handheld device detects two objects, each assigned a point code, within a single image. The proposed method allows for the positioning of the two objects relative to a common camera coordinate system. By forming a vector difference, the relative orientation of the two objects can be determined. The camera position is computationally removed in this case; that is, the method is sufficiently independent of the camera position. Identify the relative orientation of three or more objects, each equipped with a camera and / or code array. For example, within a manufacturing plant, a location network is constructed to determine the relative positions of multiple stationary or moving objects relative to each other, or relative to the factory coordinate system. For instance, multiple autonomous vehicles, each equipped with a camera, can be located by detecting a sequence of codes mounted on the ceiling of the factory.

[0048] The code array has a planar point grid, in particular, comprising multiple basic symbols, preferably with the center of each basic symbol located at a grid point, and the coarse position of each basic symbol is encoded in the code array.

[0049] Within the framework of this method, a starting field is identified. The starting field preferably has at least three adjacent basic symbols, which are arranged at predetermined angles to each other. Connecting lines between the basic symbols of the starting field define two independent principal directions along the point grid in the camera image. If the starting field has exactly three basic symbols, these basic symbols are arranged, for example, at predetermined angles to each other. These basic symbols define a coordinate system for a planar point grid, with at least one of the three basic symbols forming the origin of the coordinate system. In a preferred evolution of the present invention, the starting field has nine basic symbols arranged in a square and / or rectangle. In particular, the starting field has an edge length comprising three basic symbols. The basic symbols of the starting field are valid basic symbols. A valid basic symbol, particularly within the framework of this method, is understood to be a basic symbol selectively defined as a valid basic symbol, or a basic symbol classified as a valid basic symbol based on image features, and / or a basic symbol that is set as a valid basic symbol by the available processing results after discovering and successfully processing the basic symbol. Image structures that are not classified as valid basic symbols are interpreted as image noise and excluded from subsequent processing.

[0050] In this method, during the search step, starting from at least one valid basic symbol, further basic symbols, especially valid basic symbols, are searched along the principal direction.

[0051] If the search is successful, and especially if additional basic symbols are discovered, these additional basic symbols are characterized as valid basic symbols.

[0052] The search step is expected to be performed multiple times. If new and further basic symbols are found as valid basic symbols during the search step, the search step is selectively performed starting from the basic symbols of the starting field, or from the newly discovered valid basic symbols. In this way, the point grid is progressively filled in, starting from the starting field with valid basic symbols, for each additional valid basic symbol. The search step is repeated until a sufficient number of valid basic symbols are found, in particular so that decoding can be performed.

[0053] Based on a valid base symbol or a selected subset thereof, the coarse position of at least one base symbol in the code sequence is identified as two degrees of freedom of a camera relative to the code sequence by decoding a particularly valid base symbol.

[0054] In particular, a valid basic symbol or a subset thereof is decoded, and from the decoded coarse position and, optionally supplementarily, the coarse orientation of the coordinate system of the point grid in the camera image, at least one degree of freedom of the camera relative to the code sequence is identified.

[0055] In this case, the idea of ​​the present invention is that basic symbols within a code array and / or within a camera image can be searched and discovered, for example, using a planar function of image processing. However, discovering basic symbols using, for example, bloba analysis or pattern-based search (pattern matching) requires frequent access to pixels in the camera image, which results in cumbersome image processing.

[0056] By starting from a search field that defines the two principal directions of the point grid, and consequently the coordinate system of the point grid in the camera image, and performing the search exclusively in the principal directions, prior knowledge of the structure of the point grid can be utilized, and therefore the search is performed on lines along the principal directions, rather than on surfaces. From a practical standpoint, it is sufficient to search for further fundamental symbols along lines in the principal directions, starting from a valid fundamental symbol. It is also obvious that reducing from surface search to line search significantly reduces the number of pixel accesses and / or the cost of image processing. Therefore, the method according to the present invention enables a highly efficient implementation of the method.

[0057] In a preferred evolution of the present invention, the center position, particularly the centroid position, of each basic symbol in the camera image is identified. This allows for the identification of multiple intersections within a point grid based on the valid basic symbols.

[0058] The search step is performed starting from the center position of each valid basic symbol, and is intended to be a line-targeted search, particularly along one of the main directions in a predetermined search direction. By discovering the basic symbols, it is initially possible to roughly determine the position of the basic symbols within the point grid and / or camera image, while detecting the center position allows for the precise determination of the position within the point grid and / or camera image, thereby concretizing the point grid. It is particularly advantageous to perform the line-targeted search step starting from the center position of the basic symbols, because this prevents adjacent basic symbols from being mistakenly overlooked.

[0059] In an advantageous development of the present invention, two connection vectors are determined in two independent principal directions along a point grid in the camera image based on a starting field and / or further valid basic symbols. By determining the connection vectors, the positions of the nearest and / or adjacent basic symbols to be searched can be estimated by extrapolation within the framework of a linear combination of the connection vectors, starting from known valid basic symbols.

[0060] Particularly preferably, the center position of the basic symbol is determined by shifting the starting point within the basic symbol toward the center position along the principal direction over several intermediate steps until the center position is found at a location where the center position is located in the center of the basic symbol in the principal direction. By shifting the starting position toward the center position so that the starting position is always located in the center in the principal direction, it is only necessary to search for the boundaries of the basic symbol in the principal direction for each shift step, and then subsequently shift the starting position toward the center between the boundaries.

[0061] In the example of a circular base symbol, the boundaries of the base symbol are explored by a linear search in the first principal direction of the camera image, starting from the initial position inside the base symbol, and then the boundaries of the base symbol are explored in the negative first principal direction. Averaging the boundaries results in a more precise position estimation. This step is repeated correspondingly for the second principal direction, and then again for the first principal direction.

[0062] In this way, the shift of the starting position to the center position is computationally efficient through the flow of line evaluation along the principal direction.

[0063] In subsequent steps, the area of ​​each basic symbol in the camera image is identified, and this area forms the encoded data for the code sequence. In particular, the area is used to classify the basic symbols.

[0064] The data may be, for example, 0 or 1 in a binary system, and the basic symbols may be, for example, circular surfaces of different sizes. If the center position of each is known, and the extent of the basic symbol in the principal direction is known by the shift step, the area of ​​the basic symbol can be easily estimated using these values. This allows the center position within the point grid to be determined first, and then the area of ​​the basic symbol, very computationally efficient.

[0065] The area of ​​basic symbols in a camera image is strongly affected by perspective distortion. For example, the area decreases as the distance from the camera to the basic symbol increases, and as the camera's tilt angle increases (elliptic distortion). These effects can lead to misclassification, for example, when identical basic symbols occur in the camera image at very different distances from the camera. Therefore, to classify basic symbols, it is preferable to use the normalized area of ​​the basic symbols.

[0066] To calculate the normalized area, a reference area exposed to approximately the same perspective distortion as the base symbol is identified. The parallelogram stretched by two vectors from the grid position of the base symbol to adjacent grid positions along the principal axis is used as the reference area. The reference area is obtained as the absolute value of the cross product of the two vectors.

[0067] The normalized area of ​​a basic symbol is calculated specifically as the quotient of the area of ​​the basic symbol and the reference area.

[0068] In a preferred embodiment of the present invention, a reading field is defined in a camera image and / or a planar point grid, a two-dimensional basic symbol matrix, particularly a dot matrix, is formed corresponding to the reading field, one entry in the basic symbol matrix is ​​associated with each point in the reading field, and the position, area and / or data of a basic symbol are entered in the basic symbol matrix. From the data in the basic symbol matrix, the coarse position in the coordinate system of the point grid and, optionally, the coarse orientation can be decoded, and / or the coarse position of at least one basic symbol in the code array can be identified as two degrees of freedom.

[0069] In a preferred evolution of the present invention, at least one starting position is defined in a preliminary step for creating a starting field. The starting position is arbitrarily defined. Preferably, the starting position is located near or adjacent to a reference point. If the starting position is randomly located within a basic symbol, it is treated as the starting position for the starting basic symbol. If the starting position is located outside a basic symbol, adjacent basic symbols along the search ray starting from the starting position are searched as the starting basic symbol. Therefore, even in this preliminary step, computationally intensive image processing on a surface is not performed; rather, basic symbols are searched only along the search ray starting from the starting position. For example, the boundaries of basic symbols can be detected by contrast changes along the search direction in a camera image. In this case, the method can use at least or exactly 8 or 16 search rays that start at the starting position and are distributed in regular angular increments over 360°. A minimum radius can be defined for the search rays, and further basic symbols are searched from this minimum radius. The maximum search radius can be optionally defined. If the maximum search radius is reached without finding adjacent basic symbols, the starting position is discarded, and a replacement starting position is selected.

[0070] In subsequent steps, adjacent basic symbols are searched for as first auxiliary starting basic symbols along further search rays, starting from the position of the initial basic symbol. The search for the first auxiliary starting basic symbols may be carried out using a distribution of search rays similar to the distribution described above.

[0071] In subsequent steps, the second auxiliary starting symbol is searched for in a direction angled with respect to the connection between the starting symbol and the first auxiliary starting symbol (and / or along the first principal direction), starting from the starting symbol and the first auxiliary starting symbol. However, in this case, fewer search rays than those described above can be used, particularly distributed over a smaller angular range, thereby accelerating the search. In this case, the fact that the first three symbols should be positioned at angles to each other is fully utilized, and therefore, there is prior knowledge about the direction in which the second auxiliary starting symbol will be searched. For example, only one, two, or three to six search rays distributed around the principal search direction perpendicular to the connection between the starting symbol and the first auxiliary starting symbol may be used.

[0072] To ensure completeness, starting from the initial basic symbol, all basic symbols that are directly adjacent to the initial basic symbol in the principal axis direction or diagonal direction of the point grid are searched for by a linear combination of connections, based on the connection between the initial basic symbol and the first auxiliary initial basic symbol, and based on the connection between the initial basic symbol and the second auxiliary initial basic symbol. In particular, connection vectors and / or principal directions can be derived through the first three basic symbols found. These connection vectors and / or principal directions establish the two-dimensional coordinate system of the point grid. For the purpose of improving this method, a center position can be identified to define the connection vectors as precisely as possible. In that case, a starting field with 3 × 3 valid basic symbols is provided as a result.

[0073] In the search step, adjacent basic symbols are searched starting from the starting field and / or from any further valid basic symbols discovered in previous search steps. This method preferably searches for basic symbols at grid points that have at least two valid adjacents in the principal or diagonal direction. All adjacents of a grid point provide independent position estimates by extrapolation, which, when averaged, result in a more precise position estimate of the grid point.

[0074] The condition that only grid points with at least two valid basic symbols are searched as adjacent further promotes planar spreading of the detected region and avoids linear spreading in the form of spears, dendrites, or tips. Planar spreading is more robust and tolerant of noise in the image than linear spreading.

[0075] The basic symbol defines a first principal direction along a point grid and a second principal direction independent of the first principal direction. Depending on the observation angle, the two independent principal directions are imaged perpendicularly or obliquely to each other in the camera image.

[0076] From the camera image, a two-dimensional basic symbol matrix is ​​identified, and one entry in the basic symbol matrix is ​​associated with each grid point in the code array, with the central position of the basic symbol in the camera image being entered into the basic symbol matrix. Therefore, the entries in the basic symbol matrix indicate the position of the basic symbol at the associated grid point, more specifically, in the coordinate system of the imaging device and / or the coordinate system of the camera image.

[0077] The point grid in the camera image is particularly subject to perspective distortion; therefore, although the rows and columns of the point grid are each arranged in a straight line, they are generally positioned at an oblique angle to each other in terms of perspective distortion.

[0078] In addition, optical distortion may occur, and this optical distortion can be optionally compensated for by parallelization processing.

[0079] Preferably, a first linear function is determined from the data of the basic symbol matrix using a first function argument with respect to the first line in the coordinate system of the camera image. When the first line is conceptually transcribed to the coordinate system of the code array, the first line is always parallel to the first principal direction of the point grid, independently of the first function argument. By changing the first function argument, the first line is shifted parallel to the second principal direction in the coordinate system of the code array. Thus, by changing the first function argument, the line can be positioned, for example, on a row of the point grid, and more specifically, in both the coordinate system of the code array and the coordinate system of the camera image.

[0080] In particular, this can be done as follows:

[0081] Starting point status: a) The central point of the basic symbol forms a two-dimensional grid of points in the code plane, consisting of regularly arranged linear rows and columns. b) The camera images the code plane onto the image sensor with perspective distortion. In the camera image, rows generally appear as fan-shaped lines with a single common vanishing point. The same applies to columns. c) Due to distortion of the objective lens, rows and columns appear as curved lines in the camera image.

[0082] Method steps in the camera coordinate system: a) Through parallelization processing, the distortion of the objective lens is mathematically eliminated, and curved lines are transformed into straight lines. b) Linear interpolation is used to fit the lines to the rows, and these lines are numbered using integer indices as function arguments, following their progression. These lines form the first line bundle. The same procedure is followed for the columns, and these lines form the second line bundle. c) Each line is described by a line angle and an intercept value. The line angles of one bundle form a sequence approximated by quadratic interpolation. Integer line indices are used as function arguments. The polynomial has three interpolation parameters. The same procedure is followed for the intercept values, from which three further interpolation parameters are obtained. That is, each bundle of lines is described completely and compactly by six interpolation parameters. d) By substituting reasonable function values ​​into the interpolation function, it is also possible to calculate a straight line located between two rows or two columns.

[0083] Alternatively or supplementally, based on the basic symbol matrix, a second linear function is determined with respect to the second line using the second function argument in the coordinate system of the camera image. When the second line is transcribed to the coordinate system of the code sequence, the second line is always parallel to the second principal direction of the point grid, independently of the second function argument. By changing the second function argument, the second line is shifted parallel to the first principal direction in the coordinate system of the code sequence. Therefore, by changing the second function argument, the line can be positioned, for example, on a row of the point grid, and more specifically, in both the coordinate system of the code sequence and the coordinate system of the camera image.

[0084] It should be noted that the terms row and column are used solely for nominal purposes and do not imply any particular orientation of the point grid.

[0085] Based on at least one of the linear functions, at least one additional degree of freedom of the camera can be identified. Optionally, based on two linear functions, at least one additional degree of freedom of the camera can be identified.

[0086] In this case, there is a further consideration that the amount of data should be reduced along the way from the camera image to at least one degree of freedom in order to enable faster and / or more efficient computation of at least one degree of freedom. For example, a camera image with a size of 200 x 200 pixels still has 40,000 values, while the basic symbol matrix has already been reduced to position, and accordingly, for example, in the case of a section of the camera image with side lengths of a point grid of 15 basic symbols, there are only 225 values ​​related to position. By deriving at least one linear function, the amount of data is reduced to the parameters of the linear function.

[0087] In this case, it has been found that, in addition to the six parameters, one data point per linear function is sufficient as an optional choice to ensure that the substantial information content of the basic symbol matrix or camera image is sufficient on the way to at least one degree of freedom. This reduces the amount of data in the example with 225 entries to 12 or 14 entries. This significantly reduces the computation of at least one degree of freedom from a cost perspective.

[0088] A further advantage of this embodiment is that the rows and columns of the basic symbol matrix in the code array are arranged parallel and regularly spaced apart from each other, thereby performing a kind of averaging across the basic symbol matrix by deriving a linear function in the camera image, in which case the linear function represents the averaged information of the basic symbol matrix. The linear function performs both information compression and information content improvement.

[0089] Therefore, the method according to the present invention makes it possible to computationally perform a method for identifying at least one degree of freedom of a camera relative to a code array from a camera image of a camera. From an applied technical standpoint, for example, this identification can be performed on a microcontroller that can identify at least one degree of freedom at least 100 times per second. In this way, the method can be used for real-time applications, for example, in manufacturing.

[0090] In a preferred evolution, the first linear function is identified based on at least two rows of the basic symbol matrix, preferably three or more rows, and in particular all rows. Alternatively or supplementarily, the second linear function is identified based on at least two columns of the basic symbol matrix, preferably three or more columns, and in particular all columns. This evolution emphasizes that the linear function carries averaged and / or compressed information across multiple rows or columns.

[0091] In a preferred embodiment of the present invention, one first compensation line is formed for each row along a first principal direction. The first linear function is formed based on a plurality of first compensation lines. By forming one first compensation line for each row, this first compensation line can be adapted to the extension of the row, thereby the first compensation line already forms the averaged and / or compressed information of the underlying row. The first linear function is formed based on a plurality of first compensation lines, thereby performing a second averaging or compression, and thus the first linear function is formed by the double average of the original information.

[0092] Alternatively or supplementally, for each column along the second principal direction, one second compensation line is formed. The second linear function is formed based on multiple second compensation lines. By forming one second compensation line for each column, this second compensation line can be fitted to the extension of that column, thereby the second compensation line already forms the averaged and / or compressed information of the underlying column. The second linear function is formed based on multiple second compensation lines, thereby performing a second averaging or compression, and therefore the second linear function is formed by the double average of the original information.

[0093] In a preferred implementation, the first function argument is constructed as the first integer count in the row, and / or the second function argument is constructed as the second integer count in the column. Thus, with respect to the first integer count in the first linear function, the first line corresponds to the first compensation line. Similarly, with respect to the second integer count as the second function argument in the second linear function, the second line corresponds to one of the columns of the basic symbol matrix. However, the first line and / or the second line are not exactly the first or second compensation line, but rather the corrected first or second compensation line, since the linear function has undergone a second averaging / compression.

[0094] In a preferred embodiment, the compensation line is described by a linear angle as an intersection angle, and by at least one axis intersection with the coordinate system of the imaging device and / or camera image. With respect to the compensation line, it is sufficient to have at least one or exactly one coordinate axis angle and axis intersection of the coordinate system to uniquely determine the compensation line in the coordinate system.

[0095] In this embodiment, the position of the compensation line within the point grid of the row or column to which it belongs is reduced to two values.

[0096] In a preferred development, the linear function is formed by a combination of a linear angle function, which depends on the function argument of the linear function, and an axis intersection function, which depends on the function argument of the linear function. Thus, the linear function is also determined by the linear angle and at least one axis intersection.

[0097] It is preferable that the linear angle function is constructed as a quadratic polynomial, and / or the axis intersection function is constructed as a quadratic polynomial, and that these polynomials have the function arguments of the respective linear functions as function arguments. This makes it possible to determine the linear angle and / or axis intersections depending on the function arguments. By choosing quadratic polynomials, the approximation can be performed particularly easily, and therefore the computational efficiency is further improved.

[0098] In a preferred embodiment, the axis intersections of the coordinate system's axes are selected such that, depending on the linear angle of the linear function along with the coordinate system, they yield smaller intermediate angles with respect to the perpendiculars to each coordinate axis. Furthermore, the linear function is associated with data encoding the selected coordinate axes. In this case, it is considered that only the axis intersection of a single coordinate axis is needed to describe the line, and the axis intersections with both coordinate axes are unnecessary. To achieve the greatest possible persuasiveness, axis intersections are selected such that the associated intersection angle is more perpendicular to the intersecting coordinate axes.

[0099] Preferably, a reference point is located within the camera image. The reference point may be arbitrarily positioned. In a preferred embodiment described later, the reference point is configured as the intersection of the optical axis and the imaging device and / or camera image. Therefore, the reference point is provided by the camera in the camera's coordinate system and / or within the camera image.

[0100] Preferably, a first axis intersection function in the coordinate system of the camera image is formed by a first line. The first line is oriented parallel to the first principal direction of the point grid in the coordinate system of the code array. The axis intersection function has a first function argument, and by changing the first function argument, the first line is shifted parallel to the second principal direction in the coordinate system of the code array. The first axis intersection function, depending on the first function argument, defines a first axis intersection along a first axis in the coordinate system of the camera image, the first axis extending through a reference point. Therefore, by shifting the first line by changing the first function argument in the coordinate system of the code array, the first line in the coordinate system of the camera and / or camera image is shifted (with perspective distortion) so that the axis intersection moves along the first axis.

[0101] More preferably, a second axis intersection function in the coordinate system of the camera image is formed by a second line. The second line is oriented parallel to the second principal direction of the point grid in the coordinate system of the code array. The axis intersection function has a second function argument, and by changing the second function argument, the second line is shifted parallel to the second principal direction in the coordinate system of the code array. The second axis intersection function, depending on the second function argument, defines a second axis intersection along the second axis in the coordinate system of the camera image, the second axis extending through a reference point. Thus, by shifting the second line by changing the second function argument in the coordinate system of the code array, the second line in the coordinate system of the camera and / or camera image is shifted (with perspective distortion) so that the axis intersection moves along the second axis.

[0102] The first and second function arguments are determined based on the axis intersection function such that the reference point forms the first and second axis intersections. Figuratively speaking, the first function argument is modified until the first line extends through the reference point in the camera and / or camera image coordinate system, and / or until the first axis intersection is located at the reference point. Similarly, the second function argument is modified until the second line extends through the reference point in the camera and / or camera image coordinate system, and / or until the second axis intersection is located at the reference point.

[0103] Subsequently, based on the first and second function arguments and the coarse position, the precise position of the reference point in the coordinate system of the code sequence within the plane of the code sequence is identified as an additional degree of freedom. From a conceptual standpoint, first the coarse position of the basic symbol in the coordinate system of the code sequence is identified, and subsequently, the shift of the basic symbol to the reference point is determined based on the function arguments.

[0104] This method has the advantage of being able to identify the coarse position of basic symbols by decoding the code sequence. Subsequently, the shift to the reference point is calculated based on the axis intersection function, and by utilizing the axis intersection function, calculations can be performed using a small number of calculated values, and thus computationally efficient. This has the advantage that this method can be implemented in real-time applications in digital data processing devices, especially microcontrollers, with minimal computing power.

[0105] In a preferred embodiment of the present invention, the first function argument is configured as the count value of a row in the point grid, and / or the second function argument is configured as the count value of a column in the point grid. Figuratively speaking, the position of a basic symbol whose coarse position is known is shifted to a reference point by integer or fractional increments in the grid dimensions of the point grid.

[0106] As previously described, the reference point is preferably configured as the intersection of the camera's optical axis and the imaging device. Therefore, the reference point is defined as the structural position in the coordinate system of the camera and / or the camera image. However, it is not necessarily required that the reference point and / or intersection be located precisely in the center of the camera image and / or on the camera's imaging device. Rather, the position of the reference point can be defined through calibration.

[0107] In possible embodiments of the present invention, the axis intersection function is configured as a linear function for describing a straight line, particularly as described above. In this case, the axis intersection function includes a complete mathematical description of the straight line, depending on each function argument.

[0108] Alternatively, a linear function is used that is formed by a combination of a linear angle function (the angle of intersection between a line and one of the axes of the coordinate system of the camera image, depending particularly on the function arguments described above) and an axis intersection function. Thus, the line is completely described by the axis intersection and the linear angle. The advantage of this assignment is that only the axis intersection function needs to be determined and / or evaluated to pinpoint the precise location, and more detailed information about the linear angle is not required. This embodiment further improves the efficiency of the method.

[0109] Next, the precise position can be determined based on the function arguments and the known grid dimensions of the point grid. Figuratively speaking, the precise position is determined by starting from the position of a basic symbol whose coarse position is known, and then moving a portion of the grid dimensions in a first principal direction and a portion of the grid dimensions in a second principal direction, for example, to reach a reference point in the coordinate system of the code array. This representation is particularly computationally efficient.

[0110] Preferably, it is proposed that the first and / or second function arguments of the linear function be determined such that the first or second line intersects the reference point. For example, the first and second lines can be shifted by changing the function arguments so that they intersect the reference point.

[0111] The camera rotation angle Phiz around the optical axis is derived as the camera's degrees of freedom relative to the code array, based on a first and / or second line intersecting a reference point. Since the linear function is defined within the camera image, this derivation is possible. Therefore, the rotation angle Phiz can be identified as a linear angle in the camera and / or camera image coordinate system. The linear angle in the camera and / or camera image coordinate system corresponds to the camera rotation angle Phiz around the pivot axis, and more specifically, theoretical considerations have shown that there is no need to perform a transformation from the camera and / or camera image coordinate system to the code array coordinate system. Therefore, the rotation angle Phiz can be easily identified based on at least one linear function. Thus, this development presents a method for identifying the rotation angle Phiz very precisely and without significant computational cost. In this case, it should be emphasized that this identification is particularly easy to achieve by selecting the intersection of the camera's optical axes through the camera image or imaging device as the reference point. In this specific situation, the rotation angle Phiz can be derived particularly easily. This method makes it possible to easily and simultaneously determine the rotation angle Phiz from the camera image with high accuracy, and then provides this rotation angle Phiz as the camera's degrees of freedom relative to the code sequence.

[0112] In a preferred embodiment of the present invention, a basic symbol matrix is ​​identified from the camera image, and the positions of the basic symbols in the camera image are entered into the basic symbol matrix. Thus, the locations in the basic symbol matrix point to the positions of the basic symbols in a point grid, more specifically, in the coordinate system of the imaging device and / or the coordinate system of the camera image. A first linear function and / or a second linear function are preferably determined based on the basic symbol matrix.

[0113] Particularly preferred are two linear functions formed by combining a linear angle function, which is the angle of intersection between a line and one of the axes of the camera image coordinate system, and an axis intersection function, depending on the function argument. The axis intersection function, depending on the first function argument, defines an axis intersection along the axis of the camera image coordinate system, where the axis extends through a reference point. Thus, by changing the first function argument in the coordinate system of the code array, the line is shifted, causing the line in the camera and / or camera image coordinate system to shift (with perspective distortion) so that the axis intersection moves along the axis. One combination of linear function or linear angle function and axis intersection function is associated with a first principal direction, and the other combination of linear function or linear angle function is associated with a second principal direction. The advantage of this assignment is that the function argument is determined via the axis intersection function, and subsequently, the rotation angle Phiz can be read from the linear angle function based on this determined function argument.

[0114] In a preferred evolution of the present invention, a first value relating to the rotation angle Phiz is determined from a first linear function, a second value relating to the rotation angle Phiz is determined from a second linear function, and subsequently, the rotation angle Phiz is determined as the average of these two values. Since the rotation angle Phiz can be determined independently from the two linear functions, two independent values ​​are obtained, which can then be averaged to determine the rotation angle Phiz, thereby improving measurement accuracy. Alternatively, a validity control can be implemented to discard one of the two values ​​if it is not valid or valid.

[0115] One or more of the above-mentioned reference points are located within the camera image. The reference point is configured as the intersection of the optical axis and the imaging device and / or the camera image. Therefore, the reference point is provided by the camera in the camera's coordinate system and / or within the camera image.

[0116] In a preferred developmental form, the following values ​​are identified as characteristic values ​​at the reference point from the camera image:

[0117] One of the cameras or the rotation angle Phiz (also called the Z rotation angle) is identified around the optical axis of the camera. The rotation angle phiz may be particularly preferably identified by a linear function and / or an intersection function and / or an axial intersection function.

[0118] Furthermore, at least one of the point grids in the camera image or the local grid dimension described above is identified. The local grid dimension represents the distance between two adjacent lines of the point grid in the camera image. The local grid dimension may be identified particularly preferably by a linear function and / or an intersection function and / or an axis intersection function. The point grid in the code array consists of regularly and / or regular grid dimensions. By capturing the code array with a camera that yields a camera image, the grid dimensions are imaged with distortion, and as a result, the grid dimensions change across the camera image. The local grid dimension at a reference point is understood to be the value of the grid dimension at the reference point.

[0119] Furthermore, at least one first and second local angular divergence of the point grid in the camera image is identified. Essentially, within the point grid in the code array, lines are arranged parallel to each other. However, the point grid is distorted by the imaging of the code array onto the camera image, and therefore, lines take an angular difference of a value other than zero between any two adjacent lines. The local angular divergence of the point grid in the camera image is understood to be the angular difference between two adjacent lines of the point grid at the reference point. This angular difference is a first local angular divergence with respect to the first principal direction and a second local angular divergence with respect to the second principal direction. The first and second local angular divergences may be identified in particular via linear functions and / or linear angular functions.

[0120] The characteristic values ​​described above completely describe (from a physical standpoint) the three degrees of freedom of the camera relative to the code array: namely, the distance rz between the code array and the camera at the reference point and / or along the optical axis, and the two pitch angles phix and phiy that describe the intermediate angles between the plane of the code array and the optical axis in the first and second principal directions. Therefore, the characteristic values ​​described above are sufficient to identify the three degrees of freedom described above.

[0121] In this case, there is a further consideration that the above-described characteristic values can be easily identified from the camera image. In addition to "manually" identifying the characteristic values in the camera image, it is also possible to derive these characteristic values through digital image processing techniques. Once the characteristic values are grasped, the above-described three degrees of freedom can be estimated.

[0122] Therefore, a new method for specifying the degrees of freedom, which is excellent in that only a small number of characteristic values are used to specify the degrees of freedom by the selection of the above-described characteristic values, is proposed. As a result, it is possible to configure this method with high computational efficiency and high accuracy.

[0123] In a preferred embodiment of the present invention, the above-described characteristic values are substituted into an imaging model that describes the optical imaging of the code array on the image sensor, and the position and orientation of the camera relative to the code array are considered. This model describes the physical and thus analytical relationship between the above-described characteristic values as input values and the above-described three degrees of freedom as output values. Therefore, the specification of the three degrees of freedom is brought about by this relationship.

[0124] In a possible embodiment of the present invention, three equations that form a system of equations for the three degrees of freedom are determined. In this case, it should be emphasized that the three equations represent a possible expression of the analytical relationship between the characteristic values and the three degrees of freedom, and other mathematical or analytical expressions are possible. However, it is possible to represent the physical relationship particularly simply and compactly through the three equations and / or the system of equations.

[0125] The equation for specifying the first pitch angle phix is particularly a function of the first local angular divergence dalphac0, the distance rz between the code array and the camera, and the second pitch angle phiy.

[0126] Particularly, the first pitch angle is given by the following equation: tanφ Χ =(-dα C0 ·rZ ) / (c grid ·cos 2 φ Y ) Identified by, however, c grid : This is the grid distance of a point grid in units of [m].

[0127] The equation for determining the second pitch angle phiy is, in particular, a function of the second local angular divergence dalphac1, the distance rz between the code array and the camera, and the first pitch angle phix.

[0128] In particular, the second pitch angle is given by the following equation: tanφ Y =(-dα C1 ·r Z ) / (c grid · cosφ X ) It is identified by [the specified method].

[0129] The equation relating the distance rz between the code array and the camera is, in particular, the following function: [Table 1] However, b: Camera image width in units [m] g C0 ,g C1 : Local grid dimensions of point grids in camera images in units [m] That is the case.

[0130] Depending on the definition and / or specification of the rotation angle Phiz, the columns relate to two independent variables, and the rows relate to two alternative variables.

[0131] Alternatively or supplementally, the distance between the code array and the camera can be expressed as the mean of the variables and / or functions using the following equation: r Z =(r Z0k +r Z1k ) / 2 It is identified by [the specified method].

[0132] Essentially, the system of equations can be solved analytically. In a preferred evolution of the present invention, the system of equations, which includes three equations, is solved iteratively. In this case, first, a starting value is set, and then the degrees of freedom are determined in the iterative steps of the optimization routine.

[0133] In particular, in this method, it is assumed that first, a camera image is captured by a camera, then at least one degree of freedom of the camera relative to the code sequence is identified, and then, for example, an actuator of an automated device is driven. For example, at least one degree of freedom may be output to an optical output device such as a display. At least one degree of freedom may be used to control and / or adjust the position of the actuator of the automated device by using this at least one degree of freedom as an actual value. A robot equipped with a sensor unit of an automated device can, for example, identify its absolute orientation relative to the code sequence and output this absolute position as actual information, or it can travel to a predetermined further position, and the robot is further oriented in accordance with the code sequence with respect to its actual position.

[0134] Further aspects of the present invention relate to a control unit configured to carry out the methods described above, and / or an automated apparatus comprising the control unit. Optionally, the control unit includes and / or is data-technically connected to the camera.

[0135] A further subject of the present invention is a computer program configured to carry out the above-described method when executed in a digital data processing device and / or control unit.

[0136] A further subject of the present invention is a machine-readable storage medium equipped with a computer program.

[0137] Further features, advantages, and effects of the present invention will become apparent from the following description of preferred embodiments and the accompanying drawings. [Brief explanation of the drawing]

[0138] [Figure 1] This is a flowchart of the entire method, including one embodiment of the method according to the present invention. [Figure 2] This is a schematic diagram of an optical model in 3D. [Figure 3] This is a schematic diagram of an optical model in 2D. [Figure 4] This is a schematic diagram of an optical model in 2D. [Figure 5] This is a schematic diagram of the coordinate system. [Figure 6] This is a schematic diagram of the coordinate system. [Figure 7] This is a schematic diagram of the coordinate system. [Figure 8] This is a schematic diagram of the coordinate system for the imaging model. [Figure 9] This figure shows the imaging model in Figure 8, which has an inclined code plane. [Figure 10] This is a diagram illustrating the code array. [Figure 11] This is a further diagram illustrating the code array. [Figure 12] This figure shows a dot grid of code arrays with two exemplary filled-in reading fields. [Figure 13] This diagram shows various different angular orientations of the reading field. [Figure 14] This figure shows an example of a read field. [Figure 15] This figure shows an example of the structure of a parcel within a code array. [Figure 16] This figure shows a further illustrative structure of parcels within a code array. [Figure 17] This is a flowchart of the effectiveness test. [Figure 18] This diagram shows the details of the effectiveness test. [Figure 19]This diagram exemplifies a camera image with an identified start base and a read field. [Figure 20] This figure shows an example of a basic symbol matrix with data entered into a face. [Figure 21] This is a flowchart for identifying the starting point. [Figure 22] This figure shows multiple predefined starting positions within the camera's image field. [Figure 23] This diagram illustrates a method for identifying the starting base. [Figure 24] This diagram illustrates a method for identifying the starting base. [Figure 25] This is a flowchart of the function Center_Pos. [Figure 26] This is a diagram explaining the function Center_Pos. [Figure 27a] This is a diagram illustrating the Nearest_dot function. [Figure 27b] This is a diagram illustrating the Nearest_dot function. [Figure 28] This is a flowchart illustrating the process for detecting all dots / basic symbols within a reading field. [Figure 29a] This diagram illustrates the flow for detecting all dots / basic symbols within a reading field. [Figure 29b] This diagram illustrates the flow for detecting all dots / basic symbols within a reading field. [Figure 29c] This diagram illustrates the flow for detecting all dots / basic symbols within a reading field. [Figure 29d] This diagram illustrates the flow for detecting all dots / basic symbols within a reading field. [Figure 30] This diagram illustrates the detection of basic symbols within a distorted grid. [Figure 31] This is a diagram illustrating the parallelization process. [Figure 32]It is a flowchart for classifying basic symbols. [Figure 33] It is a flowchart for identifying the rough position. [Figure 34a] It is a diagram showing a basic symbol matrix with entered data and a reading track. [Figure 34b] It is a diagram showing a basic symbol matrix with entered data and a reading track. [Figure 35] It is a diagram explaining the decoding of the rough position. [Figure 36] It is a diagram explaining a compensation line for a linear function. [Figure 37] It is a diagram explaining the straight line angle. [Figure 38a] It is a diagram explaining the compensation line and the linear function. [Figure 38b] It is a diagram explaining the compensation line and the linear function. [Figure 38c] It is a diagram explaining the compensation line and the linear function. [Figure 39] It is a diagram explaining the identification of the fine position and the rotation angle phiz (Z rotation angle). [Figure 40] It is a flowchart for identifying the camera distance and the pitch angle.

Embodiments for Carrying Out the Invention

[0139] This method is disclosed to implement an algorithm for accurately identifying the absolute 6D position of camera 1 based on the flat code array 2 photographed by camera 1, and the algorithm receives the camera image 3 as input information.

[0140] Code array 2 is used simultaneously as an analog and digital scale in two dimensions, X and Y. Code array 2 consists of multiple symbols arranged in a regular grid. A binary code is preferred, consisting of two symbols arranged in a square grid, namely a small round dot for digital 0 and a larger dot for digital 1. The dot type contains digital information, while the dot center contains analog information.

[0141] Camera 1 detects a segment of code sequence 2 and transmits the camera image 3 to a computer or any data processing device. The algorithm selects a reading field within the camera image 3 with the smallest size code cell (e.g., 7x7 dots) and calculates the position of camera 1 relative to code sequence 2 in 6 dimensions. To this end, the method utilizes both digitally encoded positional information and the precisely measured center positions of all dots within the measurement field or reading field. These dots form a grid, which is distorted by the camera's viewpoint and the distortion of the objective lens.

[0142] The method for position detection includes the following steps (see Figure 1): Step 100: Load camera image 3 into the computer's main memory. Step 200: Optional: Perform an initial inspection of image quality based on characteristic data such as brightness and contrast.

[0143] Step 300: Evaluate the XY rough position and optionally evaluate the z rough angle. Step 310: Locate the 3x3 dot starting field (launchpad) within the reading field. Step 320: Detect the dot in the reading field of the camera image. Step 330: Accurately measure the center position and area of ​​the dot. Step 340: Optional: Mathematically correct objective lens distortion by parallelizing the dot positions. This transforms the curved lines in the dot grid within the camera image 3 into straight lines. Step 350: Classify the dots based on their area and assign them to the binary digits 0 and 1. Step 360: Read the digital code in the axial directions X and Y, and evaluate the absolute rough position in the X and Y directions, and optionally the z rough angle as φ Z Identify the absolute rough position in the direction.

[0144] Step 400: Evaluate the distortion Step 410: Fit the compensation lines (beams) to each row and column of the dot grid of the reading field. Step 420: Fit the compensation function to the first bunch of lines by interpolating all lines in each row of the code grid, and fit the second compensation function by interpolating all lines in each column of the code grid. Each bunch of lines is fully described by six interpolation parameters (bunch data). From here, the interpolation function can also be used to calculate an interpolated line that lies between two measured lines.

[0145] The 6D camera position is calculated from characteristic values ​​that describe the distorted grid within the camera image, obtained from the measured dot positions. The equation for calculating the position is derived from the optical inverse imaging model of camera 1 and the laws of geometrical optics: Step 500: Evaluate the XY precision position: The position in the X and Y directions is calculated from the coarse XY position and the grid orientation relative to the optical axis orientation (image center).

[0146] Step 600: Evaluate the Z-angle from the angle of the (interpolated) line at the center of the image.

[0147] Step 700: Evaluate the Z position and XY angles. Step 710: Derive interpolated characteristic values ​​from bunch data describing a grid with perspective distortion at the center of the image: • Grid dimensions of the lines in two bundles of lines • Angular divergence of two bundles of straight lines The camera position in the Z direction is primarily determined by the dimensions of a straight grid at the center of the image. Camera angle φ X and φ Y This is primarily determined by the angular divergence of two line bundles at the center of the image. The iterative algorithm generates variables Z and φ. X and φ Y The system of equations connecting these factors is solved numerically.

[0148] Step 800: Outputs 6D position information and validity information as an optional choice obtained from the results of numerous diagnostic functions in the flow.

[0149] The algorithm concentrates camera image data into a small number of data points relevant to localization with minimal computational cost, improving accuracy through interpolation and behaving robustly even when noise is present in the camera image. The following flow positions (the numbers are examples) are particularly important: Flow position I: Input data is a camera image (e.g., 200 x 200 pixels) 40,000 values Flow position II (step 300): Camera image data is converted to dot data. 2250 values Flow position III (step 400 / 500): Dot data changes to beam data 90 values Flow position IV (step 400 / 500): Beam data changes to bunch data. 12 values Flow position V (steps 500 / 600 / 700): Bunch data changes to position 6D. 6 values

[0150] Flow positions II-V have the following special features. Flow position II: a) By mathematically modeling the code grid with perspective distortion in code array 2, dots in camera image 3 are discovered quickly. This method is based on the already measured dot positions. → Advantage: Dots are discovered quickly, and time-consuming searches within the pixel grid are omitted. It is robust against noise in the camera image. That is, if a dot is not discovered at the expected position, it is classified as "invalid". Invalid dots are not processed subsequently, but generally do not interfere with subsequent sequences b) A stack-based algorithm for quickly and robustly searching for dots within the code grid of code array 2. → Advantage: The search method is error-tolerant because it completely surrounds these incorrect dots without being interrupted at incorrect dots. It is accurate (a high yield of dots is discovered per read field), and robust (robust because dots with many valid neighbors are preferentially detected) c) Function Center_Pos: A fast method for specifying the position and area of dots in the camera image with sub-pixel accuracy → Advantage: It is fast (few pixel accesses), accurate (due to sub-pixelization), and robust (uses a dynamic contrast threshold instead of a gray value threshold) d) Parallelize the dot positions instead of the entire camera image. → Advantage: It is fast (only 225 positions are parallelized instead of all 40,000 pixels in the camera image) e) Locally normalize the dot area with respect to the area of the dot grid cell. The normalized dot area is used to classify the dots. → Advantage: The digital code is read robustly (dot classification is resistant to perspective distortion and changes in the distance between the camera and the dots) f) Digital code reading, error detection, and partially error correction are performed redundantly. →Advantages: Low error rate when noise is present in the camera image, and correct readings are performed even if invalid dots are present.

[0151] Flow positions III and IV: g) The dot positions extracted from camera image 3 are compressed as bunch parameters and accurately represented (only 12 values). →Advantages: It is fast (due to the small amount of data), accurate (through two-stage averaging and best-fit interpolation: from dots to compensation lines, and from compensation lines to line bundles), error-tolerant, and robust (by excluding invalid dots from subsequent processing).

[0152] Flow position V: h) A mathematical method for converting 12 bunch parameters to 6D positions.

[0153] As an advantage, some or all of the following improvements will be achieved: • Complete 6D position measurement, absolute position information • All dimensions are detected simultaneously using a single measurement process (within the camera image). • The position measurement rate is fast and the delay time is short. This method requires only a few calculation steps on the computer. Using an embedded computer, a typical measurement rate of approximately 100Hz to 10000Hz can be achieved, and therefore the sensor can be used, for example, in a closed attitude control circuit. This method achieves high read speeds by minimizing the number of accesses to camera pixels. Time-consuming planar camera image manipulation is avoided. Furthermore, the amount of data in each processing step is significantly reduced. • Redundant reading of point codes ensures high reading reliability. Position detection is possible even when noise is present in the camera's image field, such as in cases of localized occlusion, invalid dots, or unfavorable lighting conditions. The validity check of the processing step identifies an incorrect state and prevents the output of unreliable position values. • High accuracy in position measurement across all degrees of freedom. This is achieved through a variety of different means, as follows: ○ Averaging across a large number of dots per camera image, for example, more than 100 or 1000 dots. ○ Classify the dots according to their effectiveness and exclude ineffective dots from subsequent processing. ○ Identify the position of the dot edges with sub-pixel precision. ○By using a contrast threshold instead of a gray threshold, robustness to localized changes in image brightness is increased. ○ Because the dots are round, there is almost no centering error when the code plane is tilted or rotated. · φ Z The measurement range is not limited. The measurement range in X and Y is virtually unrestricted. For example, using a code with 9x9 dot code cells and a 2.5mm dot grid, a measurement range of 550,000km x 550,000km can be achieved. • Angle measurement range φ Χ ,φ Y However, it has a wide range of approximately + / - 50° and high precision. It can be extended up to 360° by a spatially distributed code array on the object. • To calculate the camera's position from camera image data, an inverse imaging model based on geometrical optics is used. This method can be used even without active lighting, and therefore, for example, camera images taken under ambient light using a smartphone can be processed. • The error rate is minimized. This method identifies the validity of position measurements by checking each processing step for errors and validity using different diagnostic methods. Validity is output along with the position measurements. In this case, the goal is to output only valid measurements for subsequent processing.

[0154] Further advantages are as follows: Specifically, a single measuring sensor configured as Camera 1 detects the position of one or more objects in six dimensions. Unlike systems with multiple distributed sensors to detect individual degrees of freedom, the overall installation cost and complexity of the system are reduced, resulting in cost advantages. By detecting all six degrees of freedom, the system components are reduced. For example, a rotary angle encoder usually requires a rotary bearing for the measuring axis to prevent measurement errors caused by lateral displacement of the code disk. According to the proposed method, since the complete 6D position information of the code disk is detected simultaneously as a code sequence, mechanical guides can be omitted. Lateral displacement of the sensor will not cause errors in angle measurement. This reduces the cost and mechanical complexity of the position measurement system. This enables the realization of positioning systems where assigning position sensors to multiple axes is structurally impossible. For example, it allows for the realization of a floating, flat robot capable of 6-dimensional positioning without providing a structural connection between the robot and a stator located beneath it. • By detecting multiple dimensions simultaneously, measurement errors that could occur due to differences in measurement timing when multiple sensor systems are distributed are reduced. The cost of a sensor is largely independent of the number of degrees of freedom detected. Therefore, sensors can be advantageously used even in applications requiring fewer than six degrees of freedom. The additional information provided enables additional functions, such as system self-diagnosis or continuous observation of the operating state ("state monitoring"). This method is scalable, for example, by changing the grid dimensions of the code array and adapting the imaging optical system to the code array. The resolution can be varied over many orders of magnitude, for example, from the nanometer range (e.g., nanometer positioning systems) to the meter range (e.g., automated landing of aircraft or drones at airports where the airfield is marked by a code array). The resolution-to-measurement range ratio can involve many orders of magnitude. For example, if a path measurement system with a resolution of 10 nm is combined with a code sequence 10 m long, the resolution-to-measurement range ratio is 1:10. 9 That is the case. Sensors using this method are easy to install and can be configured by parameterizing them for specific applications, thus offering high flexibility in use. This is particularly advantageous when usage conditions change frequently. This method can read additional information contained in the location code. For example, it can read object identification data in addition to the location. When image capture is performed periodically, this method provides one independent position estimate for each individual image, and this position estimate does not depend on preceding information from previous images. Therefore, the measurement rate corresponds to the refresh rate.

[0155] According to this method, the position of at least one camera 1 relative to at least one object 8 can be determined with six degrees of freedom, and at least one code sequence is attached to the surface of each object 8. Camera 1 detects the code sequences 2 on the object 8 and displays these code sequences 2 completely or partially in the camera image 3. The 6D position of the object 8 is calculated from the camera image 3 using the proposed method and further conventional mathematical / technical methods. Code array 2 forms a digitally encoded planar scale. Code array 2 includes multiple different symbols, preferably circular symbols (dots), which are arranged in a regular, preferably square, grid, allowing for positioning in six degrees of freedom.

[0156] Camera 1 includes at least the following: • Imaging sensor element, usually a camera chip, especially an imaging device 12. • An imaging system, particularly the objective lens 9, that forms a clear and high-contrast image of the scale onto the sensor element. The imaging system includes a wide-angle objective lens, especially as the objective lens 9, because a central projection is required to identify all six degrees of freedom. • An interface for outputting camera image data and / or camera image 3. Optionally, the camera system of camera 1 is included. • A lighting system specifically for illuminating the code sequence. This allows camera 1 to be more independent of ambient lighting conditions. Shorter measurement times are possible during flash operation, which also allows for the detection of fast-moving objects. Light-emitting diodes, for example, are used as the light-emitting means. • Means for blocking external light, such as an aperture or optical filter. A spectral filter may be provided in the light path of the imaging system to ensure that only light from a limited wavelength range reaches the sensor element. Ideally, a monochromatic illumination means with wavelength characteristics similar to those of a color filter is used. Additionally, a computing system is required, for example, as a digital data processing device for information processing, configured as a control unit. The computing system receives digitized camera image data of camera image 3 as input information, and on the output side, provides the identified 6D position of the identified object and optionally an evaluation of the validity of the position measurement.

[0157] In the computing system, this method is implemented as an algorithm. The algorithm is executed in response to external requests, or periodically at fixed time intervals, for example, to detect the movement trajectory of an object (tracking).

[0158] The computing system may be configured as, for example, an embedded system, a parallel computer, a GPU, an FPGA, an ASIC, or a cloud system. A smartphone can also be used as the overall system for position detection, and its built-in camera 1 may be used for image capture, sometimes with active lighting. This method may be implemented in a smartphone application and may be implemented embedded in a smartphone or the cloud.

[0159] Figure 2 shows a typical apparatus for determining absolute position with six degrees of freedom. Camera 1 captures a camera image 3 of a flat code array 2. The method according to the present invention calculates the 6D position of the camera in the coordinate system of the code array 2 from the perspective-distorted camera image 3, and generates a position vector (r X ,r Y ,r Z ) and angular vector (φ X ,φ Y ,φ Ζ Output as ).

[0160] coordinate r X and r Y This is the Cartesian coordinate system (X) of code array 2. C ,Y C It is shown in ). Coordinate r X and r YThis describes the intersection point 4 between the optical axis 5 of camera 1 and the plane on which the code array is located. Since the optical axis 5 is perpendicular to the plane of the imaging device 12, the optical axis 5 can be described by point 7 within the camera image 3.

[0161] This method identifies the intersection point (X,Y) even if code array 2 is displayed only at the edge of the image field and not at the location of optical axis 5.

[0162] distance r Z It is located on the optical axis 5 and extends from intersection 4 to the optical center of the objective lens of camera 1. Since the optical axis 5 is not always perpendicular to the code array 2, r Z Generally speaking, (X C ,Y C ) together with a non-orthogonal coordinate system. Using the angle vector, the position vector (r X ,r Y ,r Z ) can be converted to a Cartesian coordinate system.

[0163] According to this method, multiple simultaneously captured objects 8 can be located within the camera's image field, even if these objects 8 partially overlap. In order for the location to be read, at least one region within the size of the code parcel (e.g., 7x7 dots) in the camera image 3 must be recognizable.

[0164] Figures 3 and 4 schematically show the light ray path of camera 1. In Figure 3, camera 1 equipped with an objective lens 9 and an imaging device 12 can be seen, and its line of sight is directed downwards onto the code plane of the code array 2. The optical axis 5 is indicated as a vertical dashed line, and the focal points of the objective lens 9 are indicated as points 10 and 11.

[0165] The vector arrow G on the code plane of code sequence 2 is imaged onto the imaging device 12 as vector arrow B according to the laws of optical optics. The observation ray 14 intersects the optical axis 5 at a point referred to herein as the optical center 13 of the objective lens 9.

[0166] Figure 4 shows the complete ray path, and the objective lens 9 is simplified as a lens. Ray optics, with the theorems of parallel lines and ratios of line segments, and the lens equations, forms the basis for a mathematical method for position determination. In this case: B / G = b / g and 1 / f = 1 / b + 1 / g The following holds true, however, B Image size b Image width G Object size g Object width f focal length That is the case.

[0167] Figures 5, 6, and 7 show the coordinate systems involved:

[0168] The two-dimensional coordinate system 15 of the imaging device 12 is the axis (X) of the camera chip. I ,Y I It is stretched by ). The position on the imaging device 12 is indicated by pixels, and the pixel rows and columns of the camera chip are numbered consecutively. In this example, an imaging device 12 with 200 x 200 pixels is assumed. From the numbering, integer X and Y position values ​​can be obtained for each pixel. In the method of interpolating gray values ​​between adjacent pixels by sub-pixelation, real numbers may be generated as camera positions during evaluation. To convert the unit of position from [pixels] to [m], the distance p in units [m / pixel] between adjacent pixels on the camera chip is used. grid However, it is multiplied by the position. Pixel distance p grid This is one of the characteristics of the imaging device 12. Each pixel provides an integer gray value.

[0169] Coordinate system 16(X) of code array 2C ,Y C ) is stretched by the main axis of code array 2, which corresponds to a square grid of dots. A third axis Z is perpendicular to code array 2. C It is extended to a 3D coordinate system centered on Z, and code array 2 is Z C It is positioned at =0. The unit is [dot], i.e., the numbering of consecutive dot rows and dot columns, or (dot grid distance c in units [m / dot]). grid [m] is the result after multiplication. Code array 2 is the coordinate system (X C ,Y C It uniquely encodes the local location in ).

[0170] The 3D camera coordinate system (X,Y,Z) 17 is fixedly linked to camera 1. The origin of the 3D camera coordinate system (X,Y,Z) 17 is located on the optical axis 5 between the imaging device 12 and the code array 2, at a distance of twice the image width b from the imaging device 12. The optical axis 5 forms the Z axis, and camera 1 is oriented in the Z direction.

[0171] The coordinate system for the imaging model in Figure 8 corresponds to the camera coordinate system 17(X,Y,Z). In this case, b is the image width of camera 1. The image plane is located at a height of (0,0,2b) and is parallel to the X / Y plane of the camera coordinate system 17. As a model representation, a further "virtual" image plane 18 can be constructed within the (X,Y) plane of coordinate system 17, and the virtual image is mirrored point-symmetrically with respect to the real image on the imaging device 12, with respect to the image center.

[0172] The code plane of code array 2 is located at height (0,0,z0), where (z0<0). When the camera orientation is (0°,0°,0°), the code plane is located parallel to the X / Y plane of camera coordinate system 17, and the axis (X) of coordinate system 16 of code array 2. C ,Y C ) is the axis (X) of the coordinate system 15 of the imaging device 12. I ,Y I It points in the same direction as ). This method, ·Translation:(r X ,r Y ,r Z ) in, and ·Rotation:(φ X ,φ Y ,φ Z ) Identify the 6D position of the camera. The rotation center related to the tilt angle phix and rotation angle phiy of camera 1 is located at the point (0,0,z0). The roll angle phiz is measured around the optical axis 5.

[0173] Figure 9 shows an imaging model with a tilted code plane for code array 2. Value r Z r is the distance between the rotation center (0,0,z0) and the optical center of the objective lens (0,0,b), and therefore r Z The equation = b - z0 holds true.

[0174] Starting from the theorems of parallel lines and the ratio of line segments, and geometrical optics, the point (x0, y0) in the code plane and the point (B) in the image plane X ,B Y The image formation on ) can be mathematically described as an imaging equation in camera coordinate system 17: • Optical center 13 of objective lens 9:

number

number

number

number

number

number

number

number

number

number

number

number

[0175] 2.4.3 Code Array This method requires a planar encoded scale within a code array 2, from which a sub-region is read using an imaging sensor, such as a camera 1, thereby allowing the orientation of the sensor and / or camera 1 relative to the scale to be determined from the image information in up to six spatial directions. Further details regarding the code array 2 are evident in the applicant's German Patent Application Publication No. 102016216221, the contents of which are incorporated herein by reference, particularly with respect to embodiments of the code array and decoding and variables.

[0176] The encoded scale is applied to a surface as a sensor-readable marking formed as code array 2, the surface extending substantially in two dimensions, but which may be curved. For better explanation, it is assumed below that the encoded scale is printed on a plane as an optically readable pattern, but the claims are not limited to these marking and sensor principles, or to a plane.

[0177] The encoded scale of code array 2 is formed by arranging multiple different basic symbols in a regular grid. Each basic symbol carries two pieces of information: its shape encodes digital information, and its area centroid marks a specific position on the surface.

[0178] In a simple example, digital information is encoded in a binary number system with base b=2. In this case, only two basic symbols 20 are used, for example, a small circle and a large circle that symbolically represent the values ​​"0" and "1". The area centroids (centers of the circles) of the two basic symbols 20 mark grid points on the surface.

[0179] The area centroid of the 20 basic symbols is, for example, the basic distance c between adjacent symbols in the X and Y directions, in a square grid as shown in Figure 10. gridBy placing this, a periodic two-dimensional pattern is formed on the scale plane of code array 2.

[0180] By dividing the plane into multiple parcels 19 of equal size that fill the surface, multiple basic symbols within one parcel 19 are combined into a single logical unit. Preferably, square parcels 19 are used. Figure 10 shows a dot grid with square parcels 19, each capable of containing 7 x 7 binary symbols, which corresponds to a maximum information content of 49 bits. The lines and squares drawn in Figure 10 are for illustrative purposes only and are not shown in the actual code array 2. Figure 11 shows an exemplary code array 2 without lines, with basic symbols 20 within the parcels 19.

[0181] Each parcel 19 is similarly marked by a parcel symbol 21, so that the regular arrangement of the basic symbols 20 is superimposed by the regular arrangement of the parcel symbols 21. For example, the parcel symbols 21 can be represented by omitting the basic symbols 20.

[0182] In Figures 10 and 11, the parcel symbol 21 consists of empty grid points located in the center of each parcel 19 within the point grid. The parcel symbol 21 enables recognition of the orientation of the parcel 19, thereby ensuring that the basic symbols 20 are read in the correct order along the reading track. Since the parcel symbol 21 occupies grid space, the information content of a single parcel 19 is reduced to 48 bits in this example.

[0183] The reading field 22 is a field on the encoded scale of the code array 2 that has the size of at least one parcel 19. The reading field 22 is connected to the point grid but not to the grid of the parcel 19.

[0184] Figure 12 shows a point grid of code array 2 with two exemplary filled-in reading fields 22. The orientation of the reading fields 22 in the coordinate system of the point grid is defined by their center position 23. The orientation of the reading fields 22 is defined by the base distance c grid It can be changed in integer increments.

[0185] Furthermore, the reading field 22 has an angular orientation that can change in 90° increments relative to the coordinate system of the point grid in the case of a square grid. In Figure 12, the angular orientation is visualized by markings at the corners of the reading field. Possible angular orientations are shown in Figure 13. The reading field 22 illustrated in Figure 12 is uniquely described by the following information: Left side reading field 22: Position = (4, 11); Angle orientation = 0° Right-hand reading field 22: Position = (12, 6); Angle orientation = 90°

[0186] The code of code array 2 is constructed such that the basic symbol 20 in the reading field 22 contains a sufficient amount of information to digitally encode the location (X,Y) and direction of the reading field 22 in the coordinate system 16 of code array 2. X and Y are the basic distance c grid The position is shown as an integer multiple of (rough position). Basic distance c grid The precise position in fractions and the exact angle in fractions of 90° are not digitally encoded, and are determined by the precise positioning of the base symbol 20 in the camera coordinate system 17.

[0187] Figure 14 illustrates a reading field 22 with a 15x15 space for basic symbols 20. This reading field 22 provides more information than a reading field 22 of 7x7 symbols, which is the minimum size required for one parcel 19. The redundant information is used for error recognition and / or error correction.

[0188] Figure 15 illustrates the structure of a parcel 19 with a 7x7 grid of points. A parcel symbol 21 is located in the center of parcel 19. Two fields 24 characterize the X-parcel region, which displays the subsequent code relating to the position in the X direction. The X-parcel region contains 24 grid points and, accordingly, represents a code with a length of 24 bits.

[0189] To read the code, the basic symbols 20 are read from left to right, column by column, and from top to bottom within each column. The reading order (reading track) within the X parcel area 24 is determined by the grid point coordinates (X C ,Y C ):(0,6);(0,5);(0,4);(1,6);(1,5);(1,4);(2,6);(2,5);(2,4);(3,6);(3,5);(3,4);(3,2);(3,1);(3,0);(4,2);(4,1);(4,0);(5,2);(5,1);(5,0);(6,2);(6,1);(6,0).

[0190] The two fields 25 characterize the Y parcel region, which displays the subsequent codes relating to the position in the Y direction. The reading order corresponds to the reading order of the X parcel region 24, but rotated 90° counterclockwise. That is, the reading tracks extend from bottom to top row by row, and from left to right within each row.

[0191] The code from the X parcel area 24 (X code) is a subsequence of a whole sequence of g digits, with a length of t = 24 digits, where g is considerably larger than t. The number g is large enough to completely fill the X parcel area 24 of all adjacent parcels 19 in the X direction. In particular, g is greater than 50, especially greater than 1000, and especially greater than 1,000,000. The contents of the X parcel area 24 of adjacent parcels in the Y direction are identical. The whole sequence is structured such that each subsequence of t consecutive basic symbols 20 is included in the whole sequence exactly once when read in the forward direction, and each subsequence read in the backward direction is not included in the whole sequence when read in the forward direction. The code (Y code) from Y parcel area 25 is shown inverted, meaning each digit z is replaced by the digit (b-1-z). In the case of a binary system with b=2, this corresponds to bitwise inversion. The inverted Y code is a subsequence of the whole sequence with a length of t=24 bits, and the same whole sequence can be used as for the X code. The inverted Y code also appears only once in the whole sequence and does not appear in the whole sequence when read backward. The contents of the Y parcel area 25 of adjacent parcels 19 in the X direction are identical.

[0192] The whole sequence is constructed such that the sum of the digits of any subsequence is less than half the maximum possible sum of the digits of any subsequence. In particular, the sum of the digits of any arbitrary subsequence taken from the whole sequence is less than half the maximum possible sum of the digits of any subsequence. For example, the sum of the digits of any subsequence of length t for a number system with base b is less than q = t·(b-1) / 2.

[0193] A subsequence of the overall sequence encodes a coordinate value, for example, the X-coordinate of the starting position of the reading process. Similarly, a subsequence of the inverted overall sequence encodes a Y-coordinate, for example, the starting position read in the subsequence. The X and Y codes read within parcel 19 are associated with the X and Y coordinates of a reference point within parcel 19, for example, the center of the reading field.

[0194] Each reading field 22 contains exactly one parcel symbol 21, t basic symbols from the X parcel area 24, and t basic symbols from the Y parcel area 25. From the position of the parcel symbol 21, the orientation of the parcel grid, and consequently the orientations of the X parcel area 24 and Y parcel area 25 within the reading field 22, and the corresponding reading order can be derived. The basic symbols 20 read according to the reading order generate a t-digit sequence that is a subsequence of the whole sequence (possibly after inversion).

[0195] If the orientation of the reading field 22 is unknown, it is initially unclear which of the two axes is the X-axis and which is the Y-axis. This can be determined by calculating the sum of the digits of the read code and comparing it with q. That is, the sum of the digits of the X code is less than q, and the sum of the digits of the Y code is greater than q. From the overall sequence and the direction of the referenced read code (forward or backward), the orientation of the coordinate axes of the reading field 22 relative to the coordinate axes of the code sequence 2 can be uniquely derived.

[0196] In this way, the orientation and direction of the reading field 22 in the coordinate system 16 of the code array 2 can be determined; that is, the orientation can be determined in integer increments of the grid width, and the direction can be determined in integer increments of 90°.

[0197] On the left side of Figure 16, a parcel 19 is shown, with correspondingly 9 rows and 9 columns. In addition to the X-parcel area 24 and Y-parcel area 25 representing the position code for each axis of the coordinate system 16, a parcel area 26 for additional data is provided. The parcel area 26 for additional data contains (5 × 5) grid points, the central grid point of which is left empty for the parcel symbol 21, thus leaving 24 grid points, each capable of accommodating one basic symbol 20. In this way, 24 bits of additional information, which are not necessary for position identification and are read in a specified order, can be represented within each parcel 19. On the right side of Figure 16, a further code array 2 with additional data is shown. Here, four parcel areas 26 are provided, each with 10 grid points for additional data, so a total of 40 bits of additional information are available for each parcel 19.

[0198] The algorithm for localization is shown in Figure 1 as a progression of data processing steps, and this progression will be described in detail below based on an example. Standard methods for mathematics and image processing, as well as for diagnosis and error recognition, are not described in detail.

[0199] This method is optimized to improve localization accuracy and enable high-speed implementation. The data volume is dramatically reduced through these steps, which further facilitates high-speed implementation.

[0200] At flow position I in step 100, digital image data as a matrix of gray values ​​is transmitted from camera 1 to the computer. Step 200 is used to roughly check the validity of the image data based on characteristic values. Further steps for image processing may follow, for example, to prepare the image data or to segment it into individual code regions. These steps are not shown in detail here. The data capacity is 40,000 pixels in the case of a 200x200 camera image 3, and therefore 40,000 bytes in gray values.

[0201] At this flow position, symbols (dots) contained within the image are located and entered into a dot matrix corresponding to the size of the reading field (here, 15x15 dots), according to their respective placements within the dot code grid. Each symbol is measured for its position and area. The position is parallelized to correct for objective lens distortion. The area of ​​the symbols is normalized to remove the effects of perspective distortion. Based on the normalized area, the type of symbol is classified. This reduces the data size to 225 dots with metadata, and consequently to 9000 bytes. To classify unoccupied grid points or incorrectly imaged symbols as such, a local model of the dot code grid in the camera image is created from the positions of successfully identified symbols. Within the reading field, symbols are searched for at all grid points in the model. Due to local occlusion, image errors, or image field boundaries, not all symbols can always be identified. In such cases, their corresponding grid positions are classified as "invalid".

[0202] At flow position III, compensation lines (beams) are fitted to match the rows and columns of valid dots in the reading field 22. Each compensation line is described by the angle of the line in the image field of camera 1 and its intersection with the X or Y axis of the image field coordinate system. Each reading field 22 supplies two bundles of compensation lines, corresponding to the two principal axis directions of the code array 2. In this example, each line bundle (bunch) contains a maximum of 15 lines. This reduces the position data of 15 × 15 = 225 dots to 15 + 15 = 30 compensation line data.

[0203] At flow position IV, the lines of each bundle are interpolated by two quadratic polynomials. That is, one polynomial describes the angle of the line, and the second polynomial describes the axis intersections of the lines in the bundle of lines. The polynomial parameters of the two bundles are expressed by 12 real values, which is equivalent to a reduction to one-fifth compared to the linear representation.

[0204] At flow position V, a mathematical inverse imaging model is also used to calculate the 6D position of camera 1 from the polynomial parameters of two linear bundles. Additionally, the validity of the position values ​​is estimated by evaluating numerous diagnostic results from individual steps of the program flow. The 6D camera position and validity are output to a higher-level system in step 800.

[0205] Step 200 - Image Data Inspection / Validity Test Based on statistical characteristic data, it is checked whether an evaluable camera image 3 exists (see flowchart in Figure 17). In this example, image brightness B and contrast C are estimated and checked to see if they comply with predetermined limits. If they deviate from the predetermined limits, further evaluation is stopped and the result is classified as invalid. To save computation time, only a small number of pixels, for example, only the gray values ​​at the intersections of lines and circles within intersection pattern 27, are used to determine the characteristic number (Figure 18).

[0206] In the lower step 210, a set of pixels specified according to an arbitrary intersection pattern 27 is selected. Pixels at intersections within the intersection pattern 27 are used. Thus, G i is the grayscale value of the pixel at position i, i = 1 ··· n.

[0207] In the lower step 220, the image luminance B and the contrast C are estimated. B = mean(G i ) C = (max(G i ) - min(G i )) / (max(G i ) + min(G i ) + 1)

[0208] In the lower step 230, the image luminance B and the contrast C are evaluated by the following conditions: (B ≤ B min ) or (B ≥ B max ) (C < C min ) If one of these conditions is satisfied, further evaluation is interrupted and the result is classified as not valid.

[0209] Detection of dot data / Steps 310 to 350: The objective is to identify, measure, and classify dots as basic symbols 20 within a reading field 22 of a predetermined size (here, 15 × 15 grid points). As a result, a two - dimensional dot matrix 29 is output, or generally a basic symbol matrix with dot data is output. In this case, the indices of the matrix 29 are associated with the rows and columns of the code array 2. A characteristic value is specified for each dot.

[0210] The variable Dot.Typ includes the classification result. If Dot.Typ is positive, the dot is evaluated as valid: • Dot.Typ=2: Large dot; Logical "1" • Dot.Typ=1: Small dot, logical "0" Dot.Typ=0: Missing dot; Parcel symbol 21 If Dot.Typ is negative, the dot could not be uniquely identified.

[0211] In partial step 310, the start base / start field (launch pad) 28 is first searched for within the image area of ​​the camera image 3 and / or within the read field 22 by the dot code of the code array 2. The start base / start field (launch pad) 28 is understood here to be, for example, a field of 3x3 adjacent dots of type 1 or type 2.

[0212] Figure 19 illustrates a camera image 3 with an identified starting base 28 and a read field 22, in which a dot as a basic symbol 20 is searched for (starting from the starting base 28) within this camera image 3.

[0213] In partial step 320, a local model of the dot grid around the starting base 28 is constructed using the starting base 28. Further dots are searched for in adjacent grid spaces predicted by the model, accurately measured if successful, and entered into the dot matrix 29 as valid dots. This process is repeated periodically, thereby recording more and more valid dots around the starting base 28 until the 15×15 dot reading field 22 is fully detected. This search process is robust to local reading errors; that is, if individual dots cannot be uniquely identified or if they extend beyond the edge of the image field, those dots are characterized as invalid in the dot matrix 29 and excluded from subsequent processing. Figure 20 shows the result of the successfully classified dot matrix 29 based on the camera image 3 in Figure 19.

[0214] In partial step 330, the center position and area of ​​the dot are measured.

[0215] Partial step 340 converts the measured positions of all valid dot locations in the reading field 22 into parallelized coordinates. This step is used to compensate for the distortion aberration (barrel distortion) of the wide-angle objective lens. After parallelization, points that lie on a straight line in the code coordinate system 16 also lie on a straight line in the parallelized image in the coordinate system of the imaging device 12. Optical parallelization can be performed using the corresponding objective lens 9 instead of computer-aided parallelization; therefore, partial step 340 is an optional choice.

[0216] In sub-step 350, the dots are classified based on the measured data. Then, in sub-step 360, the digital codes are read in two spatial directions and converted using a code table into integer positional information and coarse directional information in 90° increments. The identification of the coarse position in the upper step 300 is then completed.

[0217] Regarding partial step 310 - Identify the starting base The flow for identifying the starting base 28 is shown in the flowchart in Figure 21.

[0218] Sub-step 310.1: First, a starting position 30 is defined for searching for a first dot in the starting base 28. Figure 22 shows several predefined starting positions 30 in the image field of camera 1. The search begins at one of these starting positions. If the flow fails (for example, due to noise in the image field), a second search is started at the second starting position, and so on, until sub-step 310.1 can be successfully completed. If all starting positions have been used and the search has not been successful, sub-step 310.1 is terminated with a negative result.

[0219] Lower step 310.2: Starting position 30

number

number

[0220] Lower step 310.3: Next,

number

number

[0221] Sub-step 310.4: Then, the connection vector

number

[0222] Lower step 310.5:

number

number

number

[0223] Sub-step 310.6: Three points

number

number

number

number

number

[0224] Lower step 310.7: Left-handed coordinate system (condition:

number

number

number

[0225] Lower step 310.8: Vector

number

number

number

[0226] Substep 310.9: If no dot is found in this flow, it is assumed to be parcel symbol 21 (missing dot).

[0227] Next, in substep 310.10,

number

number

number

[0228] Subsequently, a lower step 310.11 for error recognition can be performed, and if an error is recognized in lower step 310.12, a new starting position 30 is used and the action is repeated starting from lower step 310.2.

[0229] As part of the flow described above, there is a challenge in identifying dots in the image field of camera 1, starting from an estimated starting position, and measuring the position and area of ​​those dots with sub-pixel accuracy. To achieve a short measurement time, this flow must be performed very quickly because a total of 225 dots must be identified within the reading field 22.

[0230] For this purpose, the function Center_Pos, described below, is used and is optimized for the minimum number of pixel accesses. Figure 25 shows a flowchart.

[0231] When the function Center_Pos is called, the estimated center position of the dot is determined.

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0232] starting point

number

number

number

[0233] Partial step 320 - Detect dots in the reading field The flowchart in Figure 28 shows the flow for detecting all dots within the reading field 22. A pre-specified start base 28, for example, consisting of 3x3 dots, is used as input information. Furthermore, a step 330 for precisely measuring the center position and area of ​​the dots is incorporated into a sub-step 320.

[0234] In the first substep 320.1, the center position 32 of the reading field 22 is defined. This center position 32 is selected so that the reading field 22 has the largest possible coverage with respect to the image area on which the code is displayed, so that as many valid dots as possible can be detected. Typically, this is the case when the center of the reading field 22 is located at the center of the code area. When the code area occupies the entire surface of the image, as in this example, the center point 32 of the reading field is placed as close as possible to the center of the image (see Figure 29a).

[0235] At the start of this method, only the 3x3 dots of the starting base 28 are known. All other dots in the reading field 22 are still unknown; that is, the exact location and area of ​​all other dots have not yet been measured. The purpose of this method is to gradually measure and classify the unknown dots.

[0236] Sub-step 320.2: For each known dot, vector

number

number

number

number

[0237] At the start of the search, only the starting base 3x3 dots are valid. Individual unknown dots with two or more valid adjacents are highlighted in Figure 29b). Figure 29d) schematically shows an unknown dot with two valid adjacents 20a and three valid adjacents as 20b. All other unknown dots (20c) still have no valid adjacents.

[0238] The algorithm is stack-based, and in lower step 320.3, all unknown dots with at least two valid neighbors are entered into the stack.

[0239] The stack is processed in a loop: • Lower step 320.3: The top entry is taken from the stack as an unknown dot and processed in the following steps. • Sub-step 320.4: The location of the unknown dot is estimated by averaging the extrapolated locations of all valid neighbors. If the unknown dot is located outside the expected read field 22, it is not processed further. If the unknown dot is located inside the expected read field 22, it is identified and measured with respect to its estimated location using the function Center_Pos. • Lower step 320.5: If an unknown dot cannot be identified, that unknown dot is classified as a defect (lower step 320.6), and the next unknown dot is removed from the stack. Step 330 / Substep 320.7: If a dot is identified, its position and area are measured using the function Center_Pos and entered into the dot matrix 29 as a valid new dot. For this dot, a vector

number

number

[0240] Partial step 340 - Parallelization process Image formation using a wide-angle objective lens can result in barrel distortion of the image field. Lines of dots that lie in a straight line within the code plane of code array 2 appear, depending on the context, on curved lines within camera image 3. This curvature can be removed by a mathematical method, namely, parallelization. For computational time reasons, parallelization is applied only to the center position 32 of the dots, and not to all pixels of the input image. Figure 31 illustrates an unparalleled grid (33) and a parallelized grid (34).

[0241] Center of distortion

number

number

number

number

number

number

number

[0242] Part 350 - Classify the dots The small and large dots symbolically represent the binary values ​​"0" and "1". Dot classification is performed based on a threshold for the normalized area of ​​the dot. This flow is shown in the flowchart in Figure 32.

[0243] Perspective distortion has a strong impact on the area of dots within the image field. For example, dots located further away are imaged smaller, and circular dots on an inclined plane are imaged as ellipses.

[0244] Lower step 350.1: To compensate for these effects, the area of the dots is normalized based on the area of the dot cells of these dots. A dot cell is a parallelogram spanned by the vector

Number

Number

Number

Number

Number

Number

[0245] Lower step 350.2: From the normalized area Dot.Area of all dots scaled a threshold is determined using a statistical method.

[0246] Sub-step 350.3: Next, all dots are classified: Missing dots: Type 0 • If the normalized area is less than the threshold: Type 1; otherwise: Type 2.

[0247] Substep 350.3: If an error occurs, the status is set to invalid; if no error occurs, the dot is valid.

[0248] Regarding step 360 - read the code After the dot classification, the dot matrix 29 contains all the information necessary to identify the 6D position.

[0249] The dot matrix 29 contains the following information: [Table 2]

[0250] A flowchart illustrating this is shown in Figure 33. The coarse digital positions in the X and Y directions are determined by reading the X-bit chain and Y-bit chain within the dot matrix.

[0251] Sub-step 360.1: Parcel symbol 21 “empty dot” (type 0) is used as a reference point to identify the orientation of the bit chain in the dot matrix 29. If there are multiple entries with type 0, these entries can be validated against each other because “empty dots” are regularly repeated within the parcel grid (in this case, 7x7 dots).

[0252] Sub-step 360.2: Starting from the parcel symbol 21, the orientation of the reading track relative to the code on the two axes of the code plane is determined. At this point, it is unknown which sequence of the sequence corresponds to the X axis, which sequence of the sequence corresponds to the Y axis, and in which direction (forward or backward) the code is read. The code is read along the reading track and stored as Code_0 and Code_1. Figure 34a) illustrates the dot types entered in the dot matrix. The parcel symbol (type 0) is highlighted in gray. Figure 34b) shows the reading tracks for the two axes additionally filled in. The reading tracks for the two axes are rotated 90° relative to each other, with the parcel symbol 21 forming the center of rotation.

[0253] The reading field (15x15 dots) is significantly larger than the parcel (7x7), allowing for redundant reading of the code. Specifically, a copy of Code_0 is located in an adjacent parcel (35a) in the j-direction, and a copy of Code_1 is located in an adjacent parcel (36a) in the i-direction.

[0254] Redundancy also exists in the length of the readable code. For localization, a code length of 24 bits in each direction is sufficient, i.e., the contents of a 7x7 parcel are sufficient. However, a larger readable field with 15x15 dots provides 51 or 54 bits in each direction. The redundant information is used to recognize and correct individual misread bits in Code_0 and Code_1.

[0255] In this example, the following code will be read:

number

[0256] Lower step 360.3: From Code_0, any interval with t=24 consecutive bits is selected and checked for completeness.

[0257] Sub-step 360.4: Subsequently, the sum of the digits of Code_0 is calculated. If this sum of the digits is less than t / 2 = 12, then Code_0 is the X code and Code_1 is the Y code. If this sum of the digits is greater than t / 2, then Code_0 is the Y code and Code_1 is the X code. The Y code is inverted. The X code remains unchanged.

[0258] Sub-step 360.5: Subsequently, Code_0 and Code_1 are searched for within the code table using an error-tolerant string search. The bit position with the best match is output as the search result; if the deviation is too large, the search result is evaluated as invalid. To convert Code_0 and Code_1 into integer location coordinates (X,Y), the bit positions of the codes are converted to spatial positions according to the code structure, as schematically shown in Figure 35:

[0259] An arbitrary interval with t = 24 consecutive bits is selected and checked for completeness:

number

[0260] This segment is searched in the forward direction within the code table and found at bit position 37 according to the underlined portion:

number

[0261] Each X dot is associated with one bit position in the code table. • The bit position corresponds to the starting point in the reading sequence within a single parcel. Each Y-dot is associated with one bit position in the code table.

[0262] 360.6 Efficacy Test According to Figure 35, this corresponds to dot position Xc=10=Pos0. Similarly, the Yc value for Pos1 is determined. For both codes, the reading direction (Dir_0 and Dir_1) is also determined by searching for these codes in both directions within the code table. That is, 0 means it was found in the forward direction, and 1 means it was found in the backward direction. From the reading directions of Code_0 and Code_1, the rough orientation of the reading window 22 in 90° increments is determined according to the table below. [Table 3]

[0263] As a result of partial step 360, the rough position of an integer (r X,int ,r Y,int ) and the orientation of the reading window 22 are provided.

[0264] Step 400 - Identify beam data To determine the precise location, the parallelized position data of the valid dots in the dot matrix 29 is evaluated. Figure 36 illustrates the dots of the read field in the camera image after parallelization.

[0265] Step 410: Fit the compensation line. For this purpose, a compensation line (beam) is fitted to the rows and columns of valid dots in the reading field 22. The compensation line is calculated from the position data of valid dots in each row or column using standard mathematical methods to minimize errors. Invalid dots are excluded from this method. The compensation lines i for each bunch k=0···1 are given by the following parameters: ○SX ki X I Intersection with the axis ○SY ki Y I Intersection with the axis ○α ki Angle of line i in the image field This is described by (Figure 37).

[0266] The compensating lines fitted to the dotted row form the first bundle of lines (bunch0)36, and these lines are Y C It is parallel to the axis. The lines of the dot row form a second bundle of lines (bunch1)37, and these lines are parallel to the X C It is parallel to the axis. As a result, the position data of 15 × 15 = 225 dots is reduced to 15 + 15 = 30 data points of the compensation line.

[0267] Interpolation further enhances the robustness of the compensation line against individual variations in the dot positions by performing averaging across the dot positions on each line. This improves the stability and accuracy of the 6D position values.

[0268] Figure 36 shows the compensation lines for a reading field with 15 × 15 dots, namely the line from bunch 0 as reference code 36 and the line from bunch 1 as reference code 37.

[0269] Step 420 - Identify bunch data In this step, the three linear parameters SX of each bundle of compensation lines 36 and 37 are ki SY ki ,α ki However, interpolation is performed using a quadratic polynomial. The interpolation method considers only valid lines. Invalid lines that have only a few valid dots available for interpolation are excluded from bunch interpolation. For this purpose, weighted interpolation based on minimizing the squared error is performed using standard mathematical methods. Valid lines are evaluated with weights 1,0, while invalid lines are evaluated with weights 0,0 and are therefore hidden. Interpolation function approach: ·SX k (i) = psxk0 +i·psx k1 +i 2 PSX k2 ·SY k (i) = psy k0 +i·psy k1 +i 2 ·psy k2 ·α k (i) = pα k0 +i·pα k1 +i 2 ·pα k2

[0270] In this case, i is the number of the next line in each line bundle (line index). CE approach for interpolating linear parameters: P k (i) = p k0 +i·p k1 +i 2 ·p k2 However, P k (i) = SX k (i), SY k (i) or α k (i)

[0271] To describe a straight line mathematically in a unique way, one of the two axis intersections SX ki or SY ki Only the straight line angle α ki It is sufficient to show it together with the axis. Ideally, the intersection with the axis as perpendicular as possible to the line should be shown. That is, for a line with a small slope, the intersection with the Y-axis is preferably shown, and for a steep line, the intersection with the X-axis is shown. Therefore, the following method is used: In bundle 1, a reference line located near the center 32 of the reading field 22 is selected. • X when the angle of the reference line is within the range of [(-1 / 4)π, (1 / 4)π] or [(3 / 4)π, (5 / 4)π] (Figure 38a) I The sector on the axis is a straight line with a small slope. In that case, for all straight lines in bundle 0(36), the intersection point SX with the X-axis is0i The specified value is given to all lines in bundle 1 (37) and the intersection point with the Y-axis SY 1i This is specified (Figure 38b). As a marker, rot_status is set to 0. • Y when the angle of the reference line is outside the range described above (Figure 38a) I The sector on the axis is a steep straight line. In that case, for all straight lines in bundle 0(36), the intersection point with the Y-axis SY 0i The specified value is given to all lines in bundle 1(37) and the intersection point SX with the X-axis. 1i This is specified (Figure 38c). As a marker, rot_status is set to 1.

[0272] This step further reduces the amount of data. That is, the bundle of lines is described by only six parameters, namely α ki Three parameters related to SX, and one of the axis intersections, i.e., SX ki or SY ki It is described by three parameters related to . In the case of two bundles, there are 12 parameters and the marker rot_status.

[0273] Therefore, the two bundles 36 and 37 are always described by 12 parameters and the boolean variable rot_status, regardless of the number of lines or the size of the measurement field. From these parameters, the 6D position is calculated in the next step.

[0274] Interpolation further enhances the robustness of the interpolated values ​​against individual variations in each line or dot by performing averaging across the lines of each bundle and, consequently, across all the dots. This improves the stability and accuracy of the 6D position values.

[0275] Step 600 - Calculate the 6D camera position. From the 12 interpolation parameters of the line bundle (Table 9.1), the six coordinates of the camera position are calculated in the following order: • Partial step 500: Position rX and r Y • Partial step 710: Calculate the characteristic values ​​of the distorted grid. • Partial step 600: Camera angle φ Ζ • Partial step 700: Position r z • Partial step 700: Camera angle φ X and φ Y • Partial step 700: φ X , φ Y and r z Iterative algorithm for solving a system of equations related to

[0276] This calculation involves deriving characteristic values ​​that describe the measured grid at the location of optical axis 5. These characteristic values ​​are linked to the 6D camera position via a system of equations derived from the imaging model and the laws of optical optics.

[0277] The camera position is determined by solving a system of equations.

[0278] Additionally, the effectiveness of the positional values ​​can be estimated by evaluating numerous diagnostic results from individual steps in the program flow.

[0279] Regarding partial step 500 - camera position r X and r Y Calculate The camera position in X and Y is an integer coarse position (r X,int ,r Y,int It is calculated by multiplying the sum of ) and a real fraction [0···1] (fine position) by the dot grid distance: r X =(r X,int +r X,fract )·c grid Unit: [m] r Y =(r Y,int +r Y,fract )·c grid Unit: [m] however, (r X,int +r Y,int ): The integer position of the reading field in units [dots], as identified in step 300. (r X,fract +r Y,fract ): A fraction of the real number of the read field position in units [dots] c grid : Dot grid distance in units [m / dot] That is the case.

[0280] The reference point for measuring the camera position is the intersection point 4 of the optical axis 5 and the code array 2, and within the camera image 3, it is the intersection point of the optical axis 5 and the imaging device 12 (point (4) in Figure 2 or the image of the intersection point 4 of the image center 4 in Figure 39). This image center IC(4) is identified by the orientation of the objective lens 9 relative to the camera chip, and the image center IC(4) does not have to be the same as the center of the camera chip. The image center IC(4) is identified in the calibration process and is represented by a certain two-dimensional vector (IC X ,I C Y It is stored in the program as a fraction r. X,fract and r Y,fract This is calculated as the intersection point of the interpolated line and the image center 4. For two ray beams k=0 and k=1 (36,37), the interpolation equation for the axis intersection is: S k (i) = ps k0 +i·ps k1 +i 2 · ps k2 However, S depends on rot_status. k (i) = SX k (i) or SY k (i) This holds true.

[0281] Figure 39 shows the central line for two bundles of lines k=0 and k=1. The following holds true. ·If rot_status=0:

number

number

number

number

number

number

number

[0282] Regarding sub-step 710 - Calculate the characteristic values ​​of the distorted grid. From the interpolation equation, further characteristic values ​​that characterize the distorted grid can be obtained. To obtain characteristic values ​​at the center of the image, the fine position r X,fract and r Y,fract These are substituted into the interpolation equation. These characteristic values ​​are given by dimension r Z ,φ X ,φ Y , and φ Ζ It is necessary to determine the location.

[0283] a) Grid dimensions of the axis intersections of the lines for both line bundles: g C0 ,g C1 . The grid dimensions are determined according to the dimensionless linear index i at the center of the image, and the axis intersection S k This is determined by deriving the interpolation function for (i): bunch k In contrast:

number

number

number

number

[0284] Regarding partial step 600 - camera angle φ Z Calculate Straight angle φ Ζ This is identified from the imaging equation derived in 2.4.2:

number

number

[0285] The gradient of the straight lines in the bundle of straight lines, bunch1, in the camera image is:

number

[0286] The gradient at the center when y0=0 is considered, and m1| y0=0 = -tan(φ Z ) can be obtained. m1| y0=0 This is the gradient of the line of bunch1 at the center of the image. This gradient is derived from the characteristic value α from 6.2b). C1 This corresponds to, that is, m1| y0=0 =tan(α C1 )

[0287] This allows the camera angle φ Ζ An equation relating to φ is obtained: Ζ =α C1

[0288] Regarding partial step 700 - camera position r Z Calculate Camera position r Z This is a bundle of two straight lines i The axis intersection g Ci It is determined from the grid distance. Since there are two line bundles in each image, there are basically two r Z The position can be determined. When rot_status=k, there are four values ​​r Zik However, (i,k=0···1) is obtained. The imaging equation for Figure 9 holds:

number

[0289] For example, the value r Z00 This is the case where rot_status=0 for B X Measured grid distance g at the axis intersection on the axis C0 It is identified from this.

[0290] B X On the axis, B Y =0 holds, and from the image equation, R 21 x0+R 22 y0=0⇒y0=-(R 21 / R 22 )x0 is obtained.

[0291] This section B X By substituting into the equation relating to,

number

[0292] Grid distance g C0 teeth,

number

number

number

[0293] From here,

number

[0294] When x0=0,

number

number

number

[0295] Similarly, position r when rot_status=1 Z01 The formula is calculated as follows:

number

[0296] position r Z1k is, g C1 Derived from, r Z10 Formulas related to:

number

number

[0297] The results are summarized in Table 4. [Table 4] Table: Camera position value r Zik Calculation

[0298] By introducing the angle β = rot_status·(π / 2), these equations can be expressed as follows:

number

[0299] One value r per image Z To output only this, the average of the two z values ​​is calculated: r Z =(r Z0k +r Z1k ) / 2

[0300] Regarding partial step 700 - camera angle φ X ,φ Y Calculate Angle φ X ,φ Y This is a bundle of two lines at the center of the image. k The angle divergence ∂α k (i) / ∂ i This is derived from the following. In step 600, the linear gradient m1 with respect to the bundle of lines bunch1 is derived as a function of the linear index y0:

number

number

number

number

number

[0301] Similarly, angle φ Χ This is obtained from the gradient m0 ​​of the straight line at bunch0. Similar to the calculation at 600,

number

[0302] · Angle α0(x0) = tan(m0(x0))

number

number

[0303] • At the center of the image x0=0, the angle diverges.

number

[0304] • Measured angular divergence dα C0 If we identify it with φ Χ Equations relating to:

number

[0305] Regarding partial step 700 - camera position Z and camera angle φ X ,φ Y Iteratively calculate function φ X , φ Y There is an interdependence between and Z: φ Χ =f(φ y ,r Z );φ Y =f(φ X ,r Z );r Z =f(φ X ,φ y )

[0306] Therefore, these equations are solved in an iterative sequence. Each iterative step solves the camera position Z and the camera angle φ. Χ and φ Y This improves accuracy. This algorithm is shown in the flowchart in Figure 40.

[0307] Lower step 700.1: Real fraction of the field position read in units [dots] (r X,fract ,r Y,fract ) calculate Lower step 700.2: position r x and r y Calculate Lower step 700.3: Camera rotation / camera angle φ Ζ Calculate Lower step 700.4: Initialize camera angles phix=0;phiy=0 Sub-step 700.5: Initialize the counter for iteration, for example, by 4. Lower step 700.6:r Z0k and r Z1k Calculate and finally r Z =(r Z0k +r Z1k Calculate ) / 2 Lower step 700.7: Calculate camera angles phix and phiy. Lower step 700.8: Counter = 0? Recalculate or otherwise abort the iteration. Lower step 700.9: Error response

[0308] This algorithm converges quickly, and the results become sufficiently stable after, for example, four iterations. Finally, the results are as follows: • Camera position (r X ,r Y ,r Z ) • Camera angle (φ X ,φ Y ,φ Z ) • Validity information The following will be output. [Explanation of Symbols]

[0309] 1 Camera 2 Code Arrays 3 images 4. Intersection point of the optical axis 5 and the code array 2; reference point; 5 Optical axis Number 6 is missing. 7. Intersection of the optical axis and the imaging device 8. An object with at least one code array 2. 9. Objective lens 10 Focal point of objective lens 9 11 Focal point of objective lens 9 12 Imaging device 13 Optical center of objective lens 9 14 Observation rays 15 The coordinate system (X) of the imaging device 12 in the plane of the imaging device 12 I ,Y I ) 16. Coordinate system of code array 2 in the plane of code array 2 (X C ,Y C ) 17. Coordinate system of camera 1 (X, Y, Z) 18 Virtual image plane 19 parcels 20 Basic Symbols 21 Parcel Symbols 22 Read Fields 23 Center position 24X parcel area 25 Y parcel area 26 Parcel area for additional data 27 Intersection Patterns 28. Starting Field / Starting Base / Launchpad 29 Dot Matrix 30 starting position 31 Search Ray 32 Center position of reading field 22 33 Unparallelized point grid 34 Parallelized point grid 35 Reading track in the first code direction 36 Reading track in the second code direction 37 First line bundle / bunch0 38 Second bundle of straight lines / bunch1

Claims

1. A method for identifying at least one degree of freedom of camera (1) relative to a code array (2) from a camera image of camera (1), The code array (2) has a point grid comprising a plurality of basic symbols (20), and the coarse positions of the basic symbols (20) are encoded in the code array (2). A start field (28) having basic symbols (20) is identified, the start field (28) having at least three basic symbols (20), two independent principal directions along the point grid in the camera image are estimated via the basic symbols (20) of the start field (28), and the basic symbols (20) of the start field form a valid basic symbol. In the search step, starting from at least one valid basic symbol (20), further basic symbols (20) are searched along at least one of the principal directions, and if the search is successful, the further basic symbols (20) are characterized as valid basic symbols (20). The aforementioned search step is performed multiple times. A method in which, based on the valid basic symbols (20) or a subset thereof, the coarse position of at least one of the basic symbols (20) in the code array is identified, and / or the coarse position of the coordinate system of the point grid is identified as two degrees of freedom of the camera relative to the code array (2).

2. The basic symbol (20) of the start field (28) defines two connection vectors in the two independent principal directions along the point grid in the camera image. The method according to claim 1.

3. With respect to the basic symbol (20), the center position of each basic symbol (20) in the camera image is identified. The search step is performed starting from the center position of each of the basic symbols (20). The method according to claim 1 or 2.

4. The area of ​​the basic symbol (20) in the camera image is identified. The area is used to form the data in the encoding of the code sequence (2). The method according to any one of claims 1 to 3.

5. In the preliminary step for creating the starting field (28), at least one starting position (30) is defined, and adjacent basic symbols (20) along the search ray (31) starting from the starting position (30) are searched as starting basic symbols. The method according to any one of claims 1 to 4.

6. Starting from the position of the aforementioned starting basic symbol, adjacent basic symbols (20) along the search ray (31) are searched as first auxiliary starting basic symbols. The method according to claim 5.

7. Starting from the aforementioned starting basic symbol and the first auxiliary starting basic symbol, a second auxiliary starting basic symbol is searched for in a direction perpendicular to the connection between the aforementioned starting basic symbol and the first auxiliary starting basic symbol. The method according to claim 6.

8. Starting from the aforementioned starting basic symbol, all basic symbols (20) directly adjacent to the starting basic symbol are searched by a linear combination of the connections, based on the connection between the starting basic symbol and the first auxiliary starting basic symbol, and based on the connection between the starting basic symbol and the second auxiliary starting basic symbol. The method according to claim 7.

9. Starting from the start field (28) and / or from the valid basic symbol (20), adjacent basic symbols (20) having at least two valid adjacents in the principal direction or diagonal direction are searched. The method according to any one of claims 1 to 8.

10. A reading field (22) is defined within the camera image and / or within a planar point grid, and a basic symbol matrix corresponding to the reading field (22) is formed. Each point in the reading field (22) is associated with one entry in the basic symbol matrix, The basic symbol matrix contains entries for the positions, in particular, the normalized area or data of the basic symbols (20). Based on the aforementioned basic symbol matrix, the coarse position of the coordinate system of the point grid, and / or the coarse position of at least one basic symbol in the code array, are identified as two degrees of freedom. The method according to any one of claims 1 to 9.

11. Based on the aforementioned basic symbol matrix, A first linear function is derived with respect to a first line using a first function argument in the coordinate system of the camera image, the first line is oriented parallel to the first principal direction in the coordinate system of the code array (2), and / or, by changing the first function argument, the first line is shifted parallel to the second principal direction in the coordinate system of the code array (2), and / or A second linear function is derived with respect to the second line using a second function argument in the coordinate system of the camera image, the second line is oriented parallel to the second principal direction in the coordinate system of the code array (2), and by changing the second function argument, the second line is shifted parallel to the first principal direction in the coordinate system of the code array (2). Based on at least one of the aforementioned linear functions, at least one further degree of freedom of the camera is determined. The method according to any one of claims 1 to 10.

12. A reference point is placed within the aforementioned camera image. The first axis intersection function of the first straight line in the coordinate system of the camera image is used, and the first straight line is defined using the first function argument in the coordinate system (X) of the code array. Ｃ , Y Ｃ In the above, the first line is oriented parallel to the first principal direction (2) of the point grid, and by changing the first function argument, the first line is oriented in the coordinate system (X) of the code array. Ｃ , Y Ｃ ) is shifted parallel to the second principal direction, The first axis intersection function depends on the first function argument to determine the coordinate system (X) of the camera image. Ｉ , Y Ｉ Determine the first axial intersection along the first axis of ), the first axis extending through the reference point, The second axis intersection function of the second straight line in the coordinate system of the camera image is used. Using the second function argument, the second straight line is oriented parallel to the second main direction in the coordinate system (X Ｃ , Y Ｃ ) of the code array. By changing the second function argument, the second straight line is shifted parallel to the first main direction in the coordinate system (X Ｃ , Y Ｃ ) of the code array. The second axis intersection function depends on the second function argument to the coordinate system (X) of the camera image. Ｉ , Y Ｉ Determine the second axial intersection along the second axis of ), the second axis extending through the reference point, The first and second function arguments are determined based on the axis intersection function such that the reference point forms the first and second axis intersections. Based on the first function argument and the second function argument and the coarse position, the coordinate system (X) of the code array (2) in the plane of the code array (2) Ｃ , Y Ｃ The precise position of the reference point in ) is identified as at least one further degree of freedom. The method according to any one of claims 1 to 11.

13. The first function argument and / or the second function argument are determined such that the first line or the second line intersects the reference point. The rotation angle (phiz) of the camera around the optical axis is derived as a degree of freedom of the camera (1) relative to the code sequence (2), based on the first straight line and / or the second straight line. The method according to claim 10 or 11.

14. A reference point or the reference point is located within the camera image. The aforementioned reference point is configured as the intersection of the optical axis of the camera and the imaging device and / or the camera image. From the aforementioned camera image, the following values ​​are obtained at the aforementioned reference point, namely, - One rotation angle (phz) of the camera, or the rotation angle (phz), with respect to the optical axis of the camera. - The first local grid dimension (gc0) of the point grid in the camera image in the first principal direction, and / or the second local grid dimension (gc1) of the point grid in the camera image in the second principal direction, - The first local angular divergence (dalphac0) of the point grid in the camera image in the first principal direction, which describes the angular difference between two adjacent lines in the first principal direction within the point grid. - The second local angular divergence (dalphac1) of the point grid in the camera image in the second principal direction, which describes the angular difference between two adjacent lines in the second principal direction within the point grid. However, it was identified as a characteristic value, Based on the aforementioned characteristic values, the following three further degrees of freedom of the camera relative to the code sequence are determined, namely, - The distance (rz) between the code array and the camera, - Two independent pitch angles (phix; phiy) of the optical axis relative to the code array. It is identified. The method according to any one of claims 1 to 13.

15. An electronic control unit, or an automated device equipped with an electronic control unit, The control unit is configured in a programmatic and / or circuitmatic manner to carry out the method described in any one of claims 1 to 14. An electronic control unit, or an automated device equipped with an electronic control unit.

16. It is a computer program, The computer program is configured to carry out the method described in any one of claims 1 to 13 when the computer program is executed in a computer or in the control unit or automation device described in claim 14. Computer program.

17. A machine-readable storage medium storing the computer program described in claim 15.

Images