Providing deformation data for deformation analysis

Abstract

The invention relates to a method for providing deformation data (135) for deformation analysis. The method comprises the step of reading in image data (105) provided by means of a camera (102) from the camera (102). The image data (105) represents a camera image of at least one predefined even and / or odd point-symmetrical region (110). The at least one predefined point-symmetrical region (110) is generated on, at or in a deformable carrier medium. The method further comprises the step of determining at least one symmetry center (112) of the at least one even and / or odd point-symmetrical region (110) using the image data (105) and a determination rule (128), and the step of comparing a position of the symmetry center (112) in the camera image with a predefined position of a reference symmetry center in a reference image (115) to determine a positional deviation (131) between the symmetry center (112) and the reference symmetry center. The positional deviation (131) is used to provide the deformation data (135).

Description

Technical Field

[0001] This invention is based on the apparatus or method of the type claimed in the independent claims. Computer programs are also the subject of this invention. Background Technology

[0002] In fields such as deformation and vibration analysis for quality control or process control of components, products, etc., and in film recording using so-called motion capture, measurement techniques can be quite labor-intensive, where speed, efficiency, accuracy, and reliability can sometimes be disproportionate to the workload.

[0003] DE 10 2020 202 160 A1, which will be published later, discloses a method for determining symmetry properties in image data and a method for controlling functions. Summary of the Invention

[0004] Against this backdrop, the proposed solution here provides a method according to the main claim, an apparatus for using the method, and a corresponding computer program. Advantageous extensions and improvements can be made to the apparatus described in the independent claim through the measures listed in the dependent claims.

[0005] According to the implementation method, in particular, the fact that points or objects in the world are marked or will be marked by means of point-symmetric regions can be utilized, enabling systems with imaging sensors and suitable methods proposed herein to detect and locate these point-symmetric regions with high precision in order to robustly and locally perform specific technical functions, optionally without being perceived as interference by humans or organisms.

[0006] For example, a symmetrical region might not be fully imaged into the camera image, for instance, because it might be partially occluded by an object, partially protrude from the image, or be cropped. Advantageously, the accuracy of the point's center of symmetry can still be maintained because partial occlusion does not distort its position: the remaining point symmetry pairs can still vote for the correct center of symmetry. Partial occlusion only reduces the strength of clustered points in the voting matrix, but the position of the center of symmetry can be preserved and still determined accurately and easily. This is a special advantage of utilizing point symmetry.

[0007] Further advantages in finding regions or patterns based on point symmetry can stem particularly from the fact that point symmetry is invariant with respect to rotation between the point-symmetric region and the camera or image recording, and is largely invariant with respect to the viewing angle. For example, the point-symmetric plane can be invariant with respect to affine imaging. The imaging of an arbitrarily oriented plane by a real camera can always be approximated well, at least locally, by affine imaging. For example, if a circular point-symmetric region is viewed from an oblique angle, the circle becomes an ellipse, in which the point symmetry characteristics and the center of point symmetry are preserved. Therefore, at least one point-symmetric region does not necessarily have to be viewed from a frontal view—even a very oblique viewpoint will not pose a difficulty, and achievable accuracy can be maintained. This invariance, especially with respect to rotation and with respect to the viewing angle, eliminates the need for precautions such as properly aligning the camera with the symmetric region or vice versa. Rather, it may be sufficient that the corresponding point-symmetric region is at least partially captured in the camera image, making it detectable. The relative positional relationship or arrangement between the point-symmetric region and the camera can be insignificant or almost insignificant in this case.

[0008] A method for providing deformation data for deformation analysis is proposed, wherein the method comprises the following steps:

[0009] Image data provided by the camera is read from the interface to the camera, wherein the image data represents a camera image of at least one predefined even and / or odd point symmetric region in the environment of the camera, wherein the at least one predefined point symmetric region is generated on, at, or in a deformable carrier medium.

[0010] The image data and determination rules are used to determine at least one center of symmetry for at least one even- and / or odd-numbered point symmetric region;

[0011] The position of the at least one center of symmetry in the camera image is compared with the predefined position of at least one reference center of symmetry in the reference image relative to the reference coordinate system to determine the positional deviation between the center of symmetry and the reference center of symmetry; and / or

[0012] The positional deviation is used to obtain displacement information of at least one subset of pixels in the camera image relative to corresponding pixels in the reference image, wherein the positional deviation and / or the displacement information are used to provide the deformation data.

[0013] This method can be implemented, for example, in software or hardware, or a hybrid of software and hardware, such as in a control device or apparatus. In the read-in step, image data from multiple cameras may also be read in, where these image data represent multiple camera images of the at least one region. Optionally, multiple reference images may be used in the comparison step. At least one predefined point-symmetric region can be manufactured by performing a variation of the manufacturing method described below. Here, the at least one predefined point-symmetric region may also be fixed at a deformable carrier medium, but not deformed together with it when the carrier medium itself deforms. The determination rules may be similar to or correspond to the process disclosed in DE 10 2020 202 160, which was later disclosed by the applicant. The reference image may represent at least one predefined point-symmetric region. The reference image may also be replaced by reference data that at least partially corresponds to or is equivalent to information that can be obtained from the reference image. Using reference data may be advantageous, particularly in the sense of reducing workload when the information that can be extracted from the reference image has already been given in a more usable form in the type of reference data. The reference data may represent the reference image in a compressed form or representation, for example, as a descriptor image, a signature image, and / or a list of all coordinates and types with existing centers of symmetry. Optional steps can be performed using optical flow, particularly dense optical flow. Deformation analysis can be performed within the scope of component testing, component monitoring, material testing, quality control, motion capture methods, etc. A carrier medium can be placed on an object. The object can deform. The object can be a component, product, etc. In deformation analysis, the deformation of the carrier medium can be analyzed, as well as the deformation of the object, additionally or alternatively. In addition to elastic and plastic deformation, deformation can also be understood as vibration. Displacement information can represent displacement vectors or absolute coordinates.

[0014] According to one embodiment, the determination rule used in the determination step can be configured such that signatures are generated for multiple pixels of at least one segment of the camera image to obtain multiple signatures. In this case, descriptors with multiple different filters can be used to generate each signature. Each filter can have at least one symmetry type. Each signature can have a symbol for each filter of the descriptor. The determination rule can also be configured such that at least one mirror signature for at least one symmetry type of the filter is obtained for the signature. The determination rule can also be configured such that at least one additional pixel with a signature corresponding to at least one mirror signature is present in a search area in the environment surrounding the pixel with the signature, to obtain the pixel coordinates of at least one symmetry signature pair from the pixel and the additional pixel when the at least one additional pixel is present. In addition, the determination rule can be configured such that the pixel coordinates of the at least one symmetry signature pair are evaluated to identify the at least one symmetry center. The descriptor can describe the image content in the local environment surrounding the pixel or reference pixel in a compact form. The signature can, for example, represent the value of the descriptor describing the pixel in binary. Therefore, multiple calculated signature images can be used to determine the at least one mirrored signature; for example, one signature image with a normal filter, one signature image with an even-numbered point mirror filter, and one signature image with an odd-numbered point mirror filter. Additionally or alternatively, at least one reflector can be applied to the symbols of one of the signatures to obtain at least one mirrored signature. In this case, each reflector can have rules for filtering based on the symmetry type and descriptor to modify the symbols. Here, the search area can depend on at least one of the applied reflectors. Such an implementation provides the advantage of enabling efficient and accurate detection of symmetry features in image data. In this case, symmetry detection in an image can be achieved with minimal effort.

[0015] Here, in the determination step, for each determined center of symmetry, using the pixel coordinates of each symmetry signature pair that helps to correctly identify the center of symmetry, a transformation rule is determined for transforming the pixel coordinates of the center of symmetry and / or at least one predefined even- and / or odd-numbered point symmetry region. This transformation rule can be applied to the pixel coordinates of the center of symmetry and / or at least one predefined even- and / or odd-numbered point symmetry region to correct the distorted viewpoint of the camera image. The advantage of this implementation is that it allows for reliable and accurate reconstruction of the correct mesh or correct topology for multiple point symmetry regions.

[0016] The symmetry type of the at least one symmetry center can also be determined in the determination step. The symmetry type can represent even-numbered point symmetry and additionally or alternatively, odd-numbered point symmetry. Additionally or alternatively, in this case, in the comparison step, the symmetry type of the at least one symmetry center in the camera image can be compared with a predefined symmetry type of the at least one reference symmetry center in a reference image to check for consistency between the at least one symmetry center and the at least one reference symmetry center. Odd-numbered point symmetry can be generated by mirroring points with inverted grayscale or color values. By using and identifying two different point symmetries, the information content of the point symmetry region and pattern can be increased.

[0017] In this case, the image data read in the read-in step can represent a camera image of at least one pattern consisting of multiple predefined even- and / or odd-numbered symmetrical regions. Here, in the determination step, the geometric arrangement of the symmetry centers of the at least one pattern can be determined, a geometric sequence of the symmetry types of the symmetry centers can be determined, and additionally or alternatively, the pattern can be determined from multiple predefined patterns using the sequence. The arrangement and / or the sequence can represent an identification code for the pattern. This implementation offers the advantage of increased reliability in identifying symmetry centers and the ability to obtain further information by recognizing specific patterns. Reliable identification of symmetry centers can also be achieved for different distances between the camera and the pattern.

[0018] In this scenario, during the determination step, the arrangement of the symmetry centers of the at least one pattern, along with, additionally or alternatively, a sequence of symmetry types of the symmetry centers, is used to determine implicit additional information of the at least one pattern, or readout rules for reading explicit additional information from the camera image. The arrangement and, additionally or alternatively, the sequence can represent the additional information in an coded form. The additional information can be related to deformation analysis. This implementation provides the advantage of conveying additional information through the topology of at least one pattern.

[0019] Here, in the comparison step, the reference image can also be selected from multiple stored reference images based on the determined arrangement, the determined sequence, and the additionally or alternatively determined pattern, or the reference image can be generated using stored generation rules. In this way, the correct reference image can be reliably identified. Optionally, when there is a link between the identified pattern and the generation rules, the memory requirements for the reference image can also be minimized, since only the generation rules need to be stored.

[0020] Furthermore, the determination steps and additional or alternative comparison steps can be performed jointly for all centers of symmetry, independent of their symmetry type, or the determination steps and additional or alternative comparison steps can be performed individually for centers of the same symmetry type, depending on their symmetry type. Therefore, joint execution allows for accurate and reliable identification of centers of symmetry with low memory and time requirements. Optionally, and particularly in particular, individual execution can minimize confusion with randomly occurring patterns in the image.

[0021] A method for controlling deformation analysis is also proposed, which includes the following steps:

[0022] Evaluate the deformation data provided according to the implementation of the above method to generate a control signal dependent on the deformation data; and

[0023] The control signal is output to the interface of the device used to perform the deformation analysis in order to control the deformation analysis.

[0024] This method can be implemented, for example, in software or hardware, or a hybrid of software and hardware, for example, in a control device or apparatus. Here, the implementation methods described above for providing the method can be advantageously combined to perform the control method.

[0025] Furthermore, a method is proposed for manufacturing at least one predefined even- and / or odd-point symmetric region for use in embodiments of the above method, wherein the method comprises the following steps:

[0026] Generate design data, which represents a graphical representation of the at least one predefined symmetrical region of even and / or odd points; and

[0027] Using the design data, at least one predefined even- and / or odd-point symmetrical region is generated on, at, or within the carrier medium to manufacture the at least one predefined even- and / or odd-point symmetrical region.

[0028] This method can be implemented, for example, in software or hardware, or a hybrid of software and hardware, such as in a control device or apparatus. By performing this manufacturing method, at least one predefined even- and / or odd-numbered point symmetrical region can be manufactured, which can be used within the scope of the embodiments of the above-described method.

[0029] According to one embodiment, design data can be generated in the generation step, said design data being a graphical representation of at least one predefined even- and / or odd-numbered point symmetrical region represented by a circle, ellipse, square, rectangle, pentagon, hexagon, polygon, or annulus. In this case, the at least one predefined even- and / or odd-numbered point symmetrical region can have a regular or quasi-random content pattern. Additionally or alternatively, a first half of the at least one predefined even- and / or odd-numbered point symmetrical region can be arbitrarily predefined, and a second half can be constructed by dot mirroring and optionally additionally inverting grayscale values ​​and additionally or alternatively inverting color values. Additionally or alternatively, in the generation step, the at least one predefined even- and / or odd-numbered point symmetrical region can be generated by additive manufacturing processes, separation, coating, molding, initial forming, or optical display. Additionally or alternatively, the carrier medium can be fabric, rubber, film, paint, sheet metal, metal, wood, plywood, plastic, fiber-reinforced plastic, paper, cardboard, composite material, coated material, leather, liquid, glass, stone, ceramic, concrete, plaster, or food. The carrier medium can be elastically deformable. Therefore, at least one predefined even- and / or odd-point symmetric region can be manufactured in a suitable manner according to the specific application or the general boundary conditions therein.

[0030] Design data representing a graphical representation of at least one pattern composed of multiple predefined even- and / or odd-point symmetrical regions can also be generated in the generation step. In this case, at least a subset of the even- and / or odd-point symmetrical regions may be aligned on a regular or irregular grid, directly adjacent to each other, and additionally or alternatively separated from at least one adjacent even- and / or odd-point symmetrical region by gaps. They may be identical or different from each other in terms of their size and / or their content patterns, and additionally or alternatively arranged in a common plane or different planes. Additionally or alternatively, in the generation step, design data representing a graphical representation of at least one pattern with layered symmetry can be generated. In this way, different patterns with specific information content and additionally or alternatively patterns with layered symmetry can be generated for different distances from the pattern.

[0031] Even when corresponding markings are known to exist, it is particularly difficult for humans to perceive the symmetry hidden within a pattern. This, for example, makes it possible to conceal such markings. This may be meaningful or desirable for various reasons, such as aesthetic reasons, because technical markings should not or do not wish to be seen, for example, because attention should not be diminished by markings that are unimportant to humans, or because the markings should be kept confidential. Aesthetic reasons play an important role, especially in the field of design. For example, in the interior space of a vehicle, on the exterior of a vehicle, on an aesthetically pleasing object, or in the field of internal or architectural architecture, conspicuous technical markings are not or are difficult to accept. However, if technical markings are to be concealed, for example, in fabric patterns, or in plastic or ceramic reliefs, or in holograms, or on printed surfaces, as may be possible according to the implementation, the technical markings can be both aesthetically pleasing and useful, for example, providing one or more reference points for a camera, so as to determine, for example, the relative camera pose. Depending on the application, the concealed aspects may also be irrelevant or have little or no relevance. Thus, the robustness of the technology still applies to the use of such designed patterns. In particular, patterns with random or pseudo-random characters can provide a variety of possibilities for finding as clearly defined pairs of symmetrical points as possible. According to the implementation, this can be utilized, for example, particularly to improve the signal-to-noise ratio of the response measured at the center of symmetry, and thus to improve robustness in the sense of error-free detection and precise localization of the center of symmetry. The pattern may, in particular, comprise one or more point-symmetric regions having odd or even point symmetry. These regions may be designed, for example, as circles, hexagons, squares, ellipses, polygons, or other shapes. The point-symmetric regions may be of the same type or of different shapes and sizes. The point-symmetric regions may be connected to each other without gaps or spaced apart.

[0032] The proposed solution also creates a device configured to perform, manipulate, or implement variations of the methods presented herein within a corresponding apparatus. The task on which this invention is based can also be solved quickly and efficiently through this embodiment of the invention in the form of a device.

[0033] To this end, the device may have at least one computing unit for processing signals or data, at least one storage unit for storing signals or data, at least one interface to a sensor or actuator, at least one communication interface for reading sensor signals from the sensor or for outputting data or control signals to the actuator, and / or for reading or outputting data embedded in a communication protocol. The computing unit may be, for example, a signal processor, a microcontroller, etc., and the storage unit may be flash memory, EEPROM, or magnetic storage. The communication interface may be configured to wirelessly and / or wiredly read or output data, wherein a communication interface capable of reading or outputting wired data may, for example, electrically or optically read the data from a corresponding data transmission line or electrically or optically output the data to a corresponding data transmission line.

[0034] In the current context, a device can be understood as an electrical apparatus that processes sensor signals and outputs control signals and / or data signals based on those signals. The device may have an interface that can be configured as hardware and / or software. When configured as hardware, the interface may, for example, be part of a so-called system ASIC that contains various functions of the device. However, the interface may also be a separate integrated circuit or at least partially composed of discrete components. When configured as software, the interface may be a software module that exists, for example, on a microcontroller along with other software modules.

[0035] A computer program product or computer program having program code is also advantageous. This program code can be stored on a machine-readable carrier or storage medium (such as semiconductor memory, hard disk storage, or optical memory) and is used to execute, implement, and / or manipulate the steps of the method according to one of the above embodiments, particularly when the program product or program is running on a computer or device. Here, the method can be implemented as a hardware accelerator on a SoC or ASIC. Attached Figure Description

[0036] Embodiments of the proposed scheme are shown in the accompanying drawings and explained in more detail in the following description.

[0037] Figure 1 An embodiment of the device for provision, an embodiment of the device for control, and a schematic diagram of the camera are shown;

[0038] Figure 2 A schematic diagram of an embodiment of the equipment used for manufacturing is shown;

[0039] Figure 3 A flowchart illustrating an embodiment of the provided method is shown;

[0040] Figure 4A flowchart illustrating an embodiment of the control method is shown.

[0041] Figure 5 A flowchart illustrating an embodiment of the method for manufacturing is shown;

[0042] Figure 6 A schematic diagram of a carrier medium having a pattern composed of predefined point symmetrical regions is shown according to an embodiment;

[0043] Figure 7 A schematic diagram of a carrier medium having a pattern composed of predefined point symmetrical regions is shown according to an embodiment;

[0044] Figure 8 It shows that it has a source Figure 7 A schematic diagram of the carrier medium of the pattern, with the graphic highlighting the pattern or the symmetrical area of ​​the predefined points;

[0045] Figure 9 A schematic diagram of a predefined point-symmetric region according to an embodiment is shown;

[0046] Figure 10 A schematic diagram of a pattern composed of predefined point-symmetric regions is shown according to an embodiment;

[0047] Figure 11 A schematic diagram illustrating the use of a lookup table according to an embodiment is shown;

[0048] Figure 12 A schematic diagram of a voting matrix according to an embodiment is shown;

[0049] Figure 13 A schematic diagram illustrating an exemplary pattern arranged in a cube-like form according to an embodiment, showing the correct identification of the grid;

[0050] Figure 14 Shown from an oblique perspective Figure 6 The first part of the illustration is a schematic diagram of the pattern shown.

[0051] Figure 15 Showing from Figure 14 The first part of the illustration shows a pattern in which a predefined point-symmetric region is highlighted;

[0052] Figure 16 An example is shown. Figure 15 A schematic diagram of the pattern after view correction;

[0053] Figure 17 A schematic diagram of an embodiment with a layered symmetrical pattern is shown;

[0054] Figure 18 A schematic diagram of an embodiment with a layered symmetrical pattern is shown;

[0055] Figure 19 A schematic diagram of an embodiment with a layered symmetrical pattern is shown;

[0056] Figure 20 A schematic diagram of a pattern according to an embodiment is shown;

[0057] Figure 21 A schematic diagram of a patterned carrier medium according to an embodiment is shown;

[0058] Figure 22 It shows that it has a source Figure 21 The carrier medium of the pattern, in which predefined point symmetrical regions are highlighted;

[0059] Figure 23 It shows that it has a source Figure 21 or Figure 22 The carrier medium of the pattern, in which the center of symmetry is highlighted;

[0060] Figure 24 A schematic diagram of a patterned carrier medium according to an embodiment is shown; and

[0061] Figure 25 A schematic diagram of a patterned carrier medium according to an embodiment is shown. Detailed Implementation

[0062] In the following description of advantageous embodiments of the invention, the same or similar reference numerals are used for elements shown in different figures and having similar effects, wherein repeated descriptions of these elements are omitted.

[0063] Figure 1 An embodiment of the device 120 for provision, an embodiment of the device 140 for control, and a schematic diagram of the camera 102 are shown. Figure 1 In the illustrations, the providing device 120 or the providing device 120 and the controlling device 140 or the controlling device 140 are shown separately or arranged externally to the camera 102. The providing device 120 and optionally the control device 140 are connected to the camera 102 in a manner capable of data transmission. According to another embodiment, the providing device 120 and / or the control device 140 may also be part of the camera 102 and / or may be combined with each other.

[0064] Camera 102 is configured to record camera images of its environment. In the environment of camera 102, exemplarily only predefined even-numbered and / or odd-numbered point symmetry regions 110 with a center of symmetry 112 are arranged. Camera 102 is also configured to provide or generate image data 105 representing a camera image, wherein the camera image also shows the predefined even-numbered and / or odd-numbered point symmetry regions 110. At least one predefined point symmetry region 110 is generated on, at, or within a deformable carrier medium. At least one predefined point symmetry region 110 may be connected to the carrier medium such that it deforms together with the carrier medium when the carrier medium deforms, or it moves only together with the carrier medium when the carrier medium deforms, but remains rigid in the process.

[0065] The providing device 120 is configured to provide deformation data 135 for deformation analysis. For this purpose, the providing device 120 includes a reading device 124, a determining device 126, an execution device 130, and optionally, a seeking device 132. The reading device 124 is configured to read image data 105 from the providing device 120 through an input interface 122 to the camera 102. Furthermore, the reading device 124 is also configured to forward the image data 105 representing a camera image to the determining device 126.

[0066] The determining device 126 of the providing device 120 is configured to determine the center of symmetry 112 of at least one point-symmetric region 110 using image data 105 and determining rule 128. Determining rule 128 will be discussed in more detail below. It should be noted that determining rule 128 is similar to or corresponds to the process disclosed in DE 10 2020 202 160, which was later disclosed by the applicant. The determining device 126 is also configured to forward the determined center of symmetry 112 to the execution device 130.

[0067] The execution device 130 is configured to compare the position of at least one center of symmetry 112 in a camera image with the predefined position of at least one reference center of symmetry in a reference image 115 relative to a reference coordinate system to determine a positional deviation 131 between the center of symmetry 112 and the reference center of symmetry. The execution device 130 is also configured to read from or receive the reference image 115 or reference data 115 from a storage device 150. The storage device 150 may be implemented as part of or separate from the providing device 120. Furthermore, according to one embodiment, the execution device 130 is configured to forward the positional deviation 131 to the obtaining device 132.

[0068] According to one embodiment, the obtaining device 132 is configured to subsequently use the positional deviation 131 to obtain displacement information 133 of at least one subset of pixels of the camera image relative to corresponding pixels of the reference image 115.

[0069] The providing device 120 is configured to provide deformation data 135 using position deviation 131 and / or displacement information 133. More precisely, the providing device 120 is configured to provide deformation data 135 to the control device 140 via the output interface 138 of the providing device 120.

[0070] Control device 140 is configured to control deformation analysis. For this purpose, control device 140 includes an evaluation device 144 and an output device 146. Control device 140 is configured to receive or read deformation data 135 from providing device 120 via input interface 142. Evaluation device 144 is configured to evaluate the deformation data 135 provided by providing device 120 to generate a control signal 145 dependent on the deformation data 135. Evaluation device 144 is also configured to forward the control signal 145 to output device 146. Output device 146 is configured to output the control signal 145 to output interface 148 of a device for performing deformation analysis to control the deformation analysis.

[0071] Specifically, determination rule 128 is configured such that signatures are generated for multiple pixels of at least one segment of the camera image to obtain multiple signatures. In this case, each signature is generated using a descriptor with multiple different filters. Each filter has at least one symmetry type. Each signature has a symbol for each filter of the descriptor. Determination rule 128 can also be configured such that at least one reflector is applied to the symbol of one of the signatures to determine at least one mirror signature for at least one symmetry type of the filter for that signature. In this case, each reflector includes a symmetry type-specific and descriptor-dependent filter rule for modifying the symbol. Determination rule is also configured such that it is checked whether a pixel with a signature exists in a search area in the environment surrounding the pixel, depending on the applied at least one reflector, and the at least one additional pixel has a signature corresponding to at least one mirror signature, to obtain the pixel coordinates of at least one symmetry signature pair from the pixel and the additional pixel when the at least one additional pixel exists. In addition, determination rule is configured such that the pixel coordinates of the at least one symmetry signature pair are evaluated to identify the at least one symmetry center.

[0072] According to one embodiment, the determining device 126 is configured to generate, for each determined center of symmetry 112, a transformation rule for transforming the pixel coordinates of the center of symmetry 112 and / or the point symmetry region 110 using pixel coordinates that help correctly identify the center of symmetry 112. The transformation rule is applied to the pixel coordinates of the center of symmetry 112 and / or the point symmetry region 110 to correct for distorted viewpoints in the camera image. Furthermore, it is advantageous to determine the transformation rule based on multiple, particularly adjacent, point symmetry regions 110, as this is more robust, more accurate, and less affected by noise, especially if these point symmetry regions lie in a common plane. The application of the transformation is particularly advantageous when considering the arrangement of multiple centers of symmetry 112.

[0073] According to one embodiment, the determining device 126 is further configured to determine the symmetry type of at least one center of symmetry 112. This symmetry type represents even-numbered-point symmetry and additionally or alternatively, odd-numbered-point symmetry. Additionally or alternatively, in this case, the executing device 130 is configured to compare the symmetry type of at least one center of symmetry 112 in the camera image with a predefined symmetry type of at least one reference center of symmetry in the reference image 115 to check for consistency between at least one center of symmetry 112 and at least one reference center of symmetry.

[0074] Specifically, image data 105 in this case represents a camera image of at least one pattern composed of multiple predefined point symmetry regions 110. Here, determining device 126 is configured to determine the geometric arrangement of the symmetry centers 112 of at least one pattern, determine the geometric sequence of the symmetry types of the symmetry centers 112, and / or use said sequence to determine the correct pattern represented by image data 105 from multiple predefined patterns. This arrangement and / or the sequence may represent an identification code of the pattern. According to one embodiment, determining device 126 in this case is configured to use the arrangement of the symmetry centers 112 of at least one pattern and / or the sequence of the symmetry types of the symmetry centers 112 to determine implicit additional information of the at least one pattern or readout rules for reading explicit additional information in the camera image. This arrangement and / or the sequence represents the additional information in coded form. The additional information is related to deformation analysis. Additionally or alternatively, the actuator 130 is configured in this case to select a reference image 115 from a plurality of stored reference images or to generate a reference image 115 using stored generation rules, based on a determined arrangement, a determined sequence, and / or a determined pattern.

[0075] Figure 2 A schematic diagram of an embodiment of the apparatus 200 for manufacturing is shown. The apparatus 200 for manufacturing is configured to manufacture at least one predefined even-numbered and / or odd-numbered point symmetric region 110 for use in manufacturing. Figure 1 The provision of equipment or similar equipment and / or Figure 1 The control device or similar device is used. For this purpose, the manufacturing apparatus 200 includes a generating device 202 and a producing device 206. The generating device 202 is configured to generate design data 204. The design data 204 represents a graphical representation of at least one predefined even- and / or odd-point symmetric region 110. The producing device 206 is configured to use the design data 204 to produce at least one predefined even- and / or odd-point symmetric region 110 on, at, or within a carrier medium, to manufacture at least one predefined even- and / or odd-point symmetric region 110.

[0076] According to one embodiment, the generating device 202 is configured to generate design data 204, which is a graphical representation of at least one predefined even- and / or odd-numbered point symmetric region 110 represented by a circle, ellipse, square, rectangle, pentagon, hexagon, polygon, or annulus. The at least one predefined even- and / or odd-numbered point symmetric region 110 has a regular or quasi-random content pattern, and / or arbitrarily pre-given a first half-face of the at least one predefined even- and / or odd-numbered point symmetric region 110, and a second half-face is constructed by dot mirroring and / or inverting grayscale and / or color values. Additionally or alternatively, the generating device 206 is configured to generate at least one predefined even- and / or odd-numbered point symmetric region 110 by additive manufacturing processes, separation, coating, forming, initial forming, or optical display. Additionally or alternatively, the carrier medium in this case may be fabric, rubber, film, paint, sheet metal, metal, wood, plywood, plastic, fiber-reinforced plastic, paper, cardboard, composite material, coated material, leather, liquid, glass, stone, ceramic, concrete, plaster, or food.

[0077] According to one embodiment, the generating device 202 is configured to generate design data 204 representing a graphical representation of at least one pattern composed of a plurality of predefined even- and / or odd-numbered point-symmetric regions 110, wherein at least one subset of the point-symmetric regions 110 are aligned on a regular or irregular grid, directly adjacent to each other and / or separated from at least one adjacent point-symmetric region 110 by gaps, are the same as or different from each other in terms of their size and / or their content patterns, and / or are arranged in a common plane or in different planes. Additionally or alternatively, the generating device 202 is configured to generate design data 204 representing a graphical representation of at least one pattern having layered symmetry.

[0078] Figure 3 A flowchart illustrating an embodiment of a method 300 for providing deformation data for deformation analysis is shown. The method 300 for providing deformation data can be used in this case. Figure 1The method 300 for providing the information is performed by a device or similar apparatus. The method includes a reading step 324, a determining step 326, an execution step 330, and optionally a obtaining step 332.

[0079] In the read-in step 324, image data provided by the camera is read from the interface to the camera. The image data represents a camera image of at least one predefined even- and / or odd-numbered point symmetry region in the camera environment. The at least one predefined point symmetry region is generated on, at, or within a deformable carrier medium. Then, in the determination step 326, the image data and determination rules are used to determine at least one center of symmetry for the at least one point symmetry region. Subsequently, in the execution step 330, the position of the at least one center of symmetry in the camera image is compared with the predefined position of at least one reference center of symmetry in a reference image relative to a reference coordinate system to determine the positional deviation between the center of symmetry and the reference center of symmetry. Subsequently, according to an embodiment, in the determination step 332, the positional deviation is used to determine the displacement information of at least one subset of pixels in the camera image relative to corresponding pixels in the reference image. The positional deviation and / or the determined displacement information are used to provide deformation data.

[0080] According to one embodiment, the image data read in step 324 represents a camera image of at least one pattern composed of multiple predefined point symmetrical regions. Here, in step 326, the geometric arrangement of the symmetry centers of at least one pattern is determined, a geometric sequence of symmetry types of the symmetry centers is determined, and / or the pattern is determined from multiple predefined patterns using the sequence. This arrangement and / or the sequence represents an identification code for the pattern. Optionally, step 326 and / or step 330 are performed jointly for all symmetry centers regardless of their symmetry type, or individually for symmetry centers of the same symmetry type, depending on their symmetry type.

[0081] Figure 4 A flowchart illustrating an embodiment of a method 400 for controlling deformation analysis is shown. The method 400 for control can use... Figure 1 The control device or similar device is used to perform the operation. Furthermore, the control method 400 can be combined with... Figure 3 The method or similar method is used to perform the control. The control method 400 includes an evaluation step 444 and an output step 446.

[0082] In evaluation step 444, the evaluation is based on... Figure 3 The deformation data provided by the method or similar method is used to generate control signals that depend on the deformation data. Subsequently, in output step 446, the control signals are output to an interface of the device used to perform the deformation analysis to control the deformation analysis.

[0083] Figure 5 A flowchart illustrating an embodiment of a method 500 for manufacturing is shown. The method 500 can be performed to manufacture at least one predefined point-symmetric region for... Figure 3 The method or similar method used and / or provided Figure 4 The method or similar method used for control. The manufacturing method 500 may also be combined with or used... Figure 2 The manufacturing method 500 is performed using equipment or similar equipment used for manufacturing. The manufacturing method 500 includes a generating step 502 and a producing step 506.

[0084] In generation step 502, design data representing a graphical representation of at least one predefined point symmetry region is generated. Subsequently, in generation step 506, the design data is used to generate at least one predefined point symmetry region on, at, or within the carrier medium to manufacture at least one predefined point symmetry region.

[0085] Figure 6 A schematic diagram of a carrier medium 600 according to an embodiment is shown, comprising a pattern 610 consisting of predefined point symmetric regions 110A and 110B. In this case, each predefined point symmetric region 110A and 110B corresponds to or is similar to... Figure 1 The pattern 610 is defined by predefined point symmetry regions. In the first part of the illustration A, a pattern 610 consisting only of 49 exemplary predefined point symmetry regions 110A and 110B is shown, and in the second part of the illustration B, a pattern 610 consisting only of eight exemplary predefined point symmetry regions 110A and 110B is shown. In this case, the first predefined point symmetry region 110A has an odd number of point symmetries as its symmetry type, while the second predefined point symmetry region 110B has an even number of point symmetries as its symmetry type. In this case, a noise-like image pattern having the corresponding pattern 610 is printed on each carrier medium 600.

[0086] Based on Figure 6 This illustrates the use of symmetry in machine vision according to an embodiment, where symmetry can be designed to be difficult or almost imperceptible to humans, but simultaneously robust to the embodiment, locally accurate, and detectable with minimal computational effort. In this case, point symmetries are more or less hidden in pattern 610, and the observer can hardly identify these point symmetries. Through... Figure 6Predefined point-symmetric regions 110A and 110B are highlighted graphically, allowing a human observer to identify these regions within a noisy image pattern on the carrier medium 600. The first part of the illustration A includes 49 exemplary circular symmetric regions 110A and 110B, with only 25 exemplary first regions 110A having an odd number of point symmetries and 24 exemplary second regions 110B having an even number of point symmetries. In the second part of the illustration B, the symmetric regions 110A and 110B are chosen to be larger than those in the first part of the illustration A, with only five exemplary having an odd number of point symmetries and only three exemplary having an even number of point symmetries, thus being particularly suitable for larger camera distances or lower image resolutions. Therefore, the circular symmetric regions 110A and 110B are located on the carrier medium 600, designed as a plate, where, in the case of odd or negative point symmetry, a bright dot mirror will be imaged as dark, and vice versa, while in the case of even or positive point symmetry, this reversal does not occur. If multiple patterns 610 are required, these patterns can be designed to be distinguishable. This can be accomplished by arranging the symmetrical centers of regions 110A and 110B, such as Figure 6 As shown, the first part of the illustration A and the second part of the illustration B are easily distinguishable, or are based on the sequence of negative or odd point symmetry and positive or even point symmetry of regions 110A and 110B within the corresponding pattern 610.

[0087] Figure 7 A schematic diagram of a carrier medium 600 according to an embodiment is shown, having a pattern 610 composed of predefined point-symmetric regions. The pattern 610 in this case corresponds to or resembles a pattern from... Figure 6 One of the patterns, in which pattern 610 is... Figure 7 It is shown in the illustration but not highlighted graphically. Figure 7 The example shown is only an illustration of the relationship between the two. Figure 6 The carrier medium is similar to ten carrier media 600.

[0088] Figure 8 It shows that it has a source Figure 7 A schematic diagram of the pattern 610 on the carrier medium 600, wherein the pattern or predefined point symmetrical regions 110A and 110B are graphically highlighted. By way of example only, the pattern 610 with predefined point symmetrical regions 110A and 110B is arranged or graphically highlighted on ten carrier media 600 in this case.

[0089] therefore, Figure 7 and Figure 8Ten patterns 610 optimized for distinguishability are shown only as an example. Each pattern 610 has a separate arrangement of odd-point symmetric regions 110A and even-point symmetric regions 110B. The pattern 610 is thus encoded through this arrangement. The encoding is chosen to be mutually coordinated and / or optimized through training so that even if the ten patterns 610 are captured by a camera rotated, mirrored, or partially hidden, these ten patterns remain clearly identifiable and distinguishable. Figure 7 and Figure 8 In pattern 610, the point symmetry regions 110A and 110B at the four corners of each carrier medium 600 are intentionally designed to be slightly more prominent. This is unrelated to the function itself, but provides practical advantages when manually assembling carrier media 600 with pattern 610. Carrier media 600 with pattern 610 can be arranged arbitrarily within the scope of the manufacturing methods already described, for example, serially in three dimensions or planarly, or as a surface. Within the scope of the supply methods already described and / or by means of the supply devices already described, the point symmetry center of pattern 610 can be found correctly and accurately. Pattern 610 can, for example, be printed on a solid plate of any size, which can optionally be placed in an arrangement partially orthogonal to each other. Even in the case of blurred imaging of pattern 610 by camera, the symmetry center can be detected sufficiently well to achieve the described function. Therefore, the detection of the point symmetry center is robust for blurred imaging. This expands the application to situations where shallow depth of field is used, such as in low-light scenes, or when the camera's focus or autofocus is incorrectly set or when perfectly sharp images cannot be achieved, such as in liquids, turbid or moving media, in the edge areas of the lens, or during relative movement between pattern 610 and the camera (motion blur, directional blur). Even though point symmetry occurs naturally and especially in artificially designed environments, the potential false detections based on them differ spatially from those based on the correct pattern 610, and therefore the two sets can be easily separated or distinguished from each other.

[0090] To demonstrate that the method described above is also applicable to moving, non-flat, and even elastic surfaces, one can... Figure 7 and Figure 8 The pattern 610 is printed on, for example, paper and assembled into a flexible box. The method described above applies without problem even to non-flat or elastic surfaces (e.g., those made of paper). This makes it possible to determine the movement of these surfaces. Unlike many materials, paper, while not permissible to be cut, exhibits point symmetry that is invariant to shearing, thus shearing poses no problem.

[0091] In particular, the orientation of the center of symmetry in the camera image can be precisely determined. However, extending this precise measurement to the entire surface of pattern 610 may also be of interest in various applications. That is, each point or pixel of pattern 610 indicates where that point or pixel is located in the camera image. This then allows, for example, determining the minimum deviation between the actually observed pattern 610 and the ideal pattern based on the ground truth. For example, it is of interest to determine the accurate shape of pattern 610 when it is applied to a non-smooth or non-rigid surface by printing and thereby creating, for example, variable folds or indentations in pattern 610. Patterns with random characteristics are particularly well-suited for finding corresponding points from a first image to a second image. Here, the first and second images can be recorded chronologically from different perspectives using the same camera or two cameras.

[0092] Specifically, we should now consider a scenario where the first image is a real image from a camera and the second image is an artificially generated (stored) image of a given pattern (also called a reference image), which is placed (e.g., scaled, rotated, affine mapped, projected) onto the second image based on a found center of symmetry, such that it approximates the real (first) image as closely as possible. For the reference image, processing steps required for the first image from the camera, such as image preprocessing, are skipped or omitted if necessary. Known methods, such as optical flow or parallax estimation, can then be applied to find, for example, the corresponding pixel in the reference image for each pixel in the camera image—or vice versa. This results in a two-step process: in the first step, the found center of symmetry, along with any necessary encoding, is used to register or coarsely align the real image with the known pattern. This then represents initialization so that, in the second step, optical flow methods are used again to precisely determine the minimum deviation in the sense of local displacement between the registered real image and the pattern, and, if necessary, for each point or pixel of the image or pattern 610. The smaller the search area, the less computational effort is required for the second step. The computational workload here is typically very small—due to the good initialization from the first step. Since both steps require very little computation, high pixel throughput is achieved on common computer platforms, defined as the product of the frame repetition rate [images / second] and the image size [pixels / image]. If local inconsistencies are not found, it can usually be explained by the occlusion of an object by the line of sight toward pattern 610. From this, the shape or outline of the occluded object can be inferred.

[0093] A reference image should be provided for the two steps described above. This can be achieved by maintaining an associated reference image for all patterns 610 under discussion in a memory. The resulting memory overhead can be reduced by storing only the relevant parameters needed to recalculate or generate the reference image when required. For example, pattern 610 can be generated according to simple rules using a quasi-random number generator. The term "quasi" here means that the random number generator actually operates according to deterministic rules, so its results are reproducible, which is advantageous here. Here, the rules should be understood, for example, as what diameters the symmetrical regions 110A and 110B have, how mirroring should be performed, and how pattern 610 is composed of multiple patterns with different levels of detail in a weighted manner, such that the pattern is well detectable at short, medium, and long distances. Thus, it is sufficient to store only the initialization data (seed) of the quasi-random number generator and, if necessary, the selection of rules for constructing pattern 610. With the help of this formation rule, the reference pattern can be generated repeatedly and identically when needed (and then deleted again).

[0094] In summary, the two-step process can be represented as follows. In the first step, the centers of symmetry are found and their symbols are determined. Here, the symbols represent the case distinction between odd and even symmetry. By comparing the symbol sequences, it can be determined which of the multiple patterns is involved. The symbol sequence of pattern 610 can also be called a code. This code can be described in a compact way and requires a maximum of 64 bits for pattern 610 with, for example, an 8×8 center of symmetry. For comparison purposes, all existing or considered codes should be stored. From this set, a code that is as consistent as possible with the observation is searched. This result is usually explicit. Even if the camera can only capture a portion of pattern 610, for example, due to occlusion, such a search is still possible because, in this example with an 8×8 center of symmetry, the codes provide a very large number of up to 2 64 This presents several possibilities, but the number of completed patterns 610 will be much smaller, thus providing a high degree of redundancy. For each stored code, information needed to generate a reference image, such as parameter and rule selection, should also be stored. This reference image is generated for the second step, for example, as needed, i.e., only generated when required, and only temporarily if necessary.

[0095] Based on the center of symmetry location found in the camera image coordinates in the first step and its corresponding known location in the reference image, a transformation rule can be calculated to map these coordinates to each other as well as possible, for example, using projection or affine mapping, which is optimized in the sense of least squares. Through this transformation and appropriate filtering of the image data, the two images can be transformed (distorted) into a common coordinate system, such as the coordinate system of the camera image, the coordinate system of the reference image, or any third coordinate system. Then, a more accurate comparison is made between the two images that have thus been aligned, for example, using optical flow methods. For example, for each pixel of the first image (preferably considering its environment), the best corresponding pixel in the second image with the environment is searched. The relative displacement of the corresponding positions can be expressed as displacement information, in particular as absolute coordinates or displacement vectors. Such displacement vectors can be obtained with sub-pixel precision, so the correspondence is usually not on the pixel grid but between pixel grids. This information allows for highly accurate analysis of the entire surface of pattern 610 captured in a camera image, for example, to analyze the deformation or distortion of pattern 610 or its carrier medium 600 using an elastic pattern, or to analyze imaging deviations in the optical path in the case of a rigid pattern.

[0096] If the searched correspondence is not found in the expected area, partial occlusion of pattern 610 can be inferred. The cause of occlusion could be, for example, an object located on pattern 610, or a second pattern that partially occludes the first pattern. Valuable information, such as the mask or outline of an object, can also be obtained from this occlusion analysis.

[0097] Figure 9 A schematic diagram of predefined point-symmetric regions 110A and 110B according to an embodiment is shown. In this case, each predefined point-symmetric region 110A and 110B corresponds to or is similar to a predefined point-symmetric region from one of the above figures. A second point-symmetric or even-numbered point-symmetric region 110B, including its center of symmetry 112, is shown in first part of the illustration A, and a first point-symmetric or odd-numbered point-symmetric region 110A, including its center of symmetry 112, is shown in second part of the illustration B. In this case, the predefined point-symmetric regions 110A and 110B represent regions formed by gray levels.

[0098] The use of point symmetry offers the following advantages over other forms of symmetry: point symmetry is preserved when the pattern and / or at least one predefined point-symmetric region is rotated about the viewing axis; point symmetry is also preserved when the pattern and / or at least one predefined point-symmetric region is tilted, i.e., at a tilted viewpoint. Rotation and tilting of the pattern and / or at least one predefined point-symmetric region do not cause problems for the detection of odd-numbered and even-numbered point symmetry, as both are preserved in the process. Therefore, the methods or approaches mentioned above are also applicable to tilted viewpoints of the pattern or at least one predefined point-symmetric region. In the case of even-numbered point symmetry, grayscale or color values ​​are preserved, for example, when the points are mirrored.

[0099] and Figure 9 In the first part of the diagram A, at the center of symmetry 112 points, symmetrically, for each gray value g, the same partner gray value g is found. PG =g. In Figure 9 The second part of the illustration, Figure B, shows odd-point symmetry, where each grayscale value is inverted: for example, white becomes black and vice versa, light gray becomes dark gray and vice versa. In the example where the grayscale value g is in the interval 0 ≤ g ≤ 1, from... Figure 9 In the diagram, half of region 110A shown at the top is derived from the original grayscale value g based on g PU =1-g forms the grayscale value g that is mirrored by the point in the simplest possible way. PU Nonlinearity can also be integrated into this inversion, for example, gamma correction, to compensate for other nonlinearities in image display and image recording. The formation of suitable odd- or even-numbered point symmetric patterns is correspondingly straightforward. For example, Figure 9 In the diagram, half of the corresponding area 110A or 110B shown at the top is arbitrarily set or randomly generated. From this, it is then concluded that... Figure 9 The lower half of the diagram is shown, and it is mirrored by points, where the gray values ​​of the odd-numbered points are reversed or the gray values ​​of the even-numbered points are not reversed.

[0100] This observation or generation can also be extended to colored patterns and / or predefined point-symmetric regions. In this case, when the number of points is odd-point symmetric, the RGB values ​​after point mirroring can be formed by inverting the individual original RGB values, which is the simplest possibility, i.e., r PU =1-r (red), g PU =1-g (g represents green here), b PU = 1 - b (blue). Thus, for example, dark violet is imaged as light green, and blue is imaged as orange. A color pattern can represent more information than a monochrome pattern, which can be advantageous. A prerequisite for using this advantage is that color information is also used to convert the raw image (i.e., a color image from a camera or other imaging sensor) into a descriptor.

[0101] The specific implementation of pattern 610 and / or at least one predefined point symmetrical region 110 or 110A and / or 110B should also be discussed below with reference to the above figures.

[0102] Regarding the arrangement of pattern 610 and / or at least one predefined point symmetrical region 110 or 110A and / or 110B, for example, Figure 6 As shown, the point-symmetric regions 110 or 110A and / or 110B can be, for example, circular, and these regions can be mostly arranged in a regular grid in pattern 610. For example, the faces between the circular regions 110 or 110A and / or 110B can remain unused. Alternatives exist: for example, regions 110 or 110A and / or 110B can be square and connected to each other without gaps, thus using an entire face; or the symmetric regions 110 or 110A and / or 110B can be regular hexagonal faces, also connected to each other without gaps, thus using an entire face.

[0103] In this association, Figure 10 A schematic diagram of a pattern 610, comprising predefined point symmetric regions 110A and 110B according to an embodiment, is shown. The predefined point symmetric regions 110A and 110B in this case correspond to or resemble... Figure 1 , Figure 6 and / or Figure 8 The predefined point symmetric region in the. Figure 10 Regions 110A and 110B in pattern 610 are both circular and arranged on a hexagonal grid. In this case, the distance between grid points or centers of symmetry can correspond to the diameter of the circle. Thus, the unused surface 1010 between regions 110A and 110B in pattern 610 can be minimized.

[0104] Other arrangements and shapes, such as rectangles, polygons, etc., are also possible, and they can be combined with each other in shape and / or size. For example, alternations of pentagons and hexagons, like a regular soccer ball. Shapes can also be arranged in other ways, such as rotation, with asymmetrical regions where necessary. The center of symmetry can also be located outside the point-symmetric region itself. This is the case, for example, when a torus is used as a shape. It is also not necessary for all point-symmetric regions to lie in a common plane. Instead, they can lie on different faces arranged in space, and these faces are also allowed to be non-flat.

[0105] Pattern 610 and / or at least one predefined point symmetrical region 110 or 110A and / or 110B can be formed in a variety of ways. Only a few examples are illustrated below. Random or quasi-random patterns, such as noise patterns. By introducing low spatial frequency components, these patterns are formed such that they are perceived as noise patterns with sufficiently high contrast even at moderate to large distances from the camera. So-called white noise, i.e., uncorrelated grayscale values, is not suitable for this purpose. Aesthetically pleasing, and where necessary, regular patterns, such as floral patterns, tendril patterns (leaves, branches, flowers), decorative patterns, mosaics, mathematical patterns, traditional patterns, onion patterns, patterns composed of iconic symbols (hearts, etc.), imitations of random patterns in nature (e.g., farmland, woodland, lawns, pebble beaches, sand, bulk materials (gravel, salt, rice, seeds), marble, rubble, concrete, brick, slate, asphalt surfaces, starry skies, water surfaces, felt, hammered paint, rusted iron sheets, sheepskin, scattered particles, etc.), and scene photographs with any content. To generate point-symmetric regions and / or patterns from this pattern suitable for the purposes described herein, half of the corresponding face is arbitrarily pre-given, and a second half is constructed by point mirroring, with grayscale or color values ​​inverted if necessary. See also [link to relevant documentation]. Figure 9 As a simple example.

[0106] There are countless possibilities regarding the materials, surfaces, and manufacturing of pattern 610 and / or at least one predefined point symmetrical region 110 or 110A and / or 110B. The following list is not exhaustive: black and white printing, grayscale printing, or multicolor printing on various materials; printing on or behind glass or transparent films; printing on or behind frosted glass or translucent films; printing or embossing on paint, metal plates, films, plastics, rubber, fabrics, wood, leather, glass, stone; embossing on plastics or paper / cardboard; etching on glass, metal, or ceramic surfaces; milling in wood, cardboard, metal, stone, etc.; fired surfaces in wood or paper; photographic exposure of paper or other materials; transient or decaying or water-soluble patterns for short-term applications on plant materials, ash, sand, wood, paper, fruit, food peels, etc.; displays as holograms; displays on monitors or displays (which may vary over time if necessary); displays on LCD films or other display films (which may vary over time if necessary), etc.

[0107] Regarding the possibilities of relief manufacturing, such as milling, embossing, stamping, etc., it should be noted that the area should be perceived by the camera as having odd and / or even point symmetry. Therefore, it may be necessary to consider, for example, later lighting (e.g., light incident obliquely on the relief) and nonlinearities and other disturbances in optical imaging during the design phase. Whether the 3D shape or relief itself has an even and / or odd point symmetry type is not important; rather, it is the image recorded by the camera that shows this symmetry. Here, the direction of light incident or illumination and the reflection of light on the surface are also relevant and should be considered together in the design. Regarding image recording and lighting, it should be noted that the recording technology should be designed to be suitable for capturing pattern 610 and / or at least one predefined point symmetry region 110 or 110A and / or 110B. In particular, when there is rapid relative movement between pattern 610 and / or (one or more) areas 110 or 110A and / or 110B and the camera, it is recommended to use appropriate lighting (e.g., flashlight, strobe light, or bright LED light) so that the exposure time and motion blur in the resulting image can be kept minimal. For various applications, it is meaningful to apply pattern 610 and / or (one or more) areas 110 or 110A and / or 110B to a transparent or translucent surface. This allows pattern 610 and / or (one or more) areas 110 or 110A and / or 110B to be illuminated from one side and observed from the other. With this solution, interfering reflections of the light source on the carrier medium can be effectively avoided. In principle, there is freedom to choose the front or back of the carrier medium for the arrangement of pattern 610 and / or (one or more) areas 110 or 110A and / or 110B, the light source, and the camera. When selected, the risk of pattern 610 and / or (one or more) areas 110 or 110A and / or 110B or the camera being contaminated or pattern 610 and / or (one or more) areas 110 or 110A and / or 110B being worn can also come into play: thus, for example, it makes sense to apply pattern 610 and / or (one or more) areas 110 or 110A and / or 110B and the camera to the back, because they are better protected there from, for example, dust or water, or because pattern 610 and / or (one or more) areas 110 or 110A and / or 110B are protected there from mechanical wear.

[0108] A method, also used in embodiments, is disclosed in DE 10 2020 202 160, which is published later, to reliably and with minimal computational effort find symmetrical regions or patterns in an image. In this case, the original image, i.e., a color or grayscale image from a camera or other imaging sensor, is converted into an image of descriptors, wherein the descriptors are formed based on the local environment of the original image. Here, a descriptor is another representative form of the local image content, prepared in a form that is easier to process. More simply here, it is understood in particular to include information about the environment of a point, not just about the point itself, a high degree of invariance to brightness or illumination and its variations, and low sensitivity to sensor noise. The descriptor image can have the same resolution as the original image, such that approximately one descriptor exists for each pixel of the original image. Alternatively or additionally, other resolutions are also possible.

[0109] A signature is formed from a corresponding descriptor, represented as a binary word in a computer unit, or from multiple adjacent descriptors. The signature describes the local environment of the pixels in the original image as characteristically as possible. The signature can also be the same as a descriptor or a portion thereof. The signature is used as an address to access a lookup table. Therefore, if the signature consists of N bits, it can access a lookup table of size 2. N A lookup table (i.e., 2 to the power of N). Advantageously, the word length N of the signature should not be chosen too large, as the storage requirements of the table grow exponentially with N: for example, 8 ≤ N ≤ 32. The signature or descriptor is constructed such that the signature symmetry can be determined using simple operations, such as a bitwise XOR operation on a subset of bits. Example: S P =s^R P , where s is a signature of length N bits, and RP is a point-symmetric (P) coordinating reflector (R). The symbol ^ represents a bitwise XOR operation. Therefore, the signature S P The point-symmetric counterpart of the signature 's'. This relationship also applies to the opposite direction.

[0110] If the construction of the descriptor or signature is fixed, the reflector is therefore automatically set (and constant). By applying it to any signature, that signature can be converted into its symmetric counterpart. An algorithm exists that can find one or more symmetric signature pixels for a given signature at the current pixel within an optional finite search window. The center of symmetry is then located in the middle of the line connecting the positions of these two pixels. Voting weights are output there or as close as possible and collected in a voting map. In the voting map, the output voting weights accumulate at the locations of the searched centers of symmetry. These centers of symmetry can thus be found, for example, by traversing the voting map to find the accumulation points. This works for point symmetry, horizontal axis symmetry, vertical axis symmetry, and other symmetries where needed, such as mirror symmetry on other axes, and rotational symmetry. More precise localization with sub-pixel accuracy can be achieved if the local environment is also included in the observation when evaluating the voting matrix to determine the accumulation points and precisely locating the centers of symmetry.

[0111] DE 10 2020 202 160 Figure 15 An algorithm is presented that can find point-symmetric correspondences with the currently observed signature. However, this paper only considers even-numbered point symmetries.

[0112] According to an embodiment, the algorithm is extended to odd-point symmetry. Particularly advantageous here is that odd-point and even-point symmetries can be determined simultaneously in a single common traversal. This saves time, as only one traversal of the signature image is needed instead of two, and reduces latency. When only one (instead of two) traversal is required, the streaming mode processing can provide the results of the symmetry search with significantly lower latency. Here, processing begins as soon as the first pixel data from the camera arrives, and the processing steps are executed intensively and sequentially. This means that the signature is computed as soon as the necessary image data from the local environment of the current pixel is available. A symmetry search is immediately performed on the newly formed signature. Once some parts of the voting matrix are complete (as is the case when these parts are no longer and will no longer be part of the search area), these parts can be evaluated immediately, and the found symmetry (strong symmetry center) can be output immediately. This process results in very low latency, typically corresponding to only a small number of image rows, depending on the height of the search area. Low latency is crucial if a rapid response is required, such as within an adjustment loop where the actuator influences the relative pose between the symmetric object and the camera. Memory is also saved. The voting map can be used for both even-point and odd-point symmetry forms, where the two symmetry forms or types with different signs participate in the voting. For example, the voting weight is subtracted for odd-point symmetry and added for even-point symmetry. This will be explained in more detail below. Furthermore, energy can be saved by conserving memory. The aforementioned low-latency implementation also results in only a small amount of intermediate data needing to be stored compared to the entire image. This memory-efficient operation is particularly important for cost-critical embedded systems and also leads to savings in energy requirements.

[0113] Figure 11 A schematic diagram illustrating the use of lookup table 1150 according to an embodiment is shown. Lookup table 1150 can be generated by... Figure 1 The specific means by which the equipment or similar equipment is provided is used. In other words, Figure 11 An example of the algorithm process for finding symmetric correspondences at search points is as follows: Figure 1 The equipment or similar equipment used for provision and / or Figure 3 A snapshot related to the method or similar method being provided. Specifically, Figure 11 The illustration in the document is also similar to that in the later published DE 10 2020 202 160. Figure 15 , among which here Figure 11 It also includes extensions to include even-point symmetry and odd-point symmetry.

[0114] Lookup table 1150, also known as an entry table, is shown. A pixel grid 1100 is illustrated, in which signatures with exemplary values ​​2412 are generated for the currently observed or processed pixel. In other words, Figure 11 A snapshot is shown during the formation of links of pixels or pixel coordinates with the same signature s. For clarity, two of at most N possible chains are shown, and this is for signature S. PG =364 and for signature S PU =3731. In pixel grid 1100, a reference to the position of the last previously signed image with the same signature value is stored for each pixel. This generates links to positions with the same signature. Therefore, the signature value itself does not need to be stored. For each signature value, the corresponding entry position in pixel grid 1100 is stored in a lookup table 1150 or an entry table with N table fields. Here, N corresponds to the number of possible signature values. The stored value can also be "invalid". The contents of lookup table 1150 or the entry table and the referenced images (linked images) change dynamically.

[0115] In pixel grid 1100, pixels are processed row by row, for example from... Figure 11 Starting from the top left, as indicated by the arrow, and having currently progressed to the pixel with signature s = 2412. Links between pixel positions having the same signature s are stored only for the first image region 1101. For the second image region 1102 in the lower image portion, links and signatures are not yet known at the indicated time point, and for the third image region 1103 in the upper image portion, links are no longer needed, for example, due to the limitations of the search area, where the link memory for pixels in the third image region 1103 can be freed again.

[0116] For the newly formed signature s, by applying reflector R PG Form an even-numbered point mirror signature S PG =364. Index PG represents point-symmetric, even. Index PU, representing point-symmetric, odd, is also used below. This value is used as the address in lookup table 1150 to find the address assigned to the same signature value S. PG The entry is found in the link at pixel position 364. At the indicated time point, lookup table 1150 includes two elements: the entry pixel position corresponding to signature s and a reference to that position indicated by the curved arrow. For clarity, other possible contents of lookup table 1150 are not shown. Signature value S PGThe link = 364 includes three pixel positions, only shown here as an example. Two of these are located within search region 1104, which can also have a different form than shown here, such as a rectangle or a circle. Here, when traversing unidirectionally along the link, starting from the bottom, two symmetrical corresponding candidates for the two points located within search region 1104 are found. The third correspondence as the first element of the link symmetrical to the even-numbered point is not of interest here, as it is located outside search region 1104 and therefore too far from the current pixel position. If the number of symmetrical center candidates 1112 is not too large, the voting weight of the corresponding symmetrical center position can be output for each symmetrical center candidate 1112. The symmetrical center candidates 1112 are located at the positions of signatures s and the corresponding even-numbered point mirror images of signatures S, respectively. PG The middle of the connecting axis between them. If there is more than one candidate symmetry center 1112, the voting weight can be reduced respectively, for example, the reciprocal of the number of candidate symmetry centers can be used as the corresponding voting weight. Thus, the unclear candidate symmetry center is weighted less than the clear candidate symmetry center.

[0117] We will now consider and use odd-point mirror signatures. Figure 11 In the snapshot shown, the newly formed signature s is processed by applying another reflector R. PU Form an odd-numbered point mirror signature S PU =3731. Similar to the process described above for mirror signatures of even-numbered points, the same steps are performed for mirror signatures of odd-numbered points. The entry for the corresponding link is found using the same lookup table 1150. Here, lookup table 1150 points to the link symmetrically represented for signature 3731 at odd-numbered points. The first two pixel positions along this link again lead to the formation of symmetry center candidates 1112, as they are arranged in search area 1104 and because the number of candidate symmetry center candidates 1112 is not too large. The last pixel position along this link is located in the third image area 1103. This area is no longer needed because it can no longer enter the search area 1104 that slides line by line here.

[0118] If the next reference within the link points to the third image region 1103, traversal along the link can be terminated. Of course, traversal also terminates when the end of the link is reached. In both cases, it makes sense to limit the number of symmetry center candidates 1112; that is, if there are too many competing symmetry center candidates 1112, all symmetry center candidates 1112 are discarded. Furthermore, it makes sense to terminate traversal along the link early if neither its end nor the third image region 1103 can be reached after a pre-given maximum number of steps along the link. In this case, all symmetry center candidates 1112 found up to there should also be discarded.

[0119] The memory used for links in the third image region 1103 can be freed up again, so only the link memory needs to be reserved for the size of the first image region 1101. Therefore, the link memory requirement is generally low and here depends essentially only on one dimension of the search region 1104 (here, the search region height) and one dimension of the signature image (here, the signature image width).

[0120] The candidate for symmetry center 1112 or the candidate for symmetry center may not always fall exactly at the pixel location; instead, there are three other possibilities. Therefore, there are a total of four possibilities:

[0121] 1. The point or center of symmetry candidate 1112 falls on the pixel position.

[0122] 2. The point or symmetry center candidate 1112 falls in the middle between two horizontally directly adjacent pixel positions.

[0123] 3. The point or symmetry center candidate 1112 falls in the middle between two vertically directly adjacent pixel positions.

[0124] 4. The point or symmetry center candidate 1112 falls in the middle between four directly adjacent pixel positions.

[0125] In cases 2 through 4 where the details are unclear, it is advantageous to distribute the voting weights to be output evenly across the participating pixel locations. The output voting weights are then input into the voting matrix and summed or accumulated therein.

[0126] Here, not only positive voting weights but also negative voting weights are used simultaneously. Specifically, even-numbered symmetry is equipped with a different sign (positive) than odd-numbered symmetry (negative here). This leads to a clear result: in image regions without symmetry, which in practice largely represent the majority, the positive and negative voting weight outputs are approximately balanced, thus roughly canceling each other out in the voting matrix. Therefore, on average, approximately zero is obtained in the voting matrix. Conversely, in either odd-numbered or even-numbered symmetric regions, strong extrema are obtained in the voting matrix, and in this embodiment, a negative minimum is obtained when symmetric at odd points, and a positive maximum is obtained when symmetric at even points.

[0127] According to the embodiment shown here, the same resources are used for both odd-point symmetry and even-point symmetry, namely lookup table 1150 or entry table, link graph, voting matrix, which in particular saves memory requirements, and both symmetry forms or types are observed in a single common traversal, which saves time and intermediate memory.

[0128] Figure 12 A schematic diagram of a voting matrix according to an embodiment is shown in Table 1200. Table 1200 relates to a voting matrix, as a means of... Figure 1 A 3D image of a camera image processed by a provided device or similar device, in which the camera recorded from... Figure 6 The second part of the illustration shows the pattern. In the voting matrix or chart 1200, three exemplary maximum values ​​1210B and five minimum values ​​1210A can be clearly identified, representing those from... Figure 6 The second part of the illustration shows the symmetrical regions of three even-numbered points and five odd-numbered points. Outside of these extreme values, the values ​​in the voting matrix are close to zero. Therefore, the extreme values ​​can be determined very easily, and the location of the center of symmetry in the camera image can be determined explicitly and precisely.

[0129] Figure 12 These extreme values ​​are shown to be very obvious and therefore can be obtained through Figure 1 The equipment or similar equipment used for provision and / or Figure 3 The method or similar method used to provide this information is simple and unambiguous in detecting the symmetry. Here, information about the type of symmetry (i.e., odd or even) is included in the symbols. If the local environment of the corresponding extrema is also considered when evaluating the voting matrix, the location of the symmetry center can be determined with high precision at sub-pixel accuracy. Corresponding methods for this purpose are known to those skilled in the art. If the pattern is constructed appropriately, odd-point symmetry and even-point symmetry will not compete with each other. Thus, image regions (if any) will either have odd-point symmetry or even-point symmetry. Even if odd-point symmetric regions and even-point symmetric regions are close to each other in the camera image, it can be ensured that their symmetry centers remain spatially separated or distinguishable. Then, by jointly processing negative and positive symmetry, advantages in terms of resources and speed are achieved.

[0130] According to an embodiment, the processing of odd-point symmetry and even-point symmetry can be set up separately. Separating them before inputting entries into the voting matrix makes sense: two unsigned voting matrices are then set up to replace the common signed voting matrix, where the voting weights for negative symmetry are input into the first voting matrix and the voting weights for positive symmetry are input into the second voting matrix. This presents a potentially interesting advantage: a pattern can also be constructed and considered by the detection algorithm, which simultaneously exhibits odd-point symmetry and even-point symmetry with their centers of symmetry locally coinciding. While this mixed form of symmetry is highly unusual, this unusualness guarantees that it is extremely unlikely to be confused with randomly occurring patterns in the image. The two voting matrices are then searched to find the maximum value present at the same location in both matrices. Another possible advantage of processing odd-point symmetry and even-point symmetry separately is that it is easier to parallelize, thereby enabling faster execution when necessary. Because by using two voting matrices, access conflicts during voting weight input can be avoided, saving waiting time.

[0131] Figure 13 A schematic diagram of pattern 610, which is arranged in a cubic form according to an embodiment, is shown in terms of the correct identification of grid 1311. Figure 13 The pattern 610 shown is, for example, from Figure 7 or Figure 8 The pattern, in which three patterns are arranged in a cubic shape. In pattern 610, the detected or identified centers of symmetry 112A and 112B of the corresponding predefined point symmetry regions of pattern 610 are shown, wherein the signs and values ​​of the associated extrema in the voting matrix can optionally also be known. In this case, the first center of symmetry 112A is assigned to the predefined point symmetry region with odd-numbered point symmetry, and the second center of symmetry 112b is assigned to the predefined point symmetry region with even-numbered point symmetry. A correct grid 1311 is drawn for one of the patterns 610, on which the predefined point symmetry regions and thus the centers of symmetry 112A and 112B are aligned. For the other two patterns 610, the correct grid is searched, wherein in Figure 13 In the diagram, an incorrect solution to the grid search is indicated by the first label 1313, and a correct solution to the grid search is indicated by the second label 1314.

[0132] Finding the correct associated grid is a task with inherent fuzziness. After detecting the odd / even coded centers of symmetry 112A and 112B, the next step is typically to group them and determine which pattern 610 this group is assigned to, since it is not always known beforehand which patterns 610 and how many patterns 610 are included in the image. Part of this task could be finding the grid 1311 on which the centers of symmetry 112A and 112B are arranged. Instead of the square grid 1311, other topologies for arranging the centers of symmetry 112A and 112B are also considered, such as a concentric annular arrangement, see, for example... Figure 6 The second part of the illustration. As a representative example, observe square grid 1311 below.

[0133] Based on only Figure 13 The task of determining the correct grid positions for all patterns 610 by identifying the symmetry centers 112A and 112B is, in some cases, an ambiguous problem. If in Figure 13Observing pattern 610, if the correct grid 1311 has already been drawn for this pattern, it is not difficult to indicate the correct grid 1311 (to the observer). However, for the other two patterns 610 captured by the camera from a significantly more oblique perspective, it is clear that the output may be ambiguous. There are several possible solutions regarding how the grid can be placed through the centers of symmetry 112A and 112B. Here, the solution that is initially most obvious when viewed locally, i.e., the solution with an approximately vertical axis, is not the correct solution, as can be seen based on the first mark 1313. Instead, the second mark 1314 is correctly located on the grid. This shows that a naive process, such as searching for the nearest neighbor of the corresponding center of symmetry, may lead to an incorrect solution when viewed from an oblique perspective. In practice, solutions with a very oblique perspective are excluded because it is no longer possible to find the centers of symmetry 112A and 112B.

[0134] Figure 14 Shown from a tilted perspective Figure 6 The first part of the illustration shows a schematic diagram of pattern 610. Figure 14 The image shows a carrier medium 600 having a pattern 610 consisting of predefined point symmetrical regions 110A and 110B. Figure 14 The second part of the diagram, Figure B, shows the method of using... Figure 1 The equipment or similar equipment used for provision and / or Figure 3 The center of symmetry 112A and 112B of pattern 610 are used to identify or detect the pattern using the provided method or similar method. The center of symmetry 112A and 112B have been detected and at least their positions are available.

[0135] Figure 15 Showing from Figure 14 The first part of the illustration depicts pattern 610, in which predefined point symmetrical regions 110B are highlighted. Here, the predefined even-numbered point symmetrical regions 110B are highlighted graphically only as an example to illustrate the distortion of pattern 610 or regions 110A and 110B due to a tilted perspective. Here, the exemplary circular predefined point symmetrical regions 110A and 110B are distorted into ellipses by the tilted perspective.

[0136] The following is a special reference Figure 14 and Figure 15 Furthermore, the reconstruction of the correct mesh or topology of pattern 610 is discussed in general with reference to the above-mentioned figures.

[0137] From an oblique perspective, each circular region 110A and 110B from which the voting for the corresponding centers of symmetry 112A and 112B originates becomes an ellipse. This is achieved by backtracking on the corresponding centers of symmetry 112A and 112B (e.g., ...). Figure 15The voting, which contributes to the highlighted even-point symmetry center 112B, allows us to infer the shape and orientation of the corresponding ellipse. The direction and ratio of the ellipse's principal axis reveal how the ellipse can be stretched or straightened to transform it back into a circle. Observe the exemplary highlighted predefined even-point symmetry region 110B of pattern 610, which contributes to the highlighted point symmetry center 112B. Depending on the design or construction, this region 110B is circular or approximately circular, such as hexagonal. At an oblique viewpoint, this circle becomes an ellipse. When voting is conducted to identify the symmetry center 112B, pairs of symmetric points help to form extrema in the voting matrix located within this ellipse.

[0138] According to one embodiment, the origin of point pairs in the camera image that lead to the formation of sufficiently strong extrema is traced. Further processing steps are performed for this purpose. First, it is assumed that voting has been conducted and a sufficiently strong center of symmetry has been found. Therefore, the starting point is as follows: Figure 14 The second part of the diagram illustrates the situation shown in Figure B. The voting process is then iterated again in a modified form. However, the existing voting matrix is ​​not reformulated here. Instead, for each pair of symmetric points that contributes to the voting matrix, it is checked whether that contribution contributes to one of the found symmetry centers 112A and 112B, and therefore has already been contributed to in the first iteration. If so, the two positions of that pair are stored or immediately further calculated. Advantageously, the indices of the symmetry centers 112A and 112B contributed by the symmetric point pair are also stored or used here. In this way, all contributions to the successful symmetry centers can be determined afterward and (intermediately) stored or further used.

[0139] The further processing steps do not need to wait until the first processing step, namely the formation of the voting matrix and the determination of the centers of symmetry, is completed. Instead, they can begin earlier and utilize the intermediate results already achieved in the first processing step, namely the found centers of symmetry 112A and 112B. Then, from the information formed in this way, all image locations contributing to each found center of symmetry 112A and 112B can be read out. These locations are substantially, or except for a few outliers, located within the ellipse, such as... Figure 15 The example shown is with respect to the center of symmetry 112B.

[0140] Methods for determining the parameters of the ellipse are known to those skilled in the art. For example, a principal axis transformation can be formed over the set of all points contributing to the centers of symmetry 112A, 112B to determine the orientation of the principal axes and the two diameters of the ellipse. This can be achieved even without the need for intermediate storage of contributing image locations: instead, these image locations can be further processed immediately upon being known. Alternatively, an elliptical envelope can be determined around the set of points, using this elliptical envelope to enclose as closely as possible the largest possible portion of the set of points (excluding possible outliers).

[0141] Alternatively, an index image, equivalent to an index matrix, can be created instead of a list of points. The index image serves the same purpose—forming the parameters of all ellipses—but stores information in a different form. Ideally, the index image has the same dimensions as the signature image and is set to store indices, specifically those assigned to the found centers of symmetry 112A and 112B. Special index values, such as 0, are set to indicate that no entry exists yet. If a pair of symmetric points or signatures contributing to the i-th index is found during further processing steps, index i is entered at the two associated positions of the corresponding signature. Thus, at the end of the traversal, the following index image is obtained, in which all indices assigned to the centers of symmetry 112A and 112B appear multiple times, forming elliptical regions: thus, apart from a few outliers, each elliptical region contains only entries with uniform indices, and index 0 at unused positions. The index image can then be easily evaluated to determine the parameters of the individual ellipses. Furthermore, it is not necessary to store the entire index image. Once the data in a segment of the indexed image stops changing, that segment can be evaluated, and the memory can be freed up again. This results in lower latency, allowing intermediate results to be provided earlier.

[0142] The known elliptic parameters can then be used to correct the two-dimensional arrangement of the detected centers of symmetry (see...). Figure 14 This allows these centers of symmetry to subsequently lie on the grid of pattern 610, which here is at least approximately square, for example only.

[0143] Figure 16 The image after viewpoint correction is shown according to an embodiment. Figure 15 A schematic diagram of pattern 610. In other words, for illustrative purposes, Figure 16 It shows that in the Figure 15 Pattern 610 is orthogonal to or perpendicular to the direction of the found ellipse or the highlighted elliptical twisted region 110B, stretched by the ratio of the two principal axis lengths. Therefore, the correct grid 1311 can be found in a simple way. Thus, with... Figure 15In contrast, the ellipse is corrected by restoring the original circular shape of region 110B. Then, determining the grid 1311 where the centers of symmetry 112A and 112B are located, or determining the adjacency relationship between the centers of symmetry 112A and 112B without errors, is a straightforward matter. Figure 16 This is for illustrative purposes only. In practice, there is no need to distort the image. Since the information about the positions of the centers of symmetry 112A and 112B already exists in compressed form, it makes sense to use only this data for further processing and to transform its coordinates, where the transformation rule is formed by the determined ellipse parameters, and makes the ellipse into a circle.

[0144] When recording camera images at a telephoto focal length, a single global transformation is sufficient to determine grid 1311 for each segment. When recording camera images using a wide-angle lens (e.g., a fisheye lens), local transformations can be used at least in some areas. Therefore, the transformation rules described above can be applied globally and / or locally. In the global transformation, all projection centers are transformed using the same common transformation rule. This is meaningful and sufficient in many cases. The common transformation rule can be formed based on the common observation of all ellipses. If the centers of symmetry 112A and 112B lie on multiple planes in space, the ellipses can be grouped according to their parameters. Here, ellipses belonging to a plane have very similar parameters—especially when the plane is flat. A global transformation rule can then be determined and applied for each group. This process is applicable to telephoto focal lengths. Local transformations are meaningful when multiple circular regions are imaged as ellipses of different shapes or orientations by camera imaging. This is especially true for wide-angle cameras or high-torque lenses.

[0145] After the transformation is applied, the centers of symmetry belonging to the same face are at least approximately located on a common grid 1311. The next task is to assign the centers of symmetry 112A and 112B to grid locations. This can be done, for example, iteratively in small steps. For example, for centers of symmetry 112A and 112B, search for up to four nearest neighbors with approximately the same distance, for which see also [link to documentation]. Figure 13The search continues from these neighbors to more distant neighbors until all captured centers of symmetry 112A and 112B belonging to pattern 610 are assigned to a common grid 1311 or can be excluded from the common grid 1311. Therefore, if a center of symmetry is encountered during this search that does not match the grid 1311 just observed in terms of distance, these centers of symmetry are not recorded, as they may be outliers or centers of symmetry belonging to other faces. This iterative search can be repeated for other faces, such that eventually every center of symmetry 112A, 112B, except for outliers, is assigned to a face. For these faces, pattern 610 can then be identified preferably based on the binary codes associated with the centers of symmetry 112A and 112B, which are respectively contained in the symbols of the extrema.

[0146] Figure 17 A schematic diagram of an embodiment of a pattern 1710 with layered symmetry is shown. Pattern 1710 corresponds to or is similar to the pattern in the above figures. More specifically, by way of example only, pattern 1710 has a two-level layered structure consisting of four predefined point-symmetric regions 110A and 110B. According to the embodiment shown here, by way of example only, pattern 1710 has two predefined odd-point-symmetric regions 110A and two predefined even-point-symmetric regions 110B. In this case, pattern 1710 has an odd-point-symmetric structure overall. The even-point-symmetric regions 110B and the odd-point-symmetric regions 110A are located at the first layered level. The overall arrangement of the odd-point-symmetric pattern 110B is located at the second layered level. The center of symmetry 112 of the second layered level is represented by a quarter circle.

[0147] Figure 18 A schematic diagram of an embodiment of pattern 1810 with layered symmetry is shown. Figure 18 The pattern 1810 in the image is similar to that from Figure 17 The pattern. More precisely, Figure 18 Another example of a two-level hierarchical structure consisting of predefined point-symmetric regions 110B is shown. In the first hierarchical level, the predefined point-symmetric regions 110B are assumed to be point-symmetric themselves. In the second hierarchical level, there is an odd-numbered point symmetry at the level of pattern 1810, where the center of symmetry 112 is located at the center of the six-part hexagon shown for illustration. This odd-numbered symmetry is represented here as a reversal of the predefined point-symmetric regions 110B, for example, mirroring a dark symbol on a light background as a light symbol on a dark background.

[0148] Figure 19 A schematic diagram of an embodiment with a layered symmetrical pattern 610 is shown. In this case, pattern 610 is composed of... Figure 17 and Figure 18Patterns 1710 and 1810, or their inversions and / or dotted mirror forms, are constructed. For example only, pattern 610 has... Figure 17 The two patterns 1710 and Figure 18 The three-tiered structure consists of two patterns 1810. Patterns 1710 and 1810 are odd in number, and therefore are mirrored at the center of symmetry 112 of pattern 610, which is located at the center of the six-part hexagon shown for illustration. For example, Figure 19 The pattern 1710 shown in the lower right corner is the reverse of the pattern 1710 in the upper left corner. This layering principle can be continued at will, that is, a fourth level, a fifth level, and so on can be constructed.

[0149] The following is for reference. Figure 17 , Figure 18 and Figure 19 Further discussion is given of patterns with hierarchical symmetry. Symmetrical patterns 610, 1710, and 1810 can be constructed in multiple levels such that, for example, there are small self-symmetrical regions in the first hierarchical level, and common observation of them leads to symmetry in the next higher hierarchical level. Figure 17 and Figure 18 Both examples exemplify how to construct two-level layered patterns 1710 or 1810. Based on this, in Figure 19 A three-level layered pattern 610 was constructed. Therefore, in Figure 19The example includes three hierarchical levels. The third hierarchical level extends across the entire surface of pattern 610 (the area enclosed by dashed lines) and includes a center of symmetry 112. In the second hierarchical level, there are four patterns 1710 and 1810 (each enclosed by solid lines), each with a central center of symmetry (not explicitly shown here). According to the embodiment shown here, there are therefore 16 predefined point symmetric regions in the first hierarchical level, each with a center of symmetry. Here, the symmetry of the third hierarchical level is visible from a greater distance. During close proximity, the four symmetries of the second hierarchical level are also visible. At shorter distances, or if the capture resolution of pattern 610 is sufficient, the symmetry of the first hierarchical level also becomes visible. Therefore, visual servoing, such as visual control of a robot in the direction of pattern 610 or in any other direction, can be implemented over a large range of distances. If finer or lower hierarchical levels can already be captured, it is generally unnecessary to capture coarser or higher hierarchical levels. It is also unnecessary to capture all symmetries at the corresponding hierarchical level simultaneously; for example, at very short distances, it is simply impossible to capture the entire pattern 610 in a camera image. Clearly, even and odd symmetries can be chosen and combined with some degree of freedom. Additional information can also be included in this setting, specifically a bit allocated for the choice between odd and even symmetries, which can be transmitted to the capture system in this way. "Some degree of freedom" here means that the remaining form of the symmetry at the corresponding hierarchical level inevitably derives from the next higher hierarchical level. In other words, for example, in… Figure 18 In the middle, for the top row, the patterns "X" and "O" can be freely chosen. Then, the second row is inevitably derived, and here it is reversed because negative point symmetry was chosen on the next layer level.

[0150] Figure 20 A schematic diagram of pattern 610 according to an embodiment is shown. In the first part of the illustration A, Figure 20 Pattern 610 is shown as an example; pattern 610 is from... Figure 8 One of the patterns. Figure 20 The first part, illustration A, is an example of implicit additional information, 8.8 = 64 bits, derived here for illustrative purposes only, based on the symmetry type of predefined point-symmetric regions 110A and 110B of pattern 610, or the symbol of the associated point symmetry. In the second part, illustration B, Figure 20Pattern 610 is shown, which is constructed, by way of example, from four predefined point-symmetric regions 110A and 110B. For example, it is constructed from one predefined odd-point-symmetric region 110A and three predefined even-point-symmetric regions 110B on a square grid. Furthermore, in this case, a code matrix 2010 for explicit additional information is arranged in pattern 610. By way of example only, the implicit additional information from the first part of the illustration A is explicitly contained in the code matrix 2010. The predefined region 110A with odd-point symmetry here represents or marks the starting row of the 8×8 matrix, thereby explicitly setting the readout order.

[0151] The following is for reference. Figure 20 The delivery of implicit or explicit additional information will be discussed in more detail.

[0152] It may be useful or necessary to transmit additional information to a recipient, such as a computer, autonomous robot, etc., based on pattern 610. The additional information can be more or less extensive. Some illustrative examples of additional information include parking spots, charging stations, southwest-facing locations at 52°07′01.9″N 9°53′57.4″E, left turns, speed limits of 20 km / h, lawnmower charging stations, etc. Various options exist for transmitting information using imaging sensors or cameras. In particular, a distinction can be made between implicitly included and explicitly included additional information; for this, see [reference needed]. Figure 20 The two examples illustrate this, one implicitly and one explicitly, providing 64 bits of additional information. Implicit additional information means that it is somehow contained within the pattern 610 itself, which is symmetrical, while explicit additional information is typically designed and captured separately from these patterns 610.

[0153] based on Figure 20 The first part of the diagram, Figure A, illustrates one possibility for transmitting implicit additional information: implicit additional information as binary code. Since the choice between odd-point symmetry and even-point symmetry is made for each symmetric region 110A and 110B when constructing pattern 610, additional binary information (corresponding to 1 bit) can be transmitted separately. If patterns with both odd and even-point symmetry are also allowed simultaneously, then the binary additional information becomes ternary additional information, i.e., three cases instead of two.

[0154] Another possibility for transmitting additional information is derived by using the non-uniform distance between the centers of symmetry of regions 110A and 110B, i.e., implicit additional information based on this arrangement. Then, with... Figure 20 The arrangements shown are different—in Figure 20The centers of symmetry are located on a square grid, and these centers of symmetry will be arranged irregularly, with additional information or a portion thereof encoded in this arrangement. Example: If the corresponding centers of symmetry are allowed to shift a fixed distance to the left / right and up / down, nine possible positions are obtained, whereby each center of symmetry can encode log2(9) = 3.17 bits of additional information. The tilt angle between the imaging sensor and pattern 610 does not pose a problem in any of the possibilities mentioned. For example, a subset of the centers of symmetry (e.g., the four outermost centers of symmetry in the corners) can be used to define the base grid of the coordinate system or rules. The deviation or binary / ternary code used for encoding is then related to this base grid.

[0155] The symmetric regions 110A and 110B used for implicit additional information should not be too small, so as to form sufficiently prominent extrema in the voting matrix. If a larger amount of additional information (especially static, location-based additional information) is to be transmitted to the receiver (e.g., a mobile robot), it is advantageous to explicitly encode this additional information.

[0156] exist Figure 20 The second part of the illustration, Figure B, shows how additional, particularly static, location-based information can be explicitly transmitted to a receiver (e.g., a mobile robot): for example, it can be agreed that additional information exists at specific coordinates in a coordinate system defined by a center of symmetry, encoded, for example, in binary (black / white) or other gradients (grayscale) or color. The process then consists of two steps: in the first step, a field, such as code matrix 2010, is found based on odd and even symmetry, and the additional information is encoded in that field. In the second step, the field is read out, and thus the information contained therein. The tilted viewing angle between the imaging sensor and pattern 610 does not pose a problem here, because for the additional information to be read out, it is neither necessary for the fundamental vectors of the found coordinate system to be perpendicular to each other nor for these fundamental vectors to have the same length. Optionally, the image can also be corrected so that a Cartesian coordinate system subsequently exists. Optionally, a display can also be mounted in the field with pattern 610, which can transmit information that changes over time and / or transmits information through time changes, in addition to information that is static over time.

[0157] Through implicit error detection, high-resolution supplementary information can also be included in the pattern 610 itself. Therefore, there is another possibility that supplementary information (particularly static, position-based) is transmitted via the pattern 610 itself: this means that supplementary information is contained within the sequence of the black-and-white, color, or grayscale pattern 610 itself. Through the above classification, this supplementary information will be both implicit and explicit. Since the pattern 610, or at least some of its parts, is symmetrical, supplementary information is automatically and redundantly included, typically doubly included separately. This applies to both odd-point and even-point symmetries. This fact can be used for error correction or error detection. For example, if the pattern 610 is contaminated, for example, with bird droppings, errors resulting from this can be detected with high reliability, because the same error is unlikely to appear at the associated symmetrical location.

[0158] Figure 21 A schematic diagram of a carrier medium 600 having pattern 610 according to an embodiment is shown. In this case, the carrier medium 600 is clothing, particularly a T-shirt or motion capture T-shirt for motion capture, on which pattern 610 is displayed. Pattern 610 corresponds to or resembles a pattern from one of the above figures. Pattern 610 includes a plurality of predefined dotted symmetrical regions. In other words, Figure 21 An example of a motion capture T-shirt, serving as the carrier medium 600, is shown, exhibiting numerous odd-numbered and even-numbered point symmetries that are difficult to perceive with the naked eye. Its function is invisible or not easily seen for the carrier medium 600.

[0159] Figure 22 It shows having Figure 21 The pattern 610 is on a carrier medium 600, in which predefined point-symmetric regions 110A and 110B are highlighted. In other words, in Figure 22 For illustrative purposes, the odd and even numbers contained in the motion capture T-shirt of carrier medium 600 or here are made symmetrically visible.

[0160] Figure 23 It shows that it has a source Figure 21 or Figure 22 The carrier medium 600 of pattern 610 highlights the centers of symmetry 112A and 112B. The centers of symmetry 112A and 112B correspond to... Figure 22 The image shows the predefined point symmetric regions 110A and 110B. In other words, Figure 23 It shows that it will come from Figure 3 The method or similar method used to provide the method is from Figure 21 The results of capturing motion images from a camera on a T-shirt or carrier medium 600. Odd and even symmetry are detected without any problems, provided that the point-symmetric regions are not excessively cropped.

[0161] The following is for reference. Figure 21 , Figure 22 and Figure 23 Especially in combination Figures 1 to 5 The application areas of motion capture with hidden symmetry will be discussed in more detail.

[0162] In traditional motion capture systems, such as those used in the film, gaming, virtual reality, augmented reality, or sports industries, actors wear motion capture suits—mostly black bodysuits with reflective balls (ping-pong ball size or smaller) or other highly visible markings. These markings can be captured by a camera-based system, typically consisting of multiple cameras. This allows the movement of the actor and their body parts to be captured and then transferred to a compositely rendered character or used for motion studies. In the context of this embodiment, such unsightly patterns and clothing that are highly intrusive or restrictive based on motion can be eliminated, for example.

[0163] According to an embodiment, point-symmetric regions 110A and 110B are applied directly to a carrier medium 600 (here, a fabric), for example, through printing, embroidery, weaving, sewing, bonding, or gluing. These regions 110A and 110B are then selectively and very densely located on the body, thus achieving better motion capture. Alternatively, the motion-capturing fabric or carrier medium 600 can be selectively worn loosely on the body, allowing it to behave like ordinary clothing and exhibit natural folds that change during movement. This dynamic, folded drape can also optionally be achieved by means of… Figure 1 Equipment or similar equipment or Figure 3 and Figure 4 One or similar methods can be used to capture motion, for example, transferring folds and droops to a compositely rendered character. Therefore, the resolution of motion capture can be improved compared to traditional methods. Patterns such as pattern 610 no longer interfere with the actor (unlike known patterns) and can be designed to be imperceptible to humans, allowing the actor to dress almost normally during recording and thus move unrecognizable, for example, in a crowd. Fabrics appropriately provided with symmetrical patterns or predefined point symmetrical regions 110A and 110B—where pattern 610 is, for example, printed, embroidered, woven, photographically exposed, or laser-burned—can also be manufactured as common rolls, from which garments for motion capture applications can be sewn. When cutting the fabric, care can be taken to ensure that the centers of symmetry 112A and 112B are located in appropriate positions, for example, symmetrical relative to the left and right halves of the body. Temporary auxiliary marks can be applied to the fabric to make cutting easier. If the cutting robot has already been equipped with a camera application... Figure 3If the method or similar method is used to identify the centers of symmetry 112A and 112B on the fabric and use these centers of symmetry to set the optimized cut, the cut can be made more accurately.

[0164] This design creates a medium for capturing movement in clothing forms—here, for example, a slightly grey T-shirt—in a way that allows for motion capture. Figure 21 As shown in the diagram. Therefore, the function of this carrier medium is not visible, and under no circumstances is its function apparent. Therefore (for illustration only) this makes the point-symmetric region in Figure 22 As can be seen in the image, odd and even point-symmetric disk-shaped regions 110A and 110B are used. The arrangement of these regions 110A and 110B is chosen, and optimized through training as needed, so that the observed body area can still be identified even if only a segment of the fabric is visible—for example, because part of the actor is obscured by other people or objects, or because their arm partially obscures the fabric. The size of the point-symmetric regions 110A and 110B can be chosen as needed or required, with smaller symmetric regions capturing greater detail of body motion. However, this also increases the demands on the camera system. To ensure very robust and accurate capture of the center of symmetry, the symmetric region in the camera image should, for example, have a diameter of at least 25 pixels, preferably 50 pixels.

[0165] Already Figure 3 The method or similar method used to provide Figure 21 The image. The result can be found in Figure 23 As observed, all unequal point symmetric regions 110A and 110B were correctly detected, and their centers of symmetry 112A and 112B were precisely located. However, if more than half of the circular surface was lost during the straight-line cutting, the centers of symmetry could no longer be found because the corresponding symmetric partners no longer existed, and the voting method for the centers of symmetry would not work. Nevertheless, if some symmetric signature pairs still existed, the centers of symmetry 112A and 112B could still be correctly found—only with a reduced intensity of local extrema. Figure 23 The size of the cross indicates the intensity of the actual measurement: small crosses are found at the edges of the cut. The same considerations apply to the partial occlusion of symmetrical regions 110A and 110B.

[0166] Another decisive advantage over traditional methods is that the accuracy of symmetry center 112A, 112B can be maintained even with partial occlusion. This is because the unoccluded symmetrically arranged point pairs continue to vote for the correct symmetry centers 112A, 112B. Conversely, in traditional methods, if a circular marker is distorted into an ellipse due to the viewing angle and is simultaneously occluded by an unknown contour portion, it is generally impossible to determine the center of the circular marker.

[0167] The following applications are conceivable, for which the resolution achievable in this manner is insufficient. For example, on a carrier medium of 600 or here... Figure 23 On the T-shirt, approximately 14 rows of symmetrical center lines 112A and 112B were found, used only as an example. Following the suggestion that each circle's diameter is approximately 50 pixels, the height of the T-shirt in the image is approximately 700 pixels, a realistically feasible value. However, it's clear that the number of symmetrical centers 112A and 112B may not be sufficient to capture the dynamic, pleated drooping. Therefore, in Figure 3 The method provided also performs a determination step. First, centers of symmetry 112A and 112B are found and registered with pattern 610, such that for each found center of symmetry 112A, 112B, its position on pattern 610 is known. Then, the texture from the image surrounding the found centers of symmetry 112A, 112B—that is, the grayscale value changes and / or color changes—is compared with the texture from the original known pattern or a reference pattern surrounding a reference center of symmetry. In this comparison, an attempt is made to reconcile the two texture changes. This can be done, for example, using a dense optical flow method, where, ideally, a corresponding pixel from the reference pattern is found for each pixel in the image. Thus, for each pixel, the optical flow vector describes the displacement between corresponding points with sub-pixel precision.

[0168] Since a coarse registration already exists after identifying the centers of symmetry 112A and 112B, only a small search area needs to be covered next to find the residual flow vector. For such a small search area, the search results are sufficiently explicit and the search workload is minimal, making this approach highly practical. Therefore, high resolution can be achieved, such as the full resolution of a camera's image sensor, sufficient to accurately capture even the smallest changes on a surface—especially the movement of skin, muscles, or tendons when wrinkled or wearing tight clothing.

[0169] Figure 24A schematic diagram of a carrier medium 600 having a pattern 610 according to an embodiment is shown. The carrier medium 600 in this case is, exemplarily, only a portion of a wing having winglets of a light aircraft. The surface of the carrier medium 600 is surrounded by a pattern 600 having a point-symmetric or predefined point-symmetric region 110, and is observed here, exemplarily, with multiple cameras 102, while the component or carrier medium 600 is circulated, for example, in a wind tunnel and is subjected to boundary forces or excited to inherent vibrations in the process. The predefined point-symmetric region 110 corresponds to or is similar to a predefined point-symmetric region from one of the above figures. The center of symmetry 112 of the predefined point-symmetric region 110 is also shown.

[0170] Figure 25 A schematic diagram of a carrier medium 600 having a pattern 610 according to an embodiment is shown. An exemplary arrangement is shown using a firmly clamped elastic fabric as the carrier medium 600, the elastic fabric having a point-symmetric pattern 610, for example, on the underside. When an object is thrown back by the elastic fabric, the pattern 610 is observed by at least one camera 102. This arrangement allows for continuous, accurate, and complete recording of the temporal dynamic deformation of the elastic fabric in a non-contact manner. A predefined point-symmetric region 110 corresponds to or is similar to a predefined point-symmetric region from one of the above figures. The center of symmetry 112 of the predefined point-symmetric region 110 is also shown. Furthermore, an illumination device 2502 is arranged to illuminate the field of view of the camera 102.

[0171] The following is for reference. Figure 24 and Figure 25 Especially in combination Figures 1 to 5 The application areas of deformation / vibration analysis and force measurement utilizing point-symmetric patterns are discussed in more detail. In this case, a symmetrical pattern 610 is used to determine the shape of a surface and, in particular, the deformation of the surface. For this purpose, the surface to be observed, or the carrier medium 600, is made as flat as possible, for example, covered with a plastic or paper film or a fabric on which the pattern 610 is previously printed. Alternatively, the carrier medium 600 may also have the pattern 610 printed directly on it. The pattern 610 can also be applied to the surface of the carrier medium 600 by engraving, etching, embossing, or exposure. The surface whose shape or deformation should be determined can be composed of different materials. It can be, for example, a component made of metal sheet, plastic, fiber-reinforced plastic, glass, rubber, film, synthetic fiber fabric, leather, and other materials. In particular, the surface of the component is relevant, for example, in various industrial or technical applications as well as in vehicle technology, aircraft technology, marine and space technology.

[0172] The following describes some application examples from the fields of deformation analysis and vibration analysis.

[0173] In road vehicle crash tests, collisions between vehicles and obstacles or other vehicles are observed to assess safety for occupants involved in the accident and to draw conclusions for improving vehicle construction. The collision process is typically observed under strong light using high-speed cameras inside and outside the vehicle. Trained experts interpret the changes in the collision process based on the image recordings and other data. In particular, they compare the observed results with those from computer simulations. Vehicle body sections are covered with markings that allow for the measurement of certain lengths based on video images before, during, and after the collision. Vehicle windows are typically untreated and are transparent.

[0174] According to one embodiment, the vehicle body, particularly on the exterior (and partly in the interior space if necessary), is covered, for example, with a plastic film, on which point-symmetric regions 110 are applied as a pattern 610. Similar patterns 610 can also be applied to tires and rims, as well as the underside of the vehicle body. When needed, vehicle windows and sunroofs can also be patterned with the pattern 610 in this manner. To ensure the vehicle windows remain at least translucent, allowing observation of the model and interior space from the outside, a less densely printed transparent medium can also be used for the cover. To prevent the film from affecting the glass's breakage behavior, the film can be removed again (warmly if necessary), leaving an imprint on the vehicle. Due to the multidimensional curvature of the body parts, the cover should be elastic during application. This may distort the original pattern. This does not pose a problem for the method, as point symmetry is preserved, for example, when the film is stretched in a directional manner. However, it is recommended to first perform a measurement process for calibration after pattern application, whereby at least the locations on the surface where the center of symmetry 110 lies precisely are determined. For this purpose, the above-described... Figure 3 The method or similar approach used to provide this information. This can be aided by using CAD data of the vehicle or additional measurement technologies (e.g., stereo cameras, multi-camera systems, laser scanners, etc.). Subsequent measurements will be more accurate if the entire texture is additionally captured and registered during the measurement process, for example, in a photographic manner. If the corresponding technology is available, the pattern 610 or the dot-symmetric region 110 can also be applied by using a printing robot that guides the print head on the vehicle body surface under CAD control.

[0175] For collision analysis, the location of the center of symmetry 112 can be precisely determined in each individual image of the image sequence. Since each symmetric region 110 is typically observed simultaneously by multiple cameras and the cameras constructed in external space are fixed, the corresponding current 3D position of the respective center of symmetry 112 can be determined by triangulation. Figure 3The method used to provide the search for the center of symmetry 112 requires very little computational effort, thus allowing the method to be implemented in real time with appropriate computer hardware, eliminating the need to wait for evaluation. If necessary, the impact on the time flow can even be adjusted to, for example, protect upper-level components from unnecessary further damage. When assigning the center of symmetry 110 from image to image, particularly for triangulation, it is helpful to encode the pattern 610, i.e., the alternation between odd-point and even-point symmetries existing in region 110. The assigned correspondence can only be correct if all the resulting correspondences have the same sign. This can be used for verification. Additionally or alternatively, well-known epipolar geometry can be used to form the correspondences because it restricts the knowledge space: thus the two-dimensional assignment problem becomes one-dimensional.

[0176] After determining the 2D or 3D position of the observed point symmetry center 112, a high-resolution temporal analysis of the deformation of the vehicle body, glass, and accessories can be performed. This analysis is more accurate and objective than methods available to date, and partially alleviates the burden of visual interpretation by experts. It offers advantages in accuracy and rich detail for comparisons between crash tests and computer simulations, and particularly for conclusions regarding improvements to vehicle structures. If the 2D or 3D positions are determined not only for the symmetry center 112 but also for all points on the patterned surface, the local resolution can be further increased if needed. For this purpose, a high-resolution temporal analysis of the deformation of the vehicle body, glass, and accessories can be performed. Figure 3 The method provided uses a known pattern as a reference and determines numerous correspondences between the real image record and the reference. Another possibility for increasing local resolution is derived from stereo vision. Stereo vision works particularly well with patterns 610 containing random characters (e.g., noisy patterns, as can be seen in the figures above), i.e., it has high availability and is particularly insensitive to system errors and errors caused by noise.

[0177] According to embodiments, at least one pattern 610 or at least one dot-symmetric region 110 can also be used to monitor or analyze the shrinkage process of films, packaging, and deep-drawn components. Shrink films are used in many areas of the packaging industry. For example, they are placed on bottles as printed film tubes and heated, where the shrink film adapts to the shape of the bottle. They are also used to seal containers or as printed sleeves around actual closures. Shrink films of greater or lesser thickness are also used to produce packaging for single-use items by deep drawing from flat film. In addition, tubular preforms with one open end are used to manufacture plastic bottles by heating and inflating. Many other objects or components are produced by deep drawing or inflating plastic (or other materials), where the plastic is, for example, heated and stretched or pressed or blown into a mold. In all these products, it is desirable for the material to shrink or stretch or blow-mold so that it takes on the desired shape. During printing, it is important that the text and images do not become excessively distorted after deep drawing. Furthermore, it is generally important that the plastic is properly distributed, i.e., the plastic is not too thin to avoid tearing, and the plastic is only thick where it must withstand the corresponding force.

[0178] According to embodiments, starting materials such as shrink films, deep-drawn films, preforms, etc., can be or will be printed as a carrier medium 600 with a symmetrical pattern 610. Subsequently, if necessary, one or more reference records can be made for at least one reference image, for example, by a camera or scanner. Further records or camera images are made after the shrink, deep-draw, or blow molding process is completed. By performing... Figure 3 The provided methods or similar methods are used to evaluate these further recordings or camera images. After establishing correspondences between the centers of symmetry 112 (front / back), it can be determined how the film deforms, and locally for each individual center of symmetry 112. Thus, it is possible to determine with high accuracy how the film deforms during the process. Optionally, the time flow of the process can also be observed by continuously recording not only at the end but also during shrinkage, deep drawing, or blow molding, and the recordings can then be evaluated to capture the dynamic flow. This knowledge can then be used, for example, to better control the local distribution of film heating, optimize the local distribution of mold heating, and / or influence the time flow, for example by controlling ambient temperature, cooling fans, forces, paths, etc., or to optimize the materials used. Machine learning methods are well-suited for optimizing process control by evaluating the meaning of observations through parameter feedback. A two-step process including a calculation step can also be used if higher position resolution or accuracy is desired.

[0179] According to one embodiment, at least one pattern 610 or at least one point-symmetric region 110 can also be used to monitor or analyze vibrations of the machine housing, body, etc., which serve as the carrier medium 600. Machine components (e.g., the housing of a machine or the body of a passenger car, truck, locomotive, or van) and many other technical objects or elements may tend to vibrate during operation, which can lead to noise or noise pollution and premature wear, such as failure due to sustained load or material fatigue. Large, thin-walled metal sheets are particularly susceptible. They may emit disruptive noise and / or fail prematurely at heavily loaded locations. Such undesirable vibrations can generally be suppressed by simple measures, such as inserting kinks or applying reinforcements or damping materials at appropriate locations.

[0180] According to one embodiment, for this purpose, the vibrating component as the carrier medium 600 is observed in the installed state and under a given vibration excitation before and after the corresponding correction measures. Figure 3 The methods and / or methods provided Figure 4 The control method is suitable for this purpose. The component or carrier medium 600 to be inspected is pasted, covered, or printed with a pattern 610 and observed during operation by a camera 102 that is fixed or moves with it. The centers of symmetry 112 are found, and the temporal changes in their positions can be used to infer the amplitude and frequency at the corresponding locations, as well as the currently existing vibration modes. If higher position resolution or accuracy is desired, a two-step process including a determination step can also be used if needed. The success of which measures are in reducing vibration or vibration modes can then be accurately inferred from the observations or deformation analysis.

[0181] Figure 3 The methods and / or methods provided Figure 4 The method used for control offers particular advantages here: the entire (oscillating) surface can be captured using a single sensor (camera 102). Therefore, this method provides high-dimensional results from which complex higher-order vibration modes can be read. The analysis results can also be easily explained to non-experts because of their graphical nature. No measuring instruments, such as strain gauges, accelerometers, or displacement sensors, are required on the object or carrier medium 600 to be measured, and no cabling is necessary. The object or carrier medium 600 to be measured is almost unaffected by the measurement because the applied pattern 610 hardly changes the mass distribution, while the installation of the sensor (point mass) alters the oscillation mode.

[0182] According to embodiments, at least one pattern 610 or at least one point-symmetric region 110 can also be used to capture or analyze the dynamic deformation of the wind turbine, components in a wind tunnel, or other carrier medium 600 during operation. Some machines or components are too large or too inaccessible for measuring their dynamic deformation during operation using alternative or conventional methods (e.g., acceleration or displacement sensors) may be difficult or even impossible. For example, it may be interesting to study how the rotor blades of a wind turbine, as carrier medium 600, behave during storms, i.e., whether torsional vibrations occur, for example, as the blades move past the tower, or how the tower moves under specific loads. From these observations, one can infer, for example, whether or under what wind conditions the wind turbine should be shut down to avoid structural damage, whether the rotor blades have been damaged due to vibrations different from specifications, or how or at what locations on the rotor blades and tower the structure can be modified to, for example, save material at lower-load locations to facilitate larger-load locations. For this purpose, the rotor blades and / or tower are patterned 610 before assembly. One or more cameras 102, installed at appropriate distances, observe the operating wind turbine. Camera 102 can preferably be mounted on the tower of an adjacent wind turbine in the same wind farm, because the camera is particularly easy to network and be powered there, and because the camera can be mounted at an appropriate height there. Unlike the previous example, the observed center of symmetry 112 moves along a circular track on the corresponding rotor blade. Due to the rotational invariance of point symmetry, the center of symmetry 112 can be easily found in a series of camera images, and their movement can be tracked over time. Ideally—when the rotor blades are not vibrating—the corresponding circular track is undisturbed. The vibration of the corresponding rotor blade can be inferred from the large number of observed deviations from the ideal circular movement. The possibly unavoidable small inherent movement of the camera relative to the object being observed does not pose a problem here, because pattern 610 brings its own reference system due to its symmetry center 112, which exists in any way, and because what is of interest is the measurement of the deformation between these centers of symmetry 112, rather than the measurement of the movement relative to the camera.

[0183] This principle can be transferred to other applications, such as the observation of components in a wind tunnel, such as aircraft wings, tail units, racing car spoilers, and air deflectors, where pattern 610 is applied flatly to the corresponding component as a carrier medium 600, and the dynamic deformation or movement of the component is captured by means of one or more observation cameras 102. Here, the inherent vibrations of the component (e.g., torsional vibrations) are particularly of interest, as inherent vibrations can accumulate and potentially damage the component. See also Figure 24Furthermore, another example of its application is the observation of components in water or other media (via camera 102 located in the same medium). Therefore, in particular, it is possible to observe ship propellers, power plant turbines, pumps, aircraft propellers, etc., during operation.

[0184] For example, the analysis of the shape and load of the sail, which serves as the carrier medium 600, can also be achieved in the manner described above. In the sails of, for example, future environmentally friendly transport ships, fast-moving vessels, or rooftops—where the sail is printed with a pattern 610 and observed by at least one camera 102—it can be used during dynamic wind loading. Figure 3 and / or Figure 4 The method continuously determines the three-dimensional shape. Various conclusions can be drawn from this, and it can be used in different ways, such as for identifying overloads based on strong tension to protect the sail in time, for optimizing the angle of attack of the sail or sail profile, for example by automatically changing the tension of the line, or for identifying and avoiding sail vibrations and flapping.

[0185] According to an embodiment, at least one pattern 610 or at least one point-symmetric region 110 can also be used to achieve dynamic measurement of weight and force. Figure 3 The methods and / or methods provided Figure 4 The methods used for control are flexible and cannot be exhaustively listed here. Another application example should illustrate this point. See [link to relevant documentation] for details. Figure 25 An elastic fabric (e.g., made of a rubber film) is used as a carrier medium 600. A pattern 610 with numerous small dot-symmetric regions 110 is printed on the fabric, and the fabric is clamped around its edges, causing it to be stretched like the membrane of a drum or the jumping surface of a trampoline. This "drum" is fixed, as is at least one camera 102, which observes the pattern 610 on the membrane, preferably from an oblique angle. An object falls onto the membrane or carrier medium 600 from another oblique incident direction (e.g., from a conveyor belt), is reflected by the elastic membrane, and continues its flight. The dynamic process of the object's reflection on the membrane with the pattern 610 is observed and utilized by a camera 102 that operates at a sufficiently high speed. Figure 3 The methods and / or methods provided Figure 4 The methods used for control are evaluated.

[0186] Pattern 610 can be mounted on the front or back of camera 102, or both, wherein at least one camera 102 observes one or more corresponding sides. It is advantageous to observe the back side with pattern 610 on it because pattern 610 is not obscured by objects, pattern 610 will not be worn, pattern 610 and camera 102 can be better protected from contamination, and pattern 610 can be more easily illuminated because there are no objects casting shadows, etc.

[0187] For example, the following information can be obtained from observing a carrier medium 600 in the form of an elastic membrane: determining the force acting on the membrane or the dynamic force change process based on deformation. Thus, if the fall height is known, the mass of the object can be inferred; if the mass is known, the fall height can be inferred. Objects can be counted. The further flight trajectory of the corresponding objects can be predicted. The flight trajectory of objects can be dynamically influenced by rapidly changing the orientation or tension of the membrane, for example, in systems used to classify and distribute different objects falling from a conveyor belt.

[0188] This idea can be easily adapted and transferred to other applications, such as mattress workshops, where the described pattern 610 is printed on the test mattress itself or the sheet on the test mattress as a carrier medium 600. When a customer tries the mattress, the unobstructed portion of the pattern 610 is recorded by a camera to determine the penetration depth, and if necessary, to determine the person's mass distribution, thereby determining the optimal mattress and slat frame recommendation or enabling product personalization based on the obtained data.

[0189] If an embodiment includes an "and / or" link between a first feature and a second feature, it should be understood that the embodiment has both the first feature and the second feature according to one implementation, and either only the first feature or only the second feature according to another implementation.

Claims

1. A method (300) for providing deformation data (135) for deformation analysis, wherein the method (300) comprises the following steps: Image data (105) provided by the camera (102) is read (324) from the interface (122) to the camera (102), wherein the image data (105) represents a camera image of at least one predefined even and / or odd point symmetric region (110; 110A, 110B) in the environment of the camera (102), wherein the at least one predefined point symmetric region (110; 110A, 110B) is generated on, at, or in the deformable carrier medium (600); The image data (105) and determination rules (128) are used to determine (326) at least one center of symmetry (112; 112A, 112B) of the at least one even and / or odd point symmetric region (110; 110A, 110B). The position of the at least one center of symmetry (112; 112A, 112B) in the camera image is compared (330) with the predefined position of at least one reference center of symmetry in the reference image (115) relative to the reference coordinate system to determine the positional deviation (131) between the center of symmetry (112; 112A, 112B) and the reference center of symmetry; and / or The positional deviation (131) is used to obtain (332) displacement information (133) of at least one subset of pixels of the camera image relative to the corresponding pixels of the reference image (115), wherein the positional deviation (131) and / or the displacement information (133) are used to provide the deformation data (135).

2. The method (300) according to claim 1, wherein the determination rule (128) used in the determination step (326) is constructed such that A signature(s) is generated for multiple pixels of at least one segment of the camera image to obtain multiple signature(s), wherein each signature(s) is generated using a descriptor having multiple different filters, wherein each filter has at least one symmetric type, and wherein each signature(s) has a symbol for each filter of the descriptor. For the signature(s), at least one mirror signature (S) of at least one symmetric type is obtained for the filter. PG S PU ), Check whether the pixel having the signature(s) exists in at least one other pixel in the search area (1104) of the environment surrounding the pixel, the at least one other pixel having a corresponding mirror signature(s). PG S PU The signature(s) is used to obtain the pixel coordinates of at least one symmetric signature pair from the pixel and the other pixel when at least one other pixel is present. And evaluate the pixel coordinates of the at least one symmetric signature pair to identify the at least one center of symmetry (112; 112A, 112B). And / or at least one reflector (R) PG R PU The symbol applied to one of the signatures(s) is used to obtain the at least one mirror signature(S). PG S PU ), where each reflector (R) PG R PU ) has rules for modifying the symbol that are specific to the symmetry type and depend on the descriptor, wherein the search region (1104) depends on the applied reflector (R PG R PU At least one reflector in ().

3. The method (300) according to claim 2, wherein in the determining step (326), for each determined center of symmetry (112; 112A, 112B), using the pixel coordinates of each symmetry signature pair that has contributed to correctly identifying the center of symmetry (112; 112A, 112B), a transformation rule is determined for transforming the pixel coordinates of the center of symmetry (112; 112A, 112B) and / or the at least one even and / or odd point symmetry region (110; 110A, 110B), wherein the transformation rule is applied to the pixel coordinates of the center of symmetry (112; 112A, 112B) and / or the at least one even and / or odd point symmetry region (110; 110A, 110B) to correct the distorted viewpoint of the camera image.

4. The method (300) according to any one of the preceding claims, wherein in the determining step (326), the symmetry type of the at least one symmetry center (112; 112A, 112B) is determined, wherein the symmetry type represents even-point symmetry and / or odd-point symmetry, and / or in the comparing step (330), the symmetry type of the at least one symmetry center (112; 112A, 112B) in the camera image is compared with a predefined symmetry type of at least one reference symmetry center in the reference image (115) to check the consistency between the at least one symmetry center (112; 112A, 112B) and the at least one reference symmetry center.

5. The method (300) according to claim 4, wherein the image data (105) read in the reading step (324) represents a camera image of at least one pattern (610; 1710, 1810) consisting of a plurality of predefined even and / or odd point symmetric regions (110; 110A, 110B), wherein the geometric arrangement of the symmetry centers (112; 112A, 112B) of the at least one pattern (610; 1710, 1810) is determined in the determining step (326), the geometric sequence of the symmetry type of the symmetry centers (112; 112A, 112B) is determined, and / or the pattern (610; 1710, 1810) is determined from a plurality of predefined patterns using the sequence, wherein the arrangement and / or the sequence represents an identifier of the pattern (610; 1710, 1810).

6. The method (300) of claim 5, wherein in the determining step (326), the arrangement of the centers of symmetry (112; 112A, 112B) of the at least one pattern (610; 1710, 1810) and / or the sequence of the symmetry types of the centers of symmetry (112; 112A, 112B) are used to determine implicit additional information of the at least one pattern (610; 1710, 1810) or readout rules for reading explicit additional information in the camera image, wherein the arrangement and / or the sequence represent the additional information in coded form, wherein the additional information is related to the deformation analysis.

7. The method (300) according to any one of claims 5 to 6, wherein in the comparison step (330), the reference image (115) is selected from a plurality of stored reference images according to the determined arrangement, the determined sequence and / or the determined pattern (610; 1710, 1810) or the reference image (115) is generated using stored generation rules.

8. The method (300) according to any one of claims 5 to 6, wherein the determination step (326) and / or comparison step (330) are performed jointly for all centers of symmetry (112; 112A, 112B) independently of the symmetry type of the centers of symmetry (112; 112A, 112B), or the determination step (326) and / or comparison step (330) are performed individually for centers of symmetry (112; 112A, 112B) of the same symmetry type according to the symmetry type of the centers of symmetry (112; 112A, 112B).

9. A method (400) for controlling deformation analysis, wherein the method (400) comprises the following steps: The evaluation (444) of the deformation data (135) provided by the method (300) according to any one of the preceding claims is used to generate a control signal (145) dependent on the deformation data (135); and The control signal (145) is output to the interface (148) of the device used to perform the deformation analysis to control the deformation analysis.

10. A method (500) for manufacturing at least one predefined even- and / or odd-point symmetric region (110; 110A, 110B) for use in the method (300; 400) according to any one of the preceding claims, wherein the method (500) comprises the following steps: Generate (502) design data (204), the design data representing a graphical representation of the at least one predefined even and / or odd point symmetric region (110; 110A, 110B); and Using the design data (204), at the carrier medium (600) or in the carrier medium (600), generate (506) the at least one predefined even and / or odd point symmetric region (110; 110A, 110B) to manufacture the at least one predefined even and / or odd point symmetric region (110; 110A, 110B).

11. The method (500) according to claim 10, wherein design data (204) is generated in the generation step (502), the design data being a graphical representation of the at least one predefined even and / or odd point symmetric region (110; 110A, 110B) as a circle, ellipse, square, rectangle, pentagon, hexagon, polygon, or annulus, wherein the at least one predefined even and / or odd point symmetric region (110; 110A, 110B) has a regular or quasi-random content pattern, and / or wherein any predefined even and / or odd point symmetric region (110; 110A, 110B) is pre-given. The first half of the carrier medium (110B) is constructed by mirroring the dots and / or reversing the grayscale values ​​and / or color values, and / or wherein the at least one predefined even and / or odd point symmetrical region (110; 110A, 110B) is generated in the generation step (506) by additive manufacturing process, separation, coating, shaping, primary forming or optical display, and / or wherein the carrier medium (600) has fabric, rubber, film, paint, sheet metal, metal, wood, plywood, plastic, fiber reinforced plastic, paper, cardboard, composite material, coated material, leather, liquid, glass, stone, ceramic, concrete, plaster or food.

12. The method (500) according to any one of claims 10 to 11, wherein design data (204) is generated in the generation step (502), the design data representing a graphical representation of at least one pattern (610; 1710, 1810) consisting of a plurality of predefined even and / or odd point symmetric regions (100; 110A, 110B), wherein at least one subset of the point symmetric regions (100; 110A, 110B) is aligned on a regular or irregular grid (1311), directly adjacent to each other and / or separated from at least one adjacent even and / or odd point symmetric region (110; 110A, 110B) by gap portions, and is the same or different from each other in terms of their size and / or their content patterns and / or arranged in a common plane or in different planes, wherein the design data (204) representing a graphical representation of at least one pattern (610; 1710, 1810) having layered symmetry is generated in the generation step (502).

13. An apparatus (120) for providing deformation data (135) for deformation analysis, configured to perform and / or manipulate the steps of the method (300) according to any one of claims 1 to 8 in corresponding units (124, 126, 130, 132).

14. An apparatus (140) for controlling deformation analysis, configured to perform and / or manipulate the steps of the method (400) according to claim 9 in corresponding units (144, 146).

15. An apparatus (200) for manufacturing at least one predefined even and / or odd point symmetric region (110; 110A, 110B), configured to perform and / or manipulate the steps of the method (500) according to any one of claims 10 to 12 in corresponding units (202, 206).

16. A computer program product configured to perform and / or manipulate the steps of the method (300; 400; 500) according to any one of claims 1 to 12.

17. A machine-readable storage medium having a computer program product according to claim 16 stored thereon.

Images