Localization system and localization method with markers having holographic optical elements

Abstract

The invention discloses a concept for improved position and orientation determination of a plurality of objects each marked with at least one optoelectronically detectable marker, in which holographic optical elements are introduced into two-dimensional flat markers, at which ambient light is reflected back with sufficiently high intensity even at high scattering angles relative to the surface normal of the two-dimensional flat markers.For this purpose, it is proposed to detect light reflected by holographic optical elements (HOE; 21, 22, 23) applied to the markers (20) in an optoelectronic sensor (32) and to generate a two-dimensional reflection image, and to detect, in an image processing device (34), on the basis of the spectrally resolved local intensity distribution of the two-dimensional reflection image detected by the optoelectronic sensor (32), differences between the reflection behavior of the optoelectronically detectable markers (20) of the current position and orientation of the plurality of objects (100; 200) and the reflection behavior of the optoelectronically detectable markers (20) in a reference configuration of the plurality of objects (100; 200), and to detect deviations in the overall configuration of the plurality of objects (100; 200) relative to the optoelectronic detection device (30) by evaluating the detected differences. determine.

Description

Technical FieldThe present invention relates to a localization system and a localization method which is based on angle-dependent reflection measurements of holographically optical elements as part of markers to be localized.Prior ArtDifferent positioning methods can be used for determining the position and / or orientation of objects. Optical object recognition, for example, frequently uses geometric structures of high contrast, which can be recognized in a simple, accurate and reliable reproducible manner. Examples of these are optoelectronically detectable color-contrasted markings which consist of differently dimensioned geometric symbols or shapes, such as lines or polygons, for example, and gaps lying therebetween, so that the luminance difference at the boundaries between shapes and gaps is as high as possible. Such markings can have a one-dimensional coding (for example bar codes or bar codes) or a two-dimensional coding (for example data matrix codes, QR codes or Ace codes), each with a different payload density. The data encoded in a marker can be read mechanically using optical readers, such as cameras or camera scanners, for example, and can be further processed electronically.Markers can be equipped with holographic optical elements (HOE) to increase the information density. Examples of such markers are disclosed, for example, in the publications US 2021 / 0248338 A1, PT 99405 A, CZ 2004-1136 A3 and U.S. Pat. No. 10 885 413 B1.Disclosure of the InventionThe present invention provides a localization system and a localization method, each having the features of the independent patent claims. Further advantageous embodiments are the subject matter of the dependent claims.A localization system for determining the position and / or orientation of a plurality of objects identified by at least one optoelectronically detectable marker in each case comprises an optoelectronic detection device. This has an optoelectronic sensor and an image processing device coupled to the optoelectronic sensor. The optoelectronic sensor is designed to detect light originating from holographically optical elements applied to the markers and reflected back onto the optoelectronic sensor by the holographic optical element and to generate a two-dimensional reflection image. The image processing device is designed to detect, on the basis of the spectrally resolved local intensity distribution of the two-dimensional reflection image captured by the optoelectronic sensor, differences between the reflection behavior of the optoelectronically detectable markers of the current position and orientation of the plurality of objects and the reflection behavior of a second of the optoelectronically detectable markers in a reference configuration of the two-dimensional reflection image relating to the plurality of objects and to determine deviations in the overall configuration of the plurality of objects relative to the optoelectronic detection device by evaluating the detected differences.A localization method for determining the position and orientation of a plurality of objects identified by at least one optoelectronically detectable marker in each case comprises the steps of: detecting light emanating from holographically optical elements applied to the markers by means of an optoelectronic sensor of an optoelectronic detection device, wherein the light is reflected back onto the optoelectronic sensor by the holographic optical element; generating a two-dimensional reflection image by the optoelectronic sensor from the detected reflected light; detecting differences between the reflection behavior of the optoelectronically detectable markers of the current position and orientation of the plurality of objects and the reflection behavior of the optoelectronically detectable markers in a reference configuration of the two-dimensional reflection image detected by the optoelectronic sensor that is different from the plurality of objects on the basis of the spectrally resolved local intensity distribution in the two-dimensional reflection image detected by the optoelectronic sensor; and determining deviations in the overall configuration of the plurality of objects relative to the optoelectronic detection device by evaluating the detected differences.Holographic optical elements (HOEs) are optically active structures whose functional principle is based on holography. In the case of an HOE, the imaging properties of a hologram are utilized or a conventional optical system is realized by a hologram, so that light waves can be deflected, divided, focused and / or widened. HOEs can also be formed, for example, as extremely thin film foils which can split incident light into its spectral colors. However, only the light beams that are incident within a specific wavelength range are diffracted in this case. By using HOEs, diffraction maxima can be generated which arise due to constructive interference of light waves at the HOEs.In contrast to conventional optics, in HOEs, a deflection of the light is realized not by refraction, but by diffraction at the volume grating. In this case, the HOEs can be manufactured both for transmission and for reflection and make new designs possible by means of a free selection of angle of incidence and angle of reflection or angle of diffraction. The holographic diffraction grating is exposed, for example, into a thin, photosensitive film. As a result of the volume diffraction, a characteristic wavelength and angle selectivity or else filter function can additionally be assigned to an HOE. Depending on the recording condition, for example a wavelength or an angle, only light from defined directions and with defined wavelengths is diffracted at the structure of the HOE.Advantages of the InventionAccording to some embodiments of the localization system and the localization method, the optoelectronic detection device can furthermore have a memory, which is coupled to the image processing device and stores a multiplicity of comparison and / or training data for the detected two-dimensional reflection images in a database. This enables a quick and reliably reproducible localization of the objects from known reference data.According to various embodiments of the localization system and of the localization method, the image processing device can furthermore perform object recognition of structures of a contrast code of the markers recorded in the two-dimensional reflection images captured by the optoelectronic sensor. The combination of contrast codes such as QR codes with holographic optical elements enables the implementation of redundant detection mechanisms, which improves robustness with respect to interference sources, reduces susceptibility to incorrect determinations and increases the accuracy of the position and distance determination.According to several embodiments of the localization system and method, at least one of the holographic optical elements applied to the markers may include a holographic deflector with high angular selectivity at low elevation angles with respect to the surface of the marker. This enables the implementation of a warning function in the case of inadequate detectable effective area of the marker in unfavorable orientations.According to some embodiments of the localization system and the localization method, the color depth of the optoelectronic sensor may be at least 8 bits. This advantageously increases the spatial and angular resolution of the position and alignment determination.According to some embodiments of the localization system and the localization method, at least one of the holographic optical elements applied to the markers can have an optical function, such that a changing distance of the detection device from the objects is indicated by a color change and / or a change in the intensity distribution in the two-dimensional reflection images detected by the optoelectronic sensor. As a result, a distance determination, and thus a determination of even very small deviations in the overall configuration, can be carried out more precisely.According to some embodiments of the localization system and the localization method, at least one of the holographic optical elements applied to the markers can have an optical function, such that a changing object orientation for dedicated angle ranges is indicated by a color change and / or a change in the intensity distribution in the two-dimensional reflection images acquired by the optoelectronic sensor. As a result, a determination of the object orientation, and thus a determination of even the smallest deviation in the overall configuration, can be carried out more precisely.According to some embodiments of the localization system and the localization method, at least two of the holographic optical elements applied to the markers may have duplicated optical functions for improving the reliability of a determination of the distance of the detection device from the objects and / or a determination of the object orientation for dedicated angular ranges. This advantageously enables a particularly robust determination of the distance and object orientation, since the evaluated back reflections of the at least two holographically optical elements can be checked for plausibility with respect to one another.The above embodiments and developments can be combined with one another as desired, insofar as appropriate. Further embodiments, developments and implementations of the invention also include combinations of features of the invention described above or below with respect to the exemplary embodiments, which combinations are not explicitly mentioned. In particular, the person skilled in the art will also add individual aspects as improvements or additions to the respective basic forms of the invention.Brief Description of the DrawingsFurther features and advantages of the invention are explained below with reference to the figures. The following are shown: FIG. 1 : an exemplary illustration of an optoelectronically detectable marker in the form of a QR code; FIG. 2 : shows a schematic illustration of an optoelectronically detectable marker in the form of a QR code with holographic optical elements superimposed on the coding, according to one embodiment; FIG. 3 : shows a schematic illustration of the optoelectronic detection of the marker of FIG. 2 with the aid of a localization system according to one embodiment; FIG. 4 : an exemplary diagram of a two-dimensionally resolved optoelectronic measurement signal distribution of the holographic optical elements of the marker of FIG. 2 in a localization system of FIG. 3 ; FIG. 5 : shows a further exemplary diagram of a two-dimensionally resolved optoelectronic measurement signal distribution of the holographic optical elements of the marker of FIG. 2 in a localization system of FIG. 3 ; FIG. 6 : shows a flow diagram of a localization method for the time-resolved location and position determination of an object identified by a plurality of optoelectronically detectable markers according to one embodiment; FIG. 7 : exemplary structures for redundant holographic optical elements on an optoelectronically detectable marker according to one embodiment; FIG. 8 : shows a schematic illustration of an optoelectronically detectable marker in the form of a QR code of holographic optical elements superimposed with the coding and a holographic deflector according to a further embodiment; FIG. 9 : shows a schematic illustration of an optoelectronically detectable marker in the form of a QR code with holographic optical elements superimposed on the coding according to a further embodiment; FIG. 10 : shows a schematic plot of the reflection intensity maxima along the distance from the optoelectronically detectable marker of FIG. 9 ; FIG. 11 : shows a schematic plot of the frequency-dependent reflection intensity along the distance from the optoelectronically detectable marker of FIG. 9 ; FIG. 12 : shows a schematically illustrated diagram of an arrangement of objects marked with optoelectronically detectable markers and of an associated optoelectronic detection device according to one embodiment; and FIG. 13 : shows a schematically illustrated diagram of an arrangement of objects marked with optoelectronically detectable markers and of an associated optoelectronic detection device according to one embodiment;DESCRIPTION OF EMBODIMENTSFIG. 1 shows an exemplary illustration of an optoelectronically detectable marker in the form of a QR code 10. The polygons are contrasted by optoelectronically detectable colors with gaps lying between them, so that the luminance difference at the boundaries between shapes and gaps is as high as possible. Such markings can have a one-dimensional coding (for example bar codes or bar codes) or a two-dimensional coding (for example data matrix codes, QR codes or Ace codes), each with a different useful information density. The data encoded in such a marker can be read mechanically and processed electronically using optical readers, such as cameras or camera scanners.A location and / or position determination of objects which have been marked by such optoelectronically detectable markers can be error-prone and often inaccurate. The accuracy of the location and / or position determination often varies with the distance of the marked object and with the illumination conditions in the respective environment. The determination of the angular orientation, i.e. the spatial orientation with respect to the optical reading device, can likewise be greatly impaired. As the distances become greater, optoelectronic detection devices with a higher resolution may be required. The recognition and localization of the patterns on such markers requires the use of complex object recognition algorithms that require a high computational capacity.By using markers equipped with holographic optical elements (HOEs), the localization of the objects marked therewith can be improved. In particular, but not exclusively, the position detection can be significantly improved by an optimized angle orientation determination. Better angular resolution, potentially better spatial resolution, lower costs due to less good camera resolution, computationally favorable and fast determination of the position, and possible extensions of existing localization solutions can be achieved by using optoelectronically detectable markers in the form of one- or two-dimensional planar contrast codes, such as bar codes, bar codes, data matrix codes, QR codes or Ace codes, holographically optical elements (HOE) superimposed with the code.FIG. 2 shows an example of such an optoelectronically detectable marker 20 with a contrast coding, here a QR code 10. The number, size, shape, type and position of the HOEs 21, 22 and 23 is shown in FIG. 2 only by way of example. It is possible to provide more or less than three HOEs 21, 22 and 23 with a different size, shape and / or position on the marker 20.The optical function of a hologram arises by diffraction of incident optical wavefronts at a diffraction grating in the volume of the holographic film. The diffraction grating was previously exposed in a recording process into the volume of the holographic film. The diffraction grating is formed in accordance with an interference pattern generated by two optical recording wavefronts as a modulation of the refractive index in the holographic film. The interference pattern is generated by the superposition of two coherent recording waves in the volume of the holographic film. The interference of the two recording wavefronts forms in the volume of the holographic film an interference grating which is permanently impressed into the volume of the holographic film by the exposure process. The parameters of the recording process (wavefront shape, spectral properties, intensity distribution and duration) determine the properties of the diffraction grating produced. HOEs can be produced, for example, as hologram structures by holographic wavefront printing, as taught, for example, in the publication DE 10 2017 218 544 A1.FIG. 3 shows a schematic illustration of the optoelectronic detection of the marker 20 of FIG. 2 with the aid of a localization system 1. the localization system 1 comprises an optoelectronic detection device 30, such as a video camera, a camera scanner or a similar device. Reflected ambient light emitted by the marker 20 and in particular by the HOEs 21, 22 and 23 is recorded via a suitable detection optical unit 31 and fed to an optoelectronic sensor 32. The optoelectronic sensor 32 may be, for example, a CCD sensor, a CMOS sensor, an active pixel sensor (APS), a photodiode array, or another suitable electromagnetic radiation detector with spatial resolution.The image captured by the optoelectronic sensor 32 is transferred to an image processing device 34, which carries out an evaluation of the captured image and transfers the evaluation result to an output interface 35 of the capturing device 35. The image processing device 34 may be, for example, a microprocessor, a graphics processor, an ASIC, an FPGA, or any other suitable computing device. The output interface 35 may be, for example, a display or another user interface. It may also be possible for the output interface 35 to establish a communication connection to a further device to which the evaluation result of the image processing device 34 is transmitted via the output interface 35.The optoelectronic detection device 30 can also have a memory 33, such as a ROM, a RAM or another type of data memory, in which, on the one hand, configuration parameters and operating software for the operation of the detection device 35 and in particular of the image processing device 34 are stored. Secondly, comparison and / or training data can be stored in a database in the memory 33, which database allows the image captured by the optoelectronic sensor 32 to be matched with known images or optoelectronic capture signatures of markers in different positions, orientations and / or positions.The optoelectronic detection device 30 serves on the one hand for the general identification and localization of an object marked with a marker 20 on the basis of contrast coding, such as the QR code 10, for example, object detection of the structures of the QR code 10 recorded in the image detected by the optoelectronic sensor 32 is carried out in the image processing device 34 and a comparison with known QR codes is carried out, for example by comparison with known QR codes stored in a database of the memory 33. Subsequently, by determining the size, spacing and mutual orientation of the patterns and structures of the QR code 10, it is possible to draw conclusions as to the orientation in space.Additionally or alternatively, the image captured by the optoelectronic sensor 32 may receive back-reflected light from the HOEs 21, 22, and 23. In addition to the geometric position of the structure of the HOEs 21, 22 and 23, the position and orientation of the object relative to the optoelectronic detection device 30 can be encoded by the color of the backscattered light. FIGS. 4 and 5 show two exemplary diagrams of two-dimensionally resolved optoelectronic measurement signal distributions P 1 and P 2 of the HOEs 21, 22 and 23 of the marker 20 of FIG. 2 by an optoelectronic detection device 30 of the localization system 1 of FIG. 3.HOEs are structures which are written into a photo-optical layer and redirect the incident light. Depending on how the structure of the HOE is designed, the optical function can be determined, for example a scattering hologram, a retroreflector, a mirror, a concave mirror or a lens. HOEs can also have a high angular or wavelength selectivity in terms of their function. For example, the components of white light are scattered to different extents in different spatial directions. Therefore, the orientation of the HOE relative to the illumination source can be encoded by the backscattered wavelength and determined with the aid of the image captured by the optoelectronic sensor 32 of an optoelectronic detection device 30. In this case, the spectrally resolved local intensity distribution P of three back reflections A, B and C of different HOEs 21, 22 and 23 can be imaged bijectively onto the location function F of the HOEs 21, 22 and 23 on a marker 20, measured at azimuth α and elevation β relative to the surface of the marker 20. For example, in FIG. 4, the location function F(α1, β1) can be deduced from the spectrally resolved local intensity distribution P1[I(A), I(B), I(C)] of the location-dependent intensities and colors on the two-dimensional image of the optoelectronic sensor 32. Analogously, for example, in FIG. 5, the location function F(α2, β2) can be deduced from the spectrally resolved local intensity distribution P2[I(A), I(B), I(C)] of the location-dependent intensities and colors on the two-dimensional image of the optoelectronic sensor 32.An optoelectronic sensor 32 of an optoelectronic detection device 30 can have a color depth of at least 8 bits, for example. This means that the optoelectronic sensor 32 can distinguish between approximately 16 million color nuances. As a result, a very precise angle measurement of both the azimuth angle α and the elevation angle β can be carried out with the optoelectronic detection device 30. Because a plurality of HOEs 21, 22 and 23 positioned differently on the marker 20 respectively generate unique, linearly independent color patterns on the optoelectronic sensor 32, the respectively associated angular orientation of the marker 20 can be encoded uniquely. If the orientation of the object or the position of the optoelectronic detection device 30 relative to the object and thus also the orientation of the marker 20 relative to the optoelectronic detection device 30 changes, the wavelengths of the light backscattered by the HOEs 21, 22 and 23 shift. This also makes it possible to draw conclusions about the change in position.The localization system can additionally have a light source 50, which can be provided as a separate device or as part of the detection device 30. The light source 50 can emit white light W onto the marker 20 in a targeted manner, for example, so that the position of the illumination and recording optics is known a priori. The determination of the location and angular orientation of the marker 20 on the basis of the back reflection patterns in the image of the optoelectronic sensor 32 thereby becomes simpler and more reliable. In addition, the light emitted by the light source 50 can be modulated, so that a clear assignment of the backscattered light and an improvement of the signal-to-noise ratio can be achieved via a lock-in mechanism.The HOEs 21, 22 and 23 can have optimized optical functions, such as, for example, a retroreflectivity with a limited angular range, scattering functions with a high angular selectivity, or any combination of these functions. Furthermore, the HOEs 21, 22 and 23 can be designed for light outside the visible range, such as in the UV or infrared range.FIG. 7 shows, by way of example, that specific patterns and / or specific color combinations of HOEs can be used, so that the determination of the distance of the detection device 30 from the marker 20 can be improved with regard to color, shape and size by way of pattern recognition. The angular selectivity of holograms enables an angular change or the object orientation in space to be encoded precisely. For example, one and the same optical function can be applied multiple times on the marker surface. The matching of wavelengths and intensities of the redundant optical functions can implement a control functionality. The optical function of the surfaces can alternatively also be selected for this purpose such that the intensities of the light diffracted back must differ. The deflection function can also be designed in opposite directions, for example, so that a point-symmetric intensity distribution of the back-diffracted light is thereby obtained. If the position of the marker 20 relative to the detection device 30 changes, this distribution must also change point-symmetrically. Regions that address different wavelengths can be arranged symmetrically or asymmetrically, whereby the color composition of the reflected light likewise changes symmetrically or asymmetrically when the position of the marker 20 changes. Oppositely directed arrangements of deflectors can be used to improve the estimation accuracy of the marker distance via the distance of the intensity maxima of the reflected light. Analogous advantages in the detection accuracy can also be achieved in the azimuth angle if the deflector structures are applied to the marker 20 rotated by 90°.The object position in space can also be extracted by using specific patterns as shown in the examples of FIGS. 9, 10 and 11. In one embodiment, a pattern may consist of, for example, circular holographic deflectors 25 arranged concentrically. They redirect light in such a way that an intensity maximum for a specific wavelength is obtained at a specific distance. It can be seen in FIG. 10, for example, that the holographic deflector 25 with the largest radius shows an interference pattern R with an interference maximum at a distance d 3 of the detection device 30 from the marker 20. Likewise, it can be seen that the holographic deflector 25 with the mean radius shows an interference pattern R with an interference maximum at a distance d 2 of the detection device 30 from the marker 20. Finally, it can be seen that the holographic deflector 25 with the smallest radius shows an interference pattern R with an interference maximum at a distance d 1 of the detection device 30 from the marker 20. FIG. 11 shows the associated wavelength-dependent intensity distributions P of the different wavelengths f1, f2 and f3 of the concentric holographic deflectors 25 as a function of the respective distances d1, d2 and d3. If the distance of the marker 20 from the detection device 30 is varied, this can therefore be read directly in the image detected by the optoelectronic sensor 32 at changing intensities of the rings and wavelengths.In a further implementation possibility, a plurality of markers 20 can be used in spatially predefined proximity to one another. For example, two, three or more markers 20 can be attached to the bumper of a car, the HOEs 21, 22, 23 of which can be designed with optical functions in such a way that incident light of different wave lengths is fanned out in different spatial directions. If the markers are then detected, for example, by a second vehicle approaching from behind, the distance between the two vehicles can be encoded as a function of the detected color(s) in the various markers 20. When the vehicle is moving vertically centrally, both markers 20 simultaneously change color.Problems in the position and location determination of markers 20 with HOEs 21, 22 and 23 can occur if the markers are detected at very low elevation angles. In this case, the effectively detectable area goes to zero. Detection and evaluation of the reflection images is thus unreliable and may not be possible at all starting from a specific elevation critical angle. Figure 8 shows how such problems can be alleviated by special holographic structures 24 printed on the marker 20. By using holographic structures 24, for example frame deflectors rotating around the QR code 10, which retroreflect light at very low elevation angles, a boundary alignment is indicated by the light scattered back. When the elevation angle reaches a critical range, the light reflected back from the frame deflector appears with particularly high intensity in the image captured by the optoelectronic sensor 32. As a result, the detection device 30 can recognize that the orientation of the marker 20 no longer permits reliable position and / or position determination with sufficient certainty.The orientation can be encoded by the intensity distribution and the spectral composition of the holographic structures 24, such as a revolving frame deflector. For example, the color of the frame director may change from green to red in the transition region to the critical region, or the back-reflected image of the frame director may change from a green rectangle to a red stop symbol.FIG. 6 schematically shows a flow diagram of a localization method M for determining the position and orientation of objects identified by at least one plurality of optoelectronically detectable markers 20, such as motor vehicles, pieces of furniture such as chairs, unmanned autonomously acting vehicles or other objects of the same or similar design. The localization method M can be implemented, for example, using a localization system 1 as shown by way of example in FIG. 3 and can be used for an application as explained by way of example in one of FIGS. 12 and 13.In a first step M 1, light reflected by holographically optical elements 21, 22, 23 applied to the markers 20 is detected by means of an optoelectronic sensor 32 of an optoelectronic detection device 30, the color depth of which is, for example, at least 8 bits. In a second step M 2, a two-dimensional reflection image is generated by the optoelectronic sensor 32 from the detected reflected light. In a step M 3, an image processing device detects differences between the reflection behavior of the optoelectronically detectable markers of the current position and the alignment of the plurality of objects and the reflection behavior of the optoelectronically detectable markers in a reference configuration of the two-dimensional reflection image captured by the optoelectronic sensor 32 with respect to the plurality of objects on the basis of the spectrally resolved local intensity distribution. This means that the intensities of different colors recorded in the reflection images are determined in a spatially resolved manner according to length and height in the reflection images, and are assigned to the respective different optoelectronically detectable markers. This can be done, for example, by comparing the two-dimensional reflection images with a multiplicity of comparison and / or training data stored in a database for the captured two-dimensional reflection images. Finally, in a step M 4, deviations in the overall configuration of the plurality of objects can be determined, relative to the optoelectronic detection device 30, by evaluating the detected differences. In particular, changes in the configuration, outer shape or contour of the objects can also be determined by this evaluation, since between optoelectronically detectable markers attached to different objects can be discriminated on the basis of the relative differences in the local reflection behavior of the markers in the two-dimensional reflection image relative to the reference configuration.Optionally, in a further step M 5, object recognition of structures of a contrast code 10 of the markers 20 recorded in the two-dimensional reflection images captured by the optoelectronic sensor 32 can be carried out.In the context of industrial production, mass logistics, hospital stores or generally in production and storage systems, desired ordering requirements can be complied with. As a result, owing to the efficiency gain in the work sequences, errors can be minimized and productivity can be significantly increased by increased work speed. In safety-relevant areas, in which objects must be arranged in a defined manner in space and / or must not be set in motion, fixed, required space distributions and object installations are known. From the standpoint of work safety, a fixed sorting and ordering of things, deposits or installations or infrastructure is advantageous in order to be able to always maintain minimum distances or safety zones in a work environment.With the use of optoelectronically detectable markers, the use of imaging cameras which may be problematic under certain circumstances with respect to data protection is no longer absolutely necessary. Nevertheless, objects which are not at their desired position or are not at their desired angle to other objects or to the space can be detected with the localization system. Sources of danger, such as an open storage bin, can be compared with the desired state and recognized. At work stations, an overview of the actual state can be obtained quickly and reliably. The efficiency of workflows is advantageously optimized, since the sorting of things is determined before the start of the workflow. Furthermore, in the case of objects in explosion-hazardous, safety-relevant or contaminated use environments or also in environments in which contact with objects is not desired, such as in museums, contactless detection of arrangements of objects is possible. The number of simultaneously detectable objects of an arrangement is arbitrary in principle and the localization system is cost-effective and easily scalable with the use of more optoelectronically detectable markers.By providing a plurality of objects of a specific arrangement with cost-effective optoelectronically detectable markers and using at least one optoelectronic detection device, a state of a space that is to be established and recognizable in the regular mode can be monitored. Anomalies from the desired state can be detected as a deviation and assigned to one of the plurality of objects of the arrangement and to a location in space, in order to enable, for example, a targeted recovery of the order.Based on the recognition of the optoelectronically detectable markers and the change of their color or pattern spectrum in the case of smallest angle changes to the optoelectronic detection device, for example, the example scenarios illustrated in FIGS. 12 and 13 can be implemented. In the case of anomalies, the colour profile changes over time in the respective image point and thus in the detected signal profile compared to the desired state. If a plurality of optoelectronic detection devices are installed, in addition, trigonometric calculations such as, for example, trilateration or triangulation can lead to a further improved localization result.The optoelectronic sensing devices may also be mobile, for example, such as by implementation on an unmanned autonomous vehicle such as a drone or by using a sensor of a portable terminal such as a laptop or smartphone.The detection of an anomaly in the signal processing of the localization system is carried out by comparing an ideal reference signal, which is learned or programmed in in the desired state, in the desired operation with the current detection signal. In this case, this comparison takes place selectively separately for each optoelectronically detectable marker or the spatio-temporal arrangement of the markers is compared with the desired state. The comparison itself can be carried out in various ways, for example by analysis of the time series of the detected color and intensity distribution.FIG. 12 shows a schematically illustrated diagram of an arrangement 100 of identical or similar objects 101 which are each marked with one or more optoelectronically detectable markers 20 at specific marking locations on the object 101. In the example of FIG. 12, cube-shaped objects 101 such as boxes or workpieces are shown as being attached to one of the side surfaces with a marker 20 in each case, wherein the number and exact attachment position of the markers 20 can of course deviate from the illustrated example. The objects 101 can also have, for example, motor vehicles, pieces of furniture such as chairs, unmanned autonomously acting vehicles or other objects of the same or similar design.An optoelectronic detection device 30 can monitor all objects 101 of the arrangement 100 and in the process record reflection images on which light reflected by holographically optical elements applied to the various markers 20 can be seen in varying color and intensity distribution. Over time, changes to the local distribution and / or orientation of the individual objects 101 of the arrangement 100 can lead to the overall configuration of the markers 20 in three-dimensional space changing. Due to the spatially and temporally fixed attachment of the markers 20, this also leads to changes in the reflection behavior of individual markers relative to one another, so that the reflection images reflect deviations or differences in the reflection behavior of individual markers relative to other markers. These changes in the reflection behavior of individual markers 20 can be used to draw conclusions about a relative change in the position and / or alignment of the objects 101 of the arrangement 100 with respect to one another.The arrangement 100 can be detected in its overall configuration, i.e. in the relative alignment and position of the respective objects 101 with respect to one another. For this purpose, a reference configuration can be established, i.e. a desired configuration or a configuration of the objects 101 characterized as normal. For this reference configuration, a corresponding reflection image can be recorded and stored in a database for reflection images as comparison and / or training date. If deviations from the reflection image of the reference configuration are ascertained after the recording of further reflection images of the arrangement 100 of the objects 101, the image processing device 34 of the localization system 1 can output a corresponding monitoring signal which indicates a deviation of the expected overall configuration of the arrangement 100.With the aid of a localization system 1 based on optoelectronically detectable markers 20, evaluations of tunnels, dams or bridge structures, for example in order to detect damage to the structure, can be carried out cost-effectively, reliably and highly accurately. In particular, no complicated visual inspection by experts is necessary, but instead a current state recording can provide direct information about the state of the structure. Even very small changes can be made visible with the aid of optoelectronically detectable markers 20 using established hardware such as cameras as optoelectronic detection devices 30, for example. As a result, highly accurate recordings can also be made of small changes over the time profile, for example with the aid of computer-assisted image evaluation methods (computer vision, object recognition, machine learning, artificial intelligence. Structural changes in the structure, such as deformations, cracks or fractures, can also be detected with the localization system 1 without complex or cost-intensive sensor systems and without expensive or sensitive technology. Due to its simplicity, the optoelectronic detection devices 30 can also be installed in vehicles which are driven past buildings. The recorded image material can be processed using appropriate algorithms and compared with image material at earlier points in time.FIG. 13 shows a schematically illustrated diagram of an arrangement 200 of identical or similar objects 201, which are each marked with one or more optoelectronically detectable markers 20 at specific marking locations on the object 201. In the example of FIG. 13, drawers 201 of a rack system 200 or a workbench 200 are shown as being attached to one of the drawer surfaces with a marker 20 in each case, wherein the number and exact attachment position of the markers 20 can of course deviate from the illustrated example. The objects 201 can also have tools, movable elements or other objects of the same or similar construction, for example.An optoelectronic detection device 30 can monitor all objects 201 of the arrangement 200 and in the process record reflection images on which light reflected by holographically optical elements applied to the various markers 20 can be seen in varying color and intensity distribution. Over time, changes to the local distribution and / or orientation of the individual objects 201 of the arrangement 200 can lead to the overall configuration of the markers 20 in three-dimensional space changing. Due to the spatially and temporally fixed attachment of the markers 20, this also leads to changes in the reflection behavior of individual markers relative to one another, so that the reflection images reflect deviations or differences in the reflection behavior of individual markers relative to other markers. These changes in the reflection behavior of individual markers 20 can be used to infer a relative change in the position and / or alignment of the objects 201 of the arrangement 200 with respect to one another.The arrangement 200 can be detected in its overall configuration, i.e. in the relative alignment and position of the respective objects 201 with respect to one another. For this purpose, a reference configuration can be established, i.e. a desired configuration or a configuration of the objects 201 characterized as normal. For this reference configuration, a corresponding reflection image can be recorded and stored in a database for reflection images as comparison and / or training date. If deviations from the reflection image of the reference configuration are ascertained after the recording of further reflection images of the arrangement 200 of the objects 201, the image processing device 34 of the localization system 1 can output a corresponding monitoring signal which indicates a deviation of the expected overall configuration of the arrangement 200.In summary, the present invention relates to a concept for improved position and orientation determination of a multiplicity of objects identified by at least one optoelectronically detectable marker in each case, in which holographically optical elements are introduced into a two-dimensional planar marker, at which ambient light or light of a dedicated additional light source is reflected back with sufficiently high intensity even at high scattering angles with respect to the surface normal of the two-dimensional planar marker. This enables a very precise position determination of the markers with respect to all solid angles, since the holographic optical elements generate reflection patterns in an optoelectronic detection device which can be distinguished depending on the angle and are linearly independent, and on the basis of which the position, location and orientation of the markers and thus the corresponding position, location and orientation of the objects identified thereby can be deduced relative to one another.By using optoelectronically detectable markers, after the markers have been attached at any location and at any construction stage, and by light sources attached per construction space, for example, the progress and the exact localization of objects of the same construction, such as pipes and / or lines, can be documented during installation. This can advantageously be effected directly and in real time, but a precise and individual digital image of each installation and of each completed construction can also be generated by previously scanned and stored surveys.In the field of manufacturing as well, machines provided with optoelectronically detectable markers can be quantified in a production shop optionally also provided with optoelectronically detectable markers, even in the case of a variable arrangement and variable parking machine, the respective location and the orientation of the machine. The same applies to the localization of vehicles / robots in manufacturing or in private areas. This can be effected, for example, by the attachment of optoelectronically detectable markers as reference points along the upper wall of a manufacturing shop or a private home at specific distances from one another, or on distinctive room parts such as columns. This can be used in the same course, or also independently of one another, for the mutual recognition of vehicles / robots, such as assembly robots, vacuum cleaner robots or lawnmower robots, if additional information about the type of optoelectronically detectable markers can be exchanged among the vehicles / robots.The concept can be supplemented by the implementation of special two-dimensionally planar patterns and / or shapes of the holographic optical elements, such that the intensity of the back-reflected ambient light generates color-dependent intensity maxima depending on the distance of the marker from the optoelectronic detection device. In addition, special holographic optical deflection elements with retroreflectivity maxima at very high scattering angles with respect to the surface normal of the two-dimensionally flat marker can also be provided, so that the detection of such intensity maxima in the optoelectronic detection device indicates an indication that a critical angular range has been reached in which a marker orientation can no longer be ensured with sufficient certainty on account of the excessively low detectable marker surface.References included in the specificationThis list of documents cited by the applicant has been produced in an automated manner and is only included for the better information of the reader. The list is not part of the German patent application or utility model application. The DPMA does not take any adhesion for any faults or omissions.Patent Literature citedUS 2021 / 0248338 A1

[0003] PT 99405 A

[0003] CZ 2004-1136 A3

[0003] U.S. Pat. No. 10,885,413 B1

[0003] DE 10 2017 218 544 A1

[0022]

Claims

Locating system (1) for determining the position and / or orientation of a multiplicity of objects (100; 200; ) identified by at least one optoelectronically detectable marker (20) in each case, having: an optoelectronic detection device (30) which has: an optoelectronic sensor (32) which is designed to detect light emanating from, in particular diffracted and / or reflected and / or backscattered, holographically optical elements (HOE; 21, 22, 23) applied to the markers (20) and to generate a two-dimensional reflection image; and an image processing device (34), which is coupled to the optoelectronic sensor (32) and is designed to detect differences between the reflection behavior of the optoelectronically detectable markers (20) of the current position and orientation of the plurality of objects (100; 200) and the reflection behavior of the optoelectronically detectable markers (20) in a reference configuration of the two-dimensional reflection image acquired by the optoelectronic sensor (32) on the basis of the spectrally resolved local intensity distribution of the two-dimensional reflection image acquired by the optoelectronic sensor (32) and to determine deviations in the overall configuration of the plurality of objects (100; 200) relative to the optoelectronic acquisition device (30) by evaluating the detected differences.Localization system (1) according to claim 1, wherein the optoelectronic detection device (30) further comprises a memory (33), which is coupled to the image processing device (34) and stores a plurality of comparison and / or training data for the detected two-dimensional reflection images in a database.Localization system (1) according to Claim 1 or 2, wherein the image processing device (34) is furthermore designed to carry out object recognition of structures of a contrast code (10) of the markers (20) recorded in the two-dimensional reflection images captured by the optoelectronic sensor (32).The locating system (1) according to any one of claims 1 to 3, wherein at least one of the holographic optical elements (21, 22, 23) applied to the markers (20) comprises a holographic deflector (24) with high angular selectivity at low elevation angles with respect to the surface of the marker (20).The localization system (1) according to any one of claims 1 to 4, wherein the color depth of the optoelectronic sensor (32) is at least 8 bits.Localization system (1) according to one of Claims 1 to 5, wherein at least one of the holographic optical elements (21, 22, 23) applied to the markers (20) has an optical function, such that a changing distance of the detection device (30) from the objects (100; 200) is indicated by a colour change and / or a change in the intensity distribution in the two-dimensional reflection images detected by the optoelectronic sensor (32).Localization system (1) according to one of Claims 1 to 6, wherein at least one of the holographic optical elements (21, 22, 23) applied to the markers (20) has an optical function, such that a changing object orientation for dedicated angle ranges is indicated by a colour change and / or a change in the intensity distribution in the two-dimensional reflection images captured by the optoelectronic sensor (32).The localization system (1) according to any one of claims 6 and 7, wherein at least two of the holographic optical elements (21, 22, 23) applied to the markers (20) have duplicated optical functions for improving the reliability of a determination of the distance of the detection device (30) from the objects (100; 200) and / or a determination of the object orientation for dedicated angular ranges.Localization method (M) for determining the position and / or orientation of a plurality of objects (100; 200), each characterized by at least one optoelectronically detectable marker (20), comprising the steps: detecting (M1) light emanating from, in particular diffracted and / or reflected and / or backscattered light applied to the markers (20) by holographically optical elements (HOE; 21, 22, 23) by means of an optoelectronic sensor (32) of an optoelectronic detection device (30); generating (M2) a two-dimensional reflection image by the optoelectronic sensor (32) from the detected reflected light; detecting (M3) differences between the reflection behavior of the optoelectronically detectable markers (20) of the current position and orientation of the plurality of objects (100; 200) and the reflection behavior of the optoelectronically detectable markers (20) in a reference configuration of the two-dimensional reflection image detected by the optoelectronic sensor (32) relating to the plurality of objects (100; 200) on the basis of the spectrally resolved local intensity distribution; and ascertaining (M4) deviations in the overall configuration of the plurality of objects (100; 200) relative to the optoelectronic detection device (30) by evaluating the detected differences.Localization method (M) according to Claim 9, wherein the evaluation (M3) of the spectral and / or local intensity distribution in the two-dimensional reflection images captured by the optoelectronic sensor (32) comprises comparing the two-dimensional reflection images with a multiplicity of comparison and / or training data stored in a database for the captured two-dimensional reflection images.Localization method (M) according to one of Claims 9 and 10, further having the steps: carrying out (M5) object detection of structures of a contrast code (10) of the markers (20) recorded in the two-dimensional reflection images captured by the optoelectronic sensor (32).Localization method (M) according to one of Claims 9 to 11, wherein at least one of the holographic optical elements (21, 22, 23) applied to the markers (20) has a holographic deflector (24) with high angle selectivity at low elevation angles with respect to the surface of the marker (20).Localization method (M) according to one of Claims 9 to 12, wherein the color depth of the optoelectronic sensor (32) is at least 8 bits.Localization method (M) according to one of Claims 9 to 13, wherein at least one of the holographic optical elements (21, 22, 23) applied to the markers (20) has an optical function, such that a changing distance of the detection device (30) from the objects (100; 200) is indicated by a change in the spectral and / or local intensity distribution in the two-dimensional reflection images detected by the optoelectronic sensor (32).Localization method (M) according to one of Claims 9 to 14, wherein at least one of the holographic optical elements (21, 22, 23) applied to the markers (20) has an optical function, such that a changing object orientation for dedicated angle ranges is indicated by a change in the spectral and / or local intensity distribution in the two-dimensional reflection images captured by the optoelectronic sensor (32).Localization method (M) according to one of Claims 14 and 15, wherein at least two of the holographic optical elements (21, 22, 23) applied to the markers (20) have duplicated optical functions for improving the reliability of a determination of the distance of the detection device (30) from the objects (100; 200) and / or a determination of the object orientation for dedicated angular ranges.

Images