Method and system for displaying a view dependent on the viewing direction

Abstract

The invention relates to a method and a system (10) for displaying a viewing direction-dependent view for a predetermined location. The predetermined location, including its surroundings, is captured from one or more viewing directions (15), with or without one or more objects (12) at the location. One or more objects (12) at the predetermined location are then obscured by a three-dimensional display device (20). Based on the images captured in the preparation phase (200), a display model is calculated. This model allows the predetermined location, including its surroundings, to be displayed using the light-activated elements (22) of the display device (20) for each of the one or more predetermined viewing directions (15) or for each viewing direction (15) within the one or more predetermined solid angles. In the absence of one or more objects (12), the location can be displayed from the respective viewing direction (15). In an application phase (206) a momentary viewing direction (15) is determined, a representation of the predetermined location together with its surroundings in the absence of one or more objects (12) is determined from the momentary viewing direction (15) by means of the display model and is displayed by means of the light-active elements (22) of the display device (20) in relation to the momentary viewing direction.

Description

[0001] The invention relates to a method and a system for displaying a viewing direction-dependent view for a predetermined location when viewed from one or more predetermined viewing directions or one or more predetermined solid angle ranges.

[0002] In many cases, it is desirable to manipulate the visual perception of a location, ideally without the manipulation being immediately noticeable, so that what is perceived blends into the location as naturally as possible. Applications for this can be aesthetic in nature, such as art installations, or representations that inform or influence the viewer, for example, by confusing, distracting, or deterring them from the location, or by attracting them to it. For instance, it could be deceptive to create the illusion that objects not actually present are there, or conversely, objects that are actually there could be concealed by presenting a view of the location that does not include the object(s), such as a background or foreground.

[0003] The targets of such manipulation of visual perception can include not only humans but also animals or artificial structures capable of visual perception, such as camera drones. Examples of applications for distraction include wildlife observation or hunting in the wild, to avoid disturbing the animals being observed or hunted. In nature observation, passive camouflage nets for people or observation devices are frequently used, but these are only suitable for a limited number of environments and are therefore not flexible. Actively adapting a view to a specific location can overcome this limitation. This can be expanded upon. A view designed to attract animals for nature observation could, for example, include a feeding station, possibly with additional attractants. For deterrence, visual effects or objects that appear dangerous could be included in the view.

[0004] When manipulating the visual perception of a place as realistically as possible, it must be taken into account that a place is a three-dimensional entity which in many cases is visible from different perspectives, including from different sides and possibly also from above, from the air.

[0005] In BBC STUDIOS [TopGear]: How To Create An Invisible Car - 50 Years of Bond Cars - Top Gear, it is shown how a vehicle can be made "invisible" using cameras and display devices attached to it. The area obscured by the vehicle or the attached display device is captured and shown on the display.

[0006] The video NETFLIX Production Technology Resources [cpodigitalbacklot]: Into the Volume: A Behind-the-Scenes Look into the Virtual Production of 1899 showcases a virtual production studio where a recording of a pre-recorded or virtual location is played back. The rendering of real locations is made possible through photogrammetry, in which previously created photographic records of the location are transferred into a virtual model. The camera used for recording can be positioned anywhere within the model to modify the image displayed. The perspective of the displayed image is adjusted to the camera's field of view.

[0007] German patent application DE 10 2016 218 602 A1 discloses a method and a device for altering the perception of the outside world through a vehicle window. The vehicle's surroundings are captured image-wise and augmented with virtual 3D objects during display. A perspective correction of the virtual object is performed, and the recorded location is obscured by the display.

[0008] WO 2023 / 162646 A1 discloses a method and apparatus for a virtual production studio in which a perspective correction of the display is performed according to its viewing cone for the content displayed. The display allows both real and virtual content to be shown.

[0009] Similarly, US 2024 / 0053943 A1 discloses a method and a device for adjusting the perspective representation of content displayed on a display device according to the viewing direction or perspective of the recording camera.

[0010] Furthermore, DE 10 2019 112 471 A1 discloses a method and a device for displaying hidden content on a screen in a motor vehicle. Image data of the vehicle's surroundings are combined in such a way that content hidden by a real object can be displayed.

[0011] Starting from this premise, the object of the present invention is to create a way to optically manipulate views of a place from different perspectives.

[0012] This task is solved by a method for displaying a viewing-direction-dependent view for a predetermined location when viewed from one or more predetermined viewing directions or one or more predetermined solid angle ranges. encompassing a preparation phase in which the predetermined location, in particular including its surroundings, is captured visually from one or more viewpoints, wherein the one or more viewpoints for the visual capture cover the one or more predetermined viewpoints or the one or more predetermined spatial angle ranges, a setup phase that takes place before, during or after the preparation phase, in which the predetermined location is covered with a three-dimensional display device which is equipped on its outside with a large number of individually controllable light-active elements, a model generation phase occurring during or after the preparation phase, in which a display model is calculated in a control device based on the images captured in the preparation phase, with which a model view of the predetermined location, in particular including its surroundings, seen from the respective viewing direction, can be displayed for each of the predetermined one or more viewing directions or for each viewing direction within the one or more predetermined solid angles, by means of the light-active elements of the display device. and after completion of the preparation phase and the model generation phase, an application phase in which a momentary viewing direction is determined, a model view of the predetermined location, in particular including its surroundings, is determined by means of the display model and is displayed by means of the light-active elements of the display device.

[0013] The inventive method makes it possible to flexibly create a display model for any possible location and, depending on the viewing direction and the location and, if applicable, its surroundings, to display manipulated views of the display model on the three-dimensional display device. For example, objects that would be perceived as foreign elements at the location can be hidden by placing the display device over the objects and showing a view of the location without them. Such objects could be recording devices for nature observations, hunting shelters, or similar. Alternatively, objects that are not present at the location can be displayed in the view.

[0014] The invention is based on the fundamental idea of ​​providing a three-dimensional display device that can present a manipulated view of a location where it is placed. The manipulated view can include the surroundings of the location, which would otherwise be visible as the background and / or surface.

[0015] The display device is three-dimensional, which in the context of the present application means that its light-active surface is structured three-dimensionally, with light-active elements pointing in different directions. Within the scope of this disclosure, "light-active" means that the corresponding element is configured either to emit light itself or to change its reflection and transmission properties for available light, i.e., to manipulate light for the viewer.The light-active elements can be, for example, LEDs attached to a curved surface, such as in a hemispherical or spherical shape similar to the Las Vegas Sphere or a geodesic dome, in a size and shape appropriate to the location and any objects present there, on honeycomb-shaped or generally polyhedral surfaces, cylindrical surfaces, or on a flexible, in particular textile, carrier that can be draped over or around a three-dimensional object, so that parts of the light-active surface of the display device are visible from various, in particular all, directions, especially also from above.

[0016] Examples of light-active elements include self-illuminating displays, optical fibers, or light-reflecting elements with variable optical properties, especially E-Ink assemblies.

[0017] To facilitate transport, the display device can be modular, particularly with polyhedrally arranged surfaces that can be disassembled and reassembled. Flexible supports can be easily rolled or folded, enabling very simple and space-saving transport.

[0018] In some embodiments of display devices designed as flexible carriers, they are provided with an irregular outer boundary. This can be implemented, for example, in the form of a ragged, randomly shaped cloth without a clear outer edge. Such a deviation from a clear round or regular angular shape can help prevent the display device from being recognized as an artificial object as quickly due to its edge structure.

[0019] To achieve perspective-correct display for one or more viewing directions, given the three-dimensional structure of the light-activated elements, the first step is to capture images of the location where one or more objects are situated or will be placed, from one or more viewing directions. This can be done from multiple perspectives, for example, particularly from ground level, using one or more cameras from different angles, or even using a drone, which, in addition to ground-level side views, can also capture still images or video footage from various angles, including from above and at an oblique angle.

[0020] These images then serve as a reference upon which the display model is based. Ideally, the reference images should show the "empty" location without any objects placed there. This is automatically the case if the location is already empty when the reference images are taken, meaning there are no objects to be concealed. If objects are to be concealed at the location, these images show what the predetermined location would look like without the objects. If, however, the images are taken with objects already in place, an additional processing step using common image editing tools, possibly or preferably AI-assisted, is possible to edit the captured images so that the objects in the reference images are replaced with image elements that match the surroundings, i.e., the background and foreground, in order to create the desired reference images of the "empty" location.

[0021] The display model can then generate views of the "empty" location based on these reference images of the "empty" location, or integrate non-existent objects using image processing, optionally or preferably AI-supported. Such a display model makes it possible to represent the predetermined location and its surroundings, either empty or with objects projected onto the location that are not actually there, from one or more desired viewing directions—that is, from one side, from different sides, or even from above—on the display device.

[0022] The display model also takes into account the shape of the display device in order to display the desired representation in perspective correctly for a given current viewing direction, even if the display device is not a flat object.

[0023] Furthermore, the display model preferably takes into account the resolution achievable with the light-active elements of the display device. This saves computing power by generating the model view at the resolution of the display device, rather than at a higher resolution that would then need to be downscaled to the resolution of the display device. The view to be displayed can include a specific color scheme and pattern that matches the environment and / or background and can be represented by the display device with a granularity appropriate to the environment and / or background within its resolution.

[0024] In further embodiments, the display model can include a movement of the pattern or view to be displayed, corresponding to the movement of structures in the vicinity. Examples include trees, shrubs, or grasses moving in the wind. To monitor the movement of these structures, an external camera is directed at the surroundings, its images are analyzed for movement of the structures, and the image displayed by the device is moved accordingly. Predominant movement patterns or directions, such as those found in swaying branches, leaves, or grass, can serve as a model.

[0025] In some embodiments, during the application phase, a monitoring device monitors the environment of a predetermined location. Upon detection of an external object, particularly a moving or mobile one, the device determines the object's current viewing direction relative to the location and tracks any movement of the external object. Suitable monitoring devices are based, for example, on optical measuring instruments such as stationary or mobile cameras, or on other measuring instruments such as lidar, radar, radio direction finding, thermal or ultrasonic reflectometers. Acoustic sensors or motion sensors can also be used, which signal when a moving external object is in the vicinity of the display device. In such cases, more precise location-sensitive sensors are used to determine the current location of the external object.Acoustic sensors can, for example, have directional characteristics, or multiple microphones can be distributed around or attached to the display device, and the position of an external object can be determined via time-of-flight measurements. External objects can include people, wild animals (including birds of prey), and inanimate objects such as vehicles, airplanes, helicopters, drones, etc.

[0026] The monitoring device or devices can be arranged externally, but can also be arranged on the surface of the display device and directed towards the environment of the display device.

[0027] Especially with moving external objects, it is advantageous to continuously determine their position relative to the display device, for example by means of one or more monitoring devices, and to track the current viewing direction, on the basis of which the displayed image is calculated, so that the object is shown a background of the place where the display device is located that changes according to its changing viewing angle, whereby the ground can also be a background.

[0028] In some embodiments, the display model implements a predetermined or spontaneously assumed shape of the display device and is refined during the model generation phase by capturing test images of the predetermined location and its surroundings from one or more viewing directions, particularly using a drone or one or more external cameras. These test images are then compared with a target representation, and if the test images deviate from the target representation, the display model is adjusted to minimize these deviations. This iterative refinement may also reveal that certain parts of the display device are not visible in the images from the various viewing directions for which the display is being optimized.This can occur, for example, if the solid angle from which the display device is to be visible does not cover all sides of the display device, or if, for instance, a flexible, fabric-like support for the light-activated elements has been draped around or over an object in such a way that an edge is partially folded over or creased and therefore perspectively obscured by an overlying layer. Such non-visible parts of the display device can then be masked in the display model and subsequently no longer controlled in order to represent part of the background of the location.

[0029] To recognize the three-dimensional structure of the display device, various solutions are conceivable. For example, a grid can be displayed on the fabric, i.e., a pattern of grid lines, perhaps with intersecting lines at right angles. Due to the three-dimensional structure of the display device, which might be a textile support draped over an object, the grid deforms in a characteristic way, similar to the display of 3D models. The deformations of the grid lines can then be used to determine the distribution of the display area in space. The line pattern can be chosen to be asymmetrical enough to clearly indicate the orientation of the display device. Instead of grids, other unambiguous patterns can also be displayed, such as randomly distributed lines, dot patterns, or the like, whose distribution is known in the flat state.

[0030] Recognition without displaying a grid or other pattern on the display device is possible, for example, by equipping a drone with a lidar that allows distance measurement and can thus scan the shape of the display device from different sides.

[0031] To ensure the perspective accuracy of the display, test images from a greater number of different directions should be used for iterative improvement with an irregularly shaped display device than would be necessary with regularly shaped displays. Video recordings, for example from a drone orbiting the irregularly shaped display device and recording from different angles, are also suitable for fine-tuning.

[0032] In certain embodiments, the display of the predetermined location and its surroundings is adapted to changing lighting conditions, which are detected in particular by means of one or more light sensors or cameras arranged on or near the display device. Specifically, to adapt the display, the intensity and, if necessary, the color of the light output of the light-active elements of the display device is adjusted to match the intensity of natural reflection. The latter can be achieved by one or more external cameras that capture the surroundings and the display device, and whose images are evaluated to make the adjustment and blend the display with the surroundings.Alternatively, a model of the reflectivity of the environment can be created using reference images of the surroundings or the substrate and taken into account in the display model, using one or more internal or external cameras to detect changes in lighting conditions.

[0033] Other environmental parameters that can influence the appearance of the location include, for example, wind direction and speed, which move blades of grass, bushes, or similar objects in a particular direction, according to the structures surrounding the display device, or ground-level thermals on hot days, where the hot ground heats and swirls the air layer near the ground, resulting in shimmering effects. In these cases, an anemometer and / or a ground-level thermometer may also be present, transmitting their readings to the control device.

[0034] In some embodiments, light sensors can detect deviations in brightness. Based on this observation, and especially its temporal development, it can be determined whether it is generally getting brighter or darker, whether, according to the sun's movement across the sky, a shadow is being cast by a nearby object, which may end after some time, or whether a cloud is only temporarily passing in front of the sun or uncovering it again.

[0035] To reduce or increase reflection, a layer can be used, for example, a film or textile whose transluminescence or surface properties can be adjusted. Films that become diffuse when an electrical voltage is applied are known. The desired diffusion level can be determined using a contrast sensor, such as a camera pointed at an object casting a shadow and the shadow cast by that object, measuring the contrast between the cast shadow and the adjacent unshaded area. Such a contrast sensor reacts very quickly to, for example, passing clouds and allows the rendering of the location to be adapted to changing lighting conditions.

[0036] In some embodiments, the display model is a generative image generation model that is iteratively adapted to the surrounding background using external images of the display device at a predetermined location. In one embodiment, this can be achieved by taking aerial photographs after an object has been placed and the display device positioned or draped over it. While the images are being taken by a drone or other camera devices, the display device is generatively adapted to the surrounding background. The generated images are repeatedly adjusted and evaluated until any discernible difference is minimized. Such a process can be performed using generative artificial intelligence. This can also be used for other applications where views of non-existent objects need to be displayed at a specific location.

[0037] The problem underlying the invention is also solved by a system for displaying a viewing-direction-dependent view for a predetermined location when viewed from one or more predetermined viewing directions or one or more predetermined solid angle ranges, comprising a three-dimensional display device which is equipped on its outside with a plurality of individually controllable light-active elements, a control device which is configured to control the plurality of light-active elements on the outside of the display device, and one or more stationary or mobile, ground-based or airborne, image acquisition devices which are configured to transmit images to the control device via a data communication interface, wherein the system is configured to carry out a method described above in accordance with the present disclosure or to be used in such a method.

[0038] The system thus achieves the same properties, characteristics, and advantages as the previously described method. The statements made regarding the components of the system in connection with the previously described method apply equally to these individual components.

[0039] In embodiments, the system comprises a monitoring device which is data-connected to the control device and configured to detect external objects and determine their direction of view in relation to the display device and report it to the control device, wherein the monitoring device comprises in particular optical, acoustic, lidar, radar, radio direction finding, ultrasonic and / or motion sensors.

[0040] In further embodiments, the system further comprises one or more light sensors, light contrast sensors or cameras arranged in, on or near the display device and connected to the control device, wherein the control device is configured to adapt the display of the predetermined location and its surroundings to changing light conditions detected by the light sensor(s), light contrast sensors or cameras.

[0041] The light-active elements are, in various embodiments, light-generating elements, in particular LEDs, LED assemblies, self-illuminating displays, or optical fibers, or light-reflecting elements with variable optical properties, in particular E-Ink assemblies. The latter have the advantage that they continue to function even during a power failure and retain their color or pattern. This makes it possible, for example, to load a static image once and then use the display device decentrally. Here, it is possible, for instance, to set the control to HOLD after initial programming, thus preventing any active changes to the display model. For this purpose, it is advantageous if the system can be used autonomously via its own battery power supply.

[0042] The light-activated elements can be applied to the display device as flexible or inflexible assemblies or components, optionally on both sides of the display device. These are designed to receive pulses, current, or other signals. They can have centralized or decentralized control chips or can be attached directly to a textile substrate without circuit boards.

[0043] In some embodiments, the display device is a pre-formed or flexible, in particular textile, planar carrier for the light-active elements. Flexible carriers are known, for example, from DE 10 2014 206 882 A1 or DE 10 2020 116 917 A1 as flexible surface lighting elements. Materials such as fabric, films, or other flexible materials, for example, woven, metal, or plastic protective coatings, comparable to chainmail, can be used as the base for the display device. The planar carrier can also be assembled from or be composed of combinable, for example, pre-assembled, sections. Such sections can be designed as strips or as individual, for example, rectangular or square pieces that are joined together at their edges and for this purpose have, in particular, mechanical and / or electrical connectors.

[0044] The display device can also be advantageously protected against water, sand, dirt, and other environmental influences. To adapt the display device to varying weather conditions or other factors requiring material adjustments during operation, it can be combined with additional materials, such as films, fabrics, nets, camouflage netting, or similar substances. This can be advantageous, for example, when the spaces between the display device's components need to be either light or dark, or when reflections from the display device need to be significantly reduced. Such composite materials can either be integrated directly into the display device or mechanically attached to it.

[0045] The display device can also include a flat film whose light transmission can be electrically controlled, and behind which a white surface is placed, so that in particular all shades of gray, shadows and / or darkenings can be produced.

[0046] Furthermore, the display device can have thermally active elements and / or a cooling system on its exterior and / or interior. These serve to adapt the display device to its environment, particularly in the infrared range. In this context, thermally active elements are those capable of altering the thermal radiation emitted by the display device. Examples include a layer of an IR-absorbing, but optically transparent material applied to the exterior of the display device, or coatings of such a material, such as domes, applied over the individual light-active elements. Such materials are known.

[0047] On the inside of the display device, the primary focus is on cooling, rather than heating, as the display device is exothermic and therefore tends to heat up relative to its surroundings. Cooling can be achieved through various methods. The heat generated at the location of the light-active elements on the outside of the display device can be dissipated via an internal layer of a metal mesh or braid, i.e., a material made of a thermally conductive metal with a high specific surface area. This can also be a tubular mesh in which the individual fibers are hollow. Alternatively or additionally, cooling tubes through which a cooling fluid flows can be provided on the inner wall of the display device.

[0048] In both cases, the heat can be extracted, for example, via a heat pump, which is operated in such a way that it cools the metal mesh or the coolant and dissipates the extracted waste heat. This heat can be released, if present, at an unobserved rear surface of the display device, or transferred to a heat sink with high heat capacity, such as one connected to a geothermal probe that already possesses a high heat capacity underground. A reflective coating on the inside of the display device can suppress heat loss from the interior to the outside.

[0049] In the case of active heat regulation or cooling, the system may have heat or infrared sensors designed and oriented to detect the heat radiation of the environment of the display device and to transmit this information to the control device, wherein the control device is designed and configured to regulate the cooling in such a way that the heat radiation from the display device is matched to the detected heat radiation of the environment.

[0050] To stabilize the three-dimensional structure of the display device, exemplary embodiments provide for internal air chambers that form a frame, ridges, or other shapes. Internal air chambers are barely detectable even using non-optical methods.

[0051] Further features of the invention will become apparent from the description of embodiments according to the invention, together with the claims and the accompanying drawings. Embodiments according to the invention may fulfill individual features or a combination of several features.

[0052] Within the scope of the invention, features marked with "in particular" or "preferably" are to be understood as optional features.

[0053] The invention is described below, without limiting the general concept, with reference to exemplary embodiments and the drawings, whereby for all details of the invention not explained in detail in the text, explicit reference is made to the drawings. The drawings show: Fig. 1 a schematic representation of a system according to the revelation, Fig. 2 schematic representations of basic types of display devices usable according to the disclosure Fig. 3 a schematic representation of a procedure according to the disclosure.

[0054] In the drawings, identical or similar elements and / or parts are provided with the same reference numbers, so that a re-presentation is omitted.

[0055] Fig. Figure 1 shows a schematic representation of a system 10 according to the disclosure. It illustrates how objects 12 at a given location are concealed by a display device 20, which has a plurality of light-active elements 22 on its outer surface. These elements can be, for example, LEDs or LED assemblies, self-illuminating displays or optical fibers, or light-reflecting elements with variable optical properties, in particular E-Ink assemblies.

[0056] In this initial example, the display device 20 is set up in nature and surrounded by plants, including grass 2, trees 4 and bushes 6.

[0057] System 10 includes a control device 24, which in this initial example is arranged within the display device 20. The display device 20 also includes monitoring devices 26, for example optical or infrared cameras, light sensors 32, radar or lidar sensor devices, which are configured to monitor the environment of the display device 20.

[0058] An external camera 30 is also attached to a nearby tree 4, which is positioned so that the display device 20 is in its field of view, along with part of the surroundings of the display device 20.

[0059] Furthermore, a drone 28 is in Fig. Figure 1 shows a drone 28 hovering obliquely above the display device and having a viewing direction 15 on the display device 20. Such a drone 28, like external cameras, can be part of the system 10 and used to record the location and its surroundings in order to provide image data for generating a display model according to the invention, or to refine the display model after its initial generation by checking whether the image displayed by the display device 20 from the viewing direction 15 for which it was generated actually looks as it should from that viewing direction 15. If this is not the case, this is reported back to the control device 24, which then adjusts the display model so that deviations between the intended image and the image actually displayed from the viewing direction 15 are minimized.

[0060] Once the display model is complete, the system 10 can be activated to display the location and its background from various viewing directions 15, thereby concealing the objects 12 located beneath the display device 20. In this case, an external object 40, such as a drone (which in this case does not belong to the system 10), a bird, a person, or the like, can be detected by a monitoring device 26 of the system 10. By determining the location of the external object 40 relative to the display device 20, the current viewing direction 15 of the external object 40 towards the display device 20 is determined and used as the new current viewing direction 15, with respect to which a representation of the background is calculated from the display model and displayed by means of the display device 20.

[0061] In the Fig. In the case shown in Figure 1, where the display device 20 is located in the field, the display model can be generated based on previously captured images of the location and its surroundings. If no objects 12 are present at the location when the images are captured, the image model can be used. Alternatively, if objects 12 are already present at the location when the images used for model generation are captured, the display model can be modified using image processing, including the use of generative AI tools, so that a representation corresponding to the surrounding structures is used at the location where the objects 12 are placed. In this case, this could be, for example, a grassy area or a grassy area with individual bushes 6.

[0062] Fig. 2. Schematic representations of basic types of display devices usable according to the disclosure 20. The display device 20 according to Fig. 2a) is a dome, in particular a geometric group, which is covered along its outer surface with the densely active elements 22. This also corresponds to the one in Fig. 1. Initial example of a display device shown in cross-section 20.

[0063] In Fig. 2b) The display device 20 is a tent which is fitted on different sides with the light-active elements 22. This is a simple polyhedron.

[0064] The exemplary embodiment of a display device 20 in Fig. 2c) is a cylindrical base body with a domed roof, wherein both the cylindrical outer wall and the domed roof are fitted with light-active elements 22.

[0065] The Fig. Figures 2d) and 2e) show a perspective and a cross-sectional view of an initial example in which the display device 20 is designed as a flexible cloth, the outer surface of which is fitted with light-activated elements 22 and which is draped over objects 12. Since the flexible cloth partially assumes the shape of the underlying objects and does not have a predetermined three-dimensional shape, iterative fine-tuning of the display model is particularly advantageous in this case and may also be more complex or involve more iterations than in the cases of rigid three-dimensional display devices. Fig. 2a) to 2c) each with a known form.

[0066] Fig. 3 A schematic representation of a process according to the disclosure. The process is divided into a preparation phase 200, a setup phase 202, a model generation phase 204, and an application phase 206. Although the four phases in Fig. Since the 3 are presented in a specific sequence, the model generation phase 204 can take place before, during, or after the preparation phase 202, depending on the circumstances. The preparation phase 200 can also, under certain circumstances, take place after the setup phase 202. In that case, the images of the environment are generated in the presence of the display device 20, and the display device 20 is then removed from the display model and replaced by structures adapted to the environment.

[0067] In preparation phase 200, the surroundings of the location are first recorded from one or more directions in step S100, and the images are transmitted to a control device 24 of the system 10 (step S102). The images are recorded from as many viewing angles as possible to cover the largest possible solid angle. If the display device is positioned against a background, such as a steep wall, a wall, a dense hedge, or the like, from which the display device 20 cannot be seen, this solid angle can be omitted.

[0068] During the setup phase, in step S104, one or more objects 12 are placed at the location. This step can be omitted if the object(s) 12 are already in place before the process starts. Once the object(s) 12 are in position, in step S106, the display device 20 is placed or draped over the object(s) 12. This can optionally be done before process step S100. The images captured in step S100 then show the display device 20 in its unprocessed state.

[0069] Once the images of the environment and the location with or without objects and / or display device 20 are available, a display model can be created in the model generation phase 204 in step S108, which on the one hand includes the representation of the location in the absence of objects 12 and on the other hand has knowledge about the shape of the display device 20 in order to ensure that the background, i.e. what should be visible from a viewing direction 15, is also displayed by the display device in the direction of the viewing direction 15.

[0070] For rigid display devices 20 with a known shape, this can be a one-shot procedure, i.e., without iterative improvement. However, verification using an external camera from at least one viewing direction 15, for which the display model is optimized, is recommended in order to correct displacements or links of the display device 20. For this purpose, the images from the external camera are transmitted to the control device 24 and compared with the expected image derived from the display model. The display model is then corrected or modified to minimize the deviations. This can be done for different viewing directions 15, so that the number model is optimized over a wider viewing angle range. This is represented by procedure step S110, which can also be executed one or more times with an arrow leading back to procedure step S108.The arrow pointing back indicates that the new data obtained in process step S110 are used to optimize and improve the display model in process step S108. In a one-shot process, process step S110 is omitted.

[0071] In the case of cloth-like flexible display devices 20 that are draped over objects 12, the shape of the display device or its light-active surface is not known in advance. In this case, it may be useful to initially start with, for example, a dome-shaped geometry of the display device 20 when generating the display model and to refine this in an iterative process in which external images from different viewing directions 15, together with information about the respective viewing direction 15, are transmitted to the control device 24 in order to be used to improve the display model.The final result, in which all deviations from different viewing directions 15 are minimized, is a display model that contains, on the one hand, a model of the background to be displayed at the location of the predetermined place and, on the other hand, learned information about the actual shape of the flexible display device draped over one or more objects 12.

[0072] After the display model has been generated, the display device 20 is put into operation (application phase 206). In this application phase 206, a momentary viewing direction is determined (process step S112), for example by detecting an external object 40, from whose viewing direction the display device 20 is to optically merge with its surroundings. If necessary, a fixed viewing direction 15 can also be specified, which is not changed, for example if a merge with the surroundings is only necessary or desired in a specific viewing direction 15 anyway.

[0073] In step S114, environmental parameters can be determined, such as ambient brightness, lighting conditions, or movements of the ground, for example, those caused by wind, which moves grass, branches, bushes, leaves, and the like. Such environmental parameters can be processed with the display model to adapt the display to current environmental parameters, thus making it appear more realistic and unobtrusive. In the exemplary embodiment following step S114, a representation is determined in step S116 to be displayed on the display device 20, which is done in process step S118. The display can remain static, or, alternatively, Fig.3. The process is repeated, indicated by arrows referring back to previous steps, with each repetition determining a current viewing direction and / or determining environmental parameters (steps S112, S114). This can also be done at different intervals. For example, with rapidly moving external objects, step S112 can be executed at a high frequency, while a significantly lower repetition frequency is sufficient for determining environmental parameters in step S114.

[0074] All features mentioned, including those discernible from the drawings alone as well as individual features disclosed in combination with other features, are considered essential to the invention, both individually and in combination. Inventive embodiments may be fulfilled by individual features or by a combination of several features. Reference symbol list 2 Gras 4 Tree 6 bushes 10 System 12 objects 15 View direction 20 Display device 22 light-active elements 24 Control device 26 Monitoring device 28 drone 30 Camera 32 light sensor 40 external object S100 pictorial capture of the environment S102 Transmission of images to the control device S104 Placing objects S106 Covering the objects with the display device S108 Creating a display model S110 iterative refinement of the display model S112 Determining a current viewing direction S114 Determination of environmental parameters S116 Determining a representation S118 Displaying the representation 200 Preparation phase 202 Setup phase 204 Model generation phase 206 Application phase

Images