Method and system for presenting a viewing-direction-dependent view

Abstract

The invention relates to a method and a system (10) for presenting a viewing-direction-dependent view for a predefined location. The predefined location is imaged together with its surroundings in the presence or absence of one or more objects (12) at the location from one or more viewing directions (15), one or more objects (12) at the predefined location being concealed by a three-dimensional display device (20). On the basis of the images captured in the preparation phase (200), a display model is calculated by means of which, for each of the predefined one or more viewing directions (15) or for each viewing direction (15) within the one or more predefined solid angles, the predefined location together with its surroundings and in the absence of the one or more objects (12), as seen from the particular viewing direction (15), can be presented using the light-active elements (22) of the display device (20). In a use phase (206), a current viewing direction (15) is determined, and a presentation of the predefined location together with its surroundings in the absence of the one or more objects (12) from the current viewing direction (15) is determined by means of the display model and is displayed by means of the light-active elements (22) of the display device (20) with respect to the current viewing direction.

Description

[0001] Method and system for displaying a view dependent on the viewing direction

[0002] Description

[0003] The invention relates to a method and a system for displaying a viewing direction-dependent view for a predetermined location when viewed from one or more predetermined viewing directions or one or more predetermined spatial angle ranges.

[0004] In many cases, it is desirable to manipulate the visual perception of a location, ideally without the manipulation being immediately noticeable, so that what is perceived blends into the location as naturally as possible. Applications for this can be aesthetic in nature, such as art installations, or representations that inform or influence the viewer, for example, by confusing, distracting, or deterring them from the location, or by attracting them to it. For instance, it could be deceptive to create the illusion that objects not actually present are there, or conversely, objects that are actually there could be concealed by presenting a view of the location that does not include the object(s), such as a background or foreground.

[0005] The targets of such manipulation of visual perception can include not only humans but also animals or artificial structures capable of visual perception, such as camera drones. These are referred to below as external observers, observers, or external objects. Examples of applications for distraction include activities in wildlife observation or hunting in the wild, to avoid disturbing the animals being observed or hunted. In nature observation, passive camouflage nets for people or observation devices are frequently used, but these are only suitable for a limited number of environments and are therefore not flexible. Actively adapting a view to a specific location can overcome this limitation. This can be expanded upon. For example, a view designed to attract animals for nature observation could include a feeding station, possibly with additional attractants.Visual effects or objects that appear dangerous may be used as deterrents.

[0006] When manipulating the visual perception of a place as realistically as possible, it must be taken into account that a place is a three-dimensional entity which in many cases is visible from different perspectives, including from different sides and possibly also from above, from the air.

[0007] Starting from this premise, the object of the present invention is to provide a means of optically manipulating views of a location from different viewing directions. This object is achieved by a method for displaying a viewing-direction-dependent view of a predetermined location when viewed from one or more predetermined viewing directions or one or more predetermined spatial angle ranges, comprising a preparation phase in which the predetermined location, in particular including its surroundings, is imaged from one or more viewing directions, wherein the one or more viewing directions for image acquisition cover the one or more predetermined viewing directions or the one or more predetermined spatial angle ranges, and a setup phase occurring before, during, or after the preparation phase in which the predetermined location is covered with a three-dimensional display device.which is equipped on its outer surface with a multitude of individually controllable light-active elements, a model generation phase taking place during or after the preparation phase, in which a display model is calculated in a control device based on the images captured in the preparation phase, with which a model view of the predetermined location, in particular including its surroundings, seen from the respective viewing direction, can be displayed for each of the predetermined one or more viewing directions or for each viewing direction within the one or more predetermined solid angles, by means of the light-active elements of the display device, and after completion of the preparation phase and the model generation phase an application phase in which a momentary viewing direction is determined, by means of the display model a model view of the predetermined location, in particular including its surroundings, related to the momentary viewing direction,The inventive method makes it possible to flexibly create a display model for any possible location and, depending on the viewing direction, to display views of the display model on the three-dimensional display device that are manipulated compared to a natural view of the location, appropriate to the location and, if applicable, its surroundings. For example, objects that would be perceived as foreign bodies at the location can be hidden by placing the display device over the objects and showing a view of the location without the objects. Such objects could be recording devices for nature observations, hunting shelters, or the like. Alternatively, objects that are not present at the location can be displayed in the view.

[0008] The invention is based on the fundamental idea of ​​providing a three-dimensional display device that can present a manipulated view of a location where it is placed. The manipulated view can include the surroundings of the location, which would otherwise be visible as the background and / or surface.

[0009] The display device is three-dimensional, which in the context of the present application means that its light-active surface is structured three-dimensionally, with light-active elements pointing in different directions. Within the scope of this disclosure, "light-active" means that the corresponding element is configured either to emit light itself or to change its reflection and transmission properties for available light, i.e., to manipulate light for the viewer.The light-active elements can be, for example, LEDs attached to a curved surface, such as in a hemispherical or spherical shape similar to the Las Vegas Sphere or a geodesic dome, in a size and shape appropriate to the location and any objects present there, on honeycomb-shaped or generally polyhedral surfaces, cylindrical surfaces, or on a flexible, in particular textile, carrier that can be draped over or around a three-dimensional object, so that parts of the light-active surface of the display device are visible from various, in particular all, directions, especially also from above.

[0010] Examples of light-active elements include self-illuminating displays, optical fibers, or light-reflecting elements with variable optical properties, especially E-Ink assemblies.

[0011] To facilitate transport, the display device can be modular, particularly with polyhedrally arranged surfaces that can be disassembled and reassembled. Flexible supports can be easily rolled or folded, enabling very simple and space-saving transport.

[0012] In some embodiments of display devices designed as flexible carriers, they are provided with an irregular outer boundary. This can be implemented, for example, in the form of a ragged, randomly shaped cloth without a clear outer edge. Such a deviation from a clear round or regular angular shape can help prevent the display device from being recognized as an artificial object as quickly due to its edge structure.

[0013] To achieve perspective-correct display for one or more viewing directions, given the three-dimensional structure of the light-activated elements, the first step is to capture images of the location where one or more objects are situated or will be placed, from one or more viewing directions. This can be done from multiple perspectives, for example, particularly from ground level, using one or more cameras from different angles, or even using a drone, which, in addition to ground-level side views, can also capture still images or video footage from various angles, including from above and at an oblique angle.

[0014] These images then serve as a reference upon which the display model is based. Ideally, the reference images should show the "empty" location without any objects placed there. This is automatically the case if the location is already empty when the reference images are taken, meaning there are no objects to be concealed. If objects are to be concealed at the location, these images show what the predetermined location would look like without the objects. If, however, the images are taken with objects already in place, an additional processing step using common image editing tools, possibly or preferably AI-supported, is possible to edit the captured images so that the objects in the reference images are replaced with image elements that match the surroundings, i.e., the background and foreground, in order to create the desired reference images of the "empty" location.

[0015] For example, if an object to be camouflaged is already in position and it's not possible to take a picture of the actual ground, the surroundings can be captured from different angles, for instance, using a drone. The object is then removed from the image, and the surroundings are filled using machine learning algorithms. This creates an image of the ground, or the entire environment, from different perspectives of the object, which is highly likely to resemble the actual ground or surroundings. Models with varying performance requirements can be used for this purpose. For example, if the algorithms need to run without a network connection, perhaps because the application is intended for use in the field without an internet connection, models with low hardware requirements are used. This is generally possible with convolutional neural networks (CNNs), global area networks (GANs), or diffusion models.Good results can be achieved if the model, for example a baseline LaMa model (Resolution-robust Large Mask Inpainting with Fourier Convolution), is pre-trained or post-trained with training data from diverse landscape environments such as forest, lake district, tundra, taiga, desert, etc. Using evaluation metrics like FID, LPIPS, and / or SSIM, a stable optimum can be established across all landscape types. With a realistic mask size of 20-40% for the environment image, times of less than 10 seconds per generated infill (image) can be achieved on simple edge hardware, such as a Raspberry Pi 5. On specialized hardware, such as NVIDIA® Jetson Nano™, times of less than one second are achieved. This chain will continue to evolve with improvements in computing power and algorithms, leading to technical changes in speed, power consumption, and quality.

[0016] The display model can then generate views of the "empty" location based on these reference images of the "empty" location, or integrate non-existent objects using image processing, optionally or preferably AI-supported. Such a display model makes it possible to represent the predetermined location and its surroundings, either empty or with objects projected into the location that are not actually there, in views from one or more desired perspectives—that is, from one side, from different sides, or even from above—on the display device.

[0017] When the display device shows objects that cast shadows, the shadows are generated using machine learning processing. In randomized test series conducted specifically for this purpose, shadows have proven to be a feature that is very quickly and reliably identifiable for the human observer. Since clouds constantly change the intensity of shadows, while the infill can be easily adjusted in brightness and color, the shadows must be recalculated depending on the brightness. This continuous calculation requires particularly efficient models on edge hardware. Several approaches can be implemented. The best results in natural landscapes are achieved with DMASNet and especially RRSGNet. The algorithms were implemented and trained using RdSOBA, DESOBA, and DESOBAv2.SSIS mask prediction allows for the creation of high-resolution shadows of subsequently added objects that appear natural to the human eye.

[0018] The display model also takes into account the shape of the display device in order to display the desired representation in perspective correctly for a given current viewing direction, even if the display device is not a flat object.

[0019] A three-dimensional object, such as a display device that projects an image of its surroundings, presents different views of the object and, if the object were transparent, different perspectives of the environment behind it, due to its three-dimensional shape. For viewers with slightly different viewing angles, a background projection that is correctly displayed for more than one viewer is impossible. The shape of the object to be camouflaged defines the possibility of correct display for different viewers. For multiple viewers with small angular differences, a background that approximates the actual background for both is generated. Interpolation can also be performed using generative machine learning algorithms, which interpolate the display for different viewers, making it appear as natural as possible.This method produces the least recognizable view for different observers from similar perspectives. The observer(s) and their viewing directions do not need to be known a priori; instead, a representation can be chosen for a viewing direction from which a potential observer is most likely to approach, for example, along a path. If the observers and their viewing directions are known, the representation can be adjusted, for example, to reflect the observer with the highest probability of recognition. The probability of recognition increases with proximity to the object being camouflaged and the observer's sensitivity. Sensitivity is derived from the assumed "visual acuity" of the observer. Once the observers are known, the representation changes from a static mode without known observers to a mode with known observers.The display is continuously recalculated and adjusted as the viewer's position changes.

[0020] Furthermore, the display model preferably takes into account the resolution achievable with the light-active elements of the display device. This saves computing power by generating the model view at the resolution of the display device, rather than at a higher resolution that would then need to be downscaled to the resolution of the display device. The view to be displayed can include a specific color scheme and pattern that matches the environment and / or background and can be represented by the display device with a granularity appropriate to the environment and / or background within its resolution.

[0021] In further embodiments, the display model can include a movement of the pattern or view to be displayed, corresponding to the movement of structures in the vicinity. Examples include trees, shrubs, or grasses moving in the wind. To monitor the movement of these structures, an external camera is directed at the surroundings, its images are analyzed for movement of the structures, and the image displayed by the device is moved accordingly. Predominant movement patterns or directions, such as those found in swaying branches, leaves, or grass, can serve as a model.

[0022] In some embodiments, during the application phase, a monitoring device monitors the environment of a predetermined location. Upon detection of an external object, particularly a moving or movable one, the device determines the object's current viewing direction relative to the location and, in particular, tracks the object's movement. An external object is potentially an external observer. For this purpose, various sensors are positioned around the display device to detect the environment, for example, visually, infrared, acoustically, and / or electromagnetically, enabling direction, angle, and / or distance determination, for example, via triangulation. Suitable monitoring devices are based, for example, on optical measuring instruments such as stationary or movable cameras, or on other measuring instruments such as lidar, radar, radio direction finding, thermal, or ultrasonic reflection measuring devices.Acoustic or motion sensors can also be used to signal when a moving external object, i.e., a potential observer, is in the vicinity of the display device. More precise location-sensitive sensors are then used to determine the current location of the external object. Acoustic sensors can, for example, have directional characteristics, or multiple microphones can be distributed around or attached to the display device, and the position of an external object can be determined by measuring the time of flight. External objects can include people, wild animals (including birds of prey), and inanimate objects such as vehicles, airplanes, helicopters, drones, etc.

[0023] The type of external object can provide information about the sensitivity of the observer or viewer and thus be relevant for generating the displayed image. Machine learning models can be trained and used to distinguish between biological observers such as humans or animals and technical observers such as cameras, LiDaR, radar, and IR, which differ, among other things, in appearance and movement patterns.

[0024] External objects can also be detected using cameras, whose video footage can be processed in real time. YOLOvl Os is one suitable tool for this. Using COCO weights and inference at 640-960 pixels yields good results. In scenarios with only vehicles and drones, retrained VisDrone and UAVDT models can deliver good results.

[0025] Acoustic detection is also suitable for vehicles and drones, among other applications. It can be implemented, for example, using an ME MS array and YAMNet, with direction determination via SRP-PHAT.

[0026] The monitoring device or devices can be arranged externally, but can also be arranged on the surface of the display device and directed towards the environment of the display device.

[0027] Especially with moving external objects, it is advantageous to continuously determine their position relative to the display device, for example by means of one or more monitoring devices, and to track the current viewing direction, on the basis of which the displayed image is calculated, so that the object is shown a background of the place where the display device is located that changes according to its changing viewing angle, whereby the ground can also be a background.

[0028] In some embodiments, the display model implements a predetermined or spontaneously assumed shape of the display device and is refined during the model generation phase by capturing test images of the predetermined location and its surroundings from one or more viewing directions, particularly using a drone or one or more external cameras. These test images are then compared with a target representation, and if the test images deviate from the target representation, the display model is adjusted to minimize these deviations. This iterative refinement may also reveal that certain parts of the display device are not visible in the images from the various viewing directions for which the display is being optimized.This can occur, for example, if the solid angle from which the display device is to be visible does not cover all sides of the display device, or if, for instance, a flexible, fabric-like support for the light-active elements has been draped around or over an object in such a way that an edge is partially folded over or creased and therefore perspectively obscured by an overlying layer. Such non-visible parts of the display device can then be masked in the display model and subsequently no longer controlled in order to represent part of the background of the location.

[0029] To recognize the three-dimensional structure of the display device, various solutions are conceivable. For example, a grid can be displayed on the fabric, i.e., a pattern of grid lines, perhaps with intersecting lines at right angles. Due to the three-dimensional structure of the display device, which might be a textile support draped over an object, the grid deforms in a characteristic way, similar to the display of 3D models. The deformations of the grid lines can then be used to determine the distribution of the display area in space. The line pattern can be chosen to be asymmetrical enough to clearly indicate the orientation of the display device. Instead of grids, other unambiguous patterns can also be displayed, such as randomly distributed lines, dot patterns, or the like, whose distribution is known in the flat state.Recognition without displaying a grid or other pattern on the display device is possible, for example, by equipping a drone with a lidar that allows distance measurement and can thus scan the shape of the display device from different sides.

[0030] To ensure the perspective accuracy of the display, test images from a greater number of different directions should be used for iterative improvement with an irregularly shaped display device than would be necessary with regularly shaped displays. Video recordings, for example from a drone orbiting the irregularly shaped display device and recording from different angles, are also suitable for fine-tuning.

[0031] In certain embodiments, the display of the predetermined location and its surroundings is adapted to changing lighting conditions, which are detected in particular by means of one or more light sensors or cameras arranged on or near the display device. Specifically, to adapt the display, the intensity and / or color of the light output of the light-active elements of the display device is adjusted to match the intensity and / or color of the natural reflection. The latter can be achieved by one or more external cameras that capture the surroundings and the display device, and whose images are evaluated to make the adjustment and blend the display with the surroundings.Alternatively, a model of the environment's reflectivity can be created using reference images of the surroundings or the ground and incorporated into the display model, using one or more internal or external cameras to detect changes in lighting conditions. Other environmental parameters that can influence the view of the location include wind direction and speed, which move blades of grass, bushes, or similar vegetation in a particular direction, according to the structures surrounding the display device, or ground-level thermals on hot days, where the hot ground heats and swirls the air near the ground, resulting in shimmering effects. For these cases, an anemometer and / or a ground-level thermometer may also be present, transmitting their readings to the control device.

[0032] In some embodiments, light sensors can detect deviations in brightness. Based on this observation, and especially its temporal development, it can be determined whether it is generally getting brighter or darker, whether, according to the sun's movement across the sky, a shadow is being cast by a nearby object, which may end after some time, or whether a cloud is only temporarily passing in front of the sun or uncovering it again.

[0033] To reduce or increase reflection, a layer can be used, for example, a film or textile whose transluminescence or surface properties can be adjusted. Films that become diffuse when an electrical voltage is applied are known. The desired diffusion level can be determined using a contrast sensor, such as a camera pointed at an object casting a shadow and the shadow cast by that object, measuring the contrast between the cast shadow and the adjacent unshaded area. Such a contrast sensor reacts very quickly to, for example, passing clouds and allows the rendering of the location to be adapted to changing lighting conditions.

[0034] In some embodiments, the display model is a generative image generation model that is iteratively adapted to the surrounding background using external images of the display device at a predetermined location. In one embodiment, this can be achieved by taking aerial photographs after an object has been placed and the display device positioned or draped over it. While the images are being taken by a drone or other camera devices, the display device is generatively adapted to the surrounding background. The generated images are repeatedly adjusted and evaluated until any discernible difference is minimized. Such a process can be performed using generative artificial intelligence. This can also be used for other applications where views of non-existent objects need to be displayed at a specific location.

[0035] The problem underlying the invention is also solved by a system for displaying a viewing-direction-dependent view for a predetermined location when viewed from one or more predetermined viewing directions or one or more predetermined solid angle ranges, comprising a three-dimensional display device which is equipped on its outside with a plurality of individually controllable light-active elements, a control device which is configured to control the plurality of light-active elements on the outside of the display device, and one or more stationary or mobile, ground-based or airborne, image acquisition devices which are configured to transmit images to the control device via a data communication interface, wherein the system is configured to carry out a method described above in accordance with the present disclosure or to be used in such a method.

[0036] The system thus achieves the same properties, characteristics, and advantages as the previously described method. The statements made regarding the components of the system in connection with the previously described method apply equally to these individual components.

[0037] In embodiments, the system comprises a monitoring device that is data-connected to the control device and configured to detect external objects, determine their viewing direction relative to the display device, and report this information to the control device. The monitoring device may, in particular, include optical, acoustic, lidar, radar, radio direction finding, ultrasonic, and / or motion sensors. Such active sensors are preferably used when it is determined or assumed that an observer cannot detect active sensors. For example, radar and lidar systems can increase the precision of measuring the angle and distance to the observer.

[0038] In further embodiments, the system also comprises one or more light sensors, light contrast sensors, or cameras arranged in, on, or near the display device and connected to the control device. The control device is configured to adapt the display of the predetermined location and its surroundings to changing light conditions detected by the light sensor(s), light contrast sensor(s), or camera(s). In various embodiments, the light-active elements are light-generating elements, in particular LEDs, LED assemblies, self-illuminating displays, or optical fibers, or light-reflecting elements with variable optical properties, in particular E-Ink assemblies. The latter have the advantage that they continue to function even in the event of a power failure and retain their color or pattern. This makes it possible, for example, to load a static image once and then use the display device in a decentralized manner.For example, it is possible to set the control to HOLD after initial programming, thus preventing any active changes to the display model. For this purpose, it is useful if the system can be used autonomously via its own battery power supply.

[0039] The light-activated elements can be applied to the display device as flexible or inflexible assemblies or components, optionally on both sides of the display device. These are designed to receive pulses, current, or other signals. They can have centralized or decentralized control chips or can be attached directly to a textile substrate without circuit boards.

[0040] In some embodiments, the display device is a pre-formed or flexible, in particular textile, planar carrier for the light-active elements. Flexible carriers are known, for example, from DE 10 2014 206 882 A1 or DE 10 2020 116 917 A1 as flexible surface lighting elements. Materials such as fabric, films, or other flexible materials, for example, woven, metal, or plastic protective coatings, comparable to chainmail, can be used as the base for the display device. The planar carrier can also be assembled from or be composed of combinable, for example, pre-assembled, sections. Such sections can be designed as strips or as individual, for example, rectangular or square pieces that are joined together at their edges and for this purpose have, in particular, mechanical and / or electrical connectors.

[0041] The display device can also be advantageously protected against water, sand, dirt, and other environmental influences. To adapt the display device to varying weather conditions or other factors requiring material adjustments during operation, it can be combined with additional materials, such as films, fabrics, nets, camouflage netting, or similar substances. This can be advantageous, for example, when the spaces between the display device's components need to be either light or dark, or when reflections from the display device need to be significantly reduced. Such composite materials can either be integrated directly into the display device or mechanically attached to it.

[0042] The display device can also include a flat film whose light transmission can be electrically controlled, and behind which a white surface is placed, so that in particular all shades of gray, shadows and / or darkenings can be produced.

[0043] Furthermore, the display device can have thermally active elements and / or a cooling system on its exterior and / or interior. These serve to adapt the display device to its environment, particularly in the infrared range. In this context, thermally active elements are those capable of altering the thermal radiation emitted by the display device. Examples include a layer of an IR-absorbing, but optically transparent material applied to the exterior of the display device, or coatings of such a material, such as domes, applied over the individual light-active elements. Such materials are known.

[0044] On the inside of the display device, the primary focus is on cooling, rather than heating, as the display device is exothermic and therefore tends to heat up relative to its surroundings. Cooling can be achieved through various methods. The heat generated at the location of the light-active elements on the outside of the display device can be dissipated via an internal layer of a metal mesh or braid, i.e., a material made of a thermally conductive metal with a high specific surface area. This can also be a tubular mesh in which the individual fibers are hollow. Alternatively or additionally, cooling tubes through which a cooling fluid flows can be provided on the inner wall of the display device.

[0045] In both cases, heat can be extracted, for example, via a heat pump, which is operated in such a way that it cools the metal mesh or the coolant and dissipates the extracted waste heat. This heat can be released, if present, at an unobserved rear of the display device, or transferred to a heat sink with high heat capacity, such as a geothermal probe into the ground, which already possesses a high heat capacity. A reflective coating on the inside of the display device can suppress heat loss from the interior to the outside. In the case of active thermal regulation or cooling, the system can be controlled via heat or...The display device has infrared sensors designed and aligned to detect the heat radiation of the environment and transmit it to the control device, the control device being designed and configured to regulate the cooling so that heat radiation from the display device is matched to the detected heat radiation of the environment.

[0046] To stabilize the three-dimensional structure of the display device, exemplary embodiments provide for internal air chambers that form a frame, ridges, or other shapes. Internal air chambers are barely detectable even using non-optical methods.

[0047] Further features of the invention will become apparent from the description of embodiments according to the invention, together with the claims and the accompanying drawings. Embodiments according to the invention may fulfill individual features or a combination of several features.

[0048] Within the scope of the invention, features marked with "in particular" or "preferably" are to be understood as optional features.

[0049] The invention is described below, without limiting the general concept of the invention, with reference to exemplary embodiments and the drawings, whereby for all details of the invention not explained in detail in the text, explicit reference is made to the drawings. The drawings show:

[0050] Fig. 1 is a schematic representation of a system according to the disclosure,

[0051] Fig. 2 schematic representations of basic types of display devices usable according to the disclosure

[0052] Fig. 3 is a schematic representation of a method according to the disclosure.

[0053] In the drawings, identical or similar elements and / or parts are provided with the same reference numbers, so that a re-presentation is omitted.

[0054] Fig. 1 shows a schematic representation of a system 10 according to the disclosure. It illustrates how objects 12 at a given location are concealed by a display device 20, which has a plurality of light-active elements 22 on its outer surface. These elements can be, for example, LEDs or LED assemblies, self-illuminating displays or optical fibers, or light-reflecting elements with variable optical properties, in particular E-Ink assemblies.

[0055] In this initial example, the display device 20 is set up in nature and surrounded by plants, including grass 2, trees 4 and bushes 6.

[0056] System 10 includes a control device 24, which in this initial example is arranged within the display device 20. The display device 20 also includes monitoring devices 26, for example optical or infrared cameras, light sensors 32, radar or lidar sensor devices, which are configured to monitor the environment of the display device 20. An external camera 30 is also attached to a nearby tree 4, positioned such that the display device 20 is within its field of view, along with part of the environment of the display device 20.

[0057] Furthermore, a drone 28 is shown in Fig. 1, which hovers obliquely above the display device and has a viewing direction 15 on the display device 20. Such a drone 28, like external cameras, can be part of the system 10 and be used to record the location and its surroundings in order to provide image data for generating a display model according to the invention or to refine the display model after its initial generation by checking whether the image displayed by the display device 20 from the viewing direction 15 for which it was generated actually looks as it should from this viewing direction 15. If this is not the case, this is reported back to the control device 24, which then adjusts the display model so that deviations between the intended image and the image actually displayed from the viewing direction 15 are minimized.

[0058] Once the display model is complete, the system 10 can be put into operation to display the location and its background from various viewing directions 15, thereby concealing the objects 12 located beneath the display device 20. In this case, an external object 40, for example, a drone (which in this case does not belong to the system 10), a bird, a person, or the like, can be detected by a monitoring device 26 of the system 10. By determining the location of the external object 40 relative to the display device 20, the instantaneous viewing direction 15 of the external object 40 towards the display device 20 is determined and used as the new instantaneous viewing direction 15, with respect to which a representation of the background is calculated from the display model and displayed by means of the display device 20.

[0059] In the case shown in Fig. 1, where the display device 20 is located in the field, the display model can be generated from previously captured images of the location and its surroundings. If no objects 12 are present at the location when the images are captured, the image can be used directly. Alternatively, if objects 12 are already present when the images used for model generation are captured, the display model can be modified using image processing, including the use of generative AI tools, so that a representation corresponding to the surrounding structures is inserted at the location where the objects 12 are placed. This could, for example, be a grassy area or a grassy area with individual bushes 6.

[0060] Fig. 2 schematically represents basic types of display devices 20 usable according to the disclosure. The display device 20 according to Fig. 2a) is a dome, in particular a geometric group, which is covered along its outer surface with the densely active elements 22. This also corresponds to the basic example of a display device 20 shown in cross-section in Fig. 1.

[0061] In Fig. 2b), the display device 20 is a tent which is fitted on different sides with the light-active elements 22. This is a simple polyhedron.

[0062] The embodiment of a display device 20 in Fig. 2c) is a cylindrical base body with a domed roof, wherein both the cylindrical outer wall and the domed roof are fitted with light-active elements 22.

[0063] Figures 2d) and 2e) show a perspective and a cross-sectional view of an initial example in which the display device 20 is designed as a flexible cloth, the outer surface of which is covered with light-activated elements 22 and draped over objects 12. Since the flexible cloth partially assumes the shape of the underlying objects and does not have a predetermined three-dimensional shape, iterative fine-tuning of the display model is particularly advantageous in this case and may also be more complex or involve more iterations than in the cases of the rigid three-dimensional display devices of Figures 2a) to 2c), each with a known shape.

[0064] Fig. 3 shows a schematic representation of a method according to the disclosure. The method is divided into a preparation phase 200, a setup phase 202, a model generation phase 204, and an application phase 206. Although the four phases are shown in a specific order in Fig. 3, the model generation phase 204 can take place before, during, or after the preparation phase 202, depending on the circumstances. Similarly, the preparation phase 200 can, under certain circumstances, take place after the setup phase 202. In that case, images of the environment are generated in the presence of the display device 20, and the display device 20 is subsequently removed from the display model and replaced by structures adapted to the environment.

[0065] In preparation phase 200, the surroundings of the location are first recorded from one or more directions in step S100, and the images are transmitted to a control device 24 of the system 10 (step S102). The images are recorded from as many viewing angles as possible to cover the largest possible solid angle. If the display device is positioned against a background, such as a steep wall, a wall, a dense hedge, or the like, from which the display device 20 cannot be seen, this solid angle can be omitted.

[0066] During the setup phase, in step S104, one or more objects 12 are placed at the location. This step can be omitted if the object(s) 12 are already in place before the process starts. Once the object(s) 12 are in position, in step S106, the display device 20 is placed or positioned over the object(s) 12. This can optionally be done before process step S100. The images captured in step S100 then show the display device 20 in its unprocessed state.

[0067] Once the images of the environment and the location with or without objects and / or display device 20 are available, a display model can be created in the model generation phase 204 in step S108, which on the one hand includes the representation of the location in the absence of objects 12 and on the other hand has knowledge about the shape of the display device 20 in order to ensure that the background, i.e. what should be visible from a viewing direction 15, is also displayed by the display device in the direction of the viewing direction 15.

[0068] For rigid display devices 20 with a known shape, this can be a one-shot procedure, i.e., without iterative improvement. However, verification using an external camera from at least one viewing direction 15, for which the display model is optimized, is recommended in order to correct displacements or links of the display device 20. For this purpose, the images from the external camera are transmitted to the control device 24 and compared with the expected image derived from the display model. The display model is then corrected or modified to minimize the deviations. This can be done for different viewing directions 15, so that the number model is optimized over a wider viewing angle range. This is represented by procedure step S110, which can also be executed one or more times with an arrow leading back to procedure step S108.The arrow pointing back indicates that the new data obtained in process step S1 10 are used to optimize and improve the display model in process step S108. In a one-shot process, process step S1 10 is omitted.

[0069] In the case of cloth-like flexible display devices 20 that are draped over objects 12, the shape of the display device or its light-active surface is not known in advance. In this case, it may be useful to initially start with, for example, a dome-shaped geometry of the display device 20 when generating the display model and to refine this in an iterative process in which external images from different viewing directions 15, together with information about the respective viewing direction 15, are transmitted to the control device 24 in order to be used to improve the display model.The final result, in which all deviations from different viewing directions 15 are minimized, is a display model that contains, on the one hand, a model of the background to be displayed at the predetermined location and, on the other hand, learned information about the actual shape of the flexible display device 20 draped over one or more objects 12. After the display model has been generated, the display device 20 is put into operation (application phase 206). In this application phase 206, a momentary viewing direction is determined (process step S1 12), for example, by detecting an external object 40 from whose viewing direction the display device 20 is to optically merge with its surroundings. If necessary, a fixed viewing direction 15 can also be specified, which is not changed, for example, if merging with the surroundings is only necessary or desired in a specific viewing direction 15 anyway.

[0070] In step S1 14, environmental parameters can be determined, such as ambient brightness, lighting conditions, or movements of the ground, for example, those caused by wind, which moves grass, branches, bushes, leaves, and the like. Such environmental parameters can be processed with the display model to adapt the display to current environmental parameters, thus making it appear more realistic and unobtrusive. In the exemplary embodiment following step S1 14, a representation is determined in step S1 16 to be displayed on the display device 20, which is done in process step S1 18. The display can remain static or, as indicated in Fig. 3 by arrows referring back to previous process steps, be repeated, with a momentary viewing direction being determined and / or environmental parameters being determined for each repetition (steps S1 12, S1 14).This can also be done at different intervals. For example, with rapidly moving external objects 40, process step S112 can be performed at a high frequency, while a significantly lower repetition frequency is sufficient for determining environmental parameters in step S1 14. All the features mentioned, including those that can be seen from the drawings alone as well as individual features disclosed in combination with other features, are considered essential to the invention, both individually and in combination. Inventive embodiments can be fulfilled by individual features or a combination of several features.

[0071] List of reference signs

[0072] 2 Gras

[0073] 4 Tree

[0074] 6 bushes

[0075] 10 System

[0076] 12 objects

[0077] 15 View direction

[0078] 20 Display device

[0079] 22 light-active elements

[0080] 24 Control device

[0081] 26 Monitoring device

[0082] 28 drone

[0083] 30 Camera

[0084] 32 light sensor

[0085] 40 external object

[0086] S100 pictorial capture of the environment

[0087] S102 Transmission of images to the control device

[0088] S104 Placing objects

[0089] S106 Covering the objects with the display device

[0090] S108 Creating a display model

[0091] S1 10 iterative refinement of the display model

[0092] S1 12 Determining a current viewing direction

[0093] S1 14 Determination of environmental parameters

[0094] S1 16 Determining a representation

[0095] S1 18 Displaying the representation

[0096] 200 Preparation phase

[0097] 202 Setup phase

[0098] 204 Model generation phase

[0099] 206 Application phase

Claims

Method and system for displaying a view dependent on the viewing direction 1. A method for displaying a viewing-direction-dependent view of a predetermined location when viewed from one or more predetermined viewing directions (15) or one or more predetermined solid angle regions, comprising a preparation phase (200) in which the predetermined location, in particular including its surroundings, is imaged from one or more viewing directions (15) (S100), wherein the one or more viewing directions (15) for image capture cover the one or more predetermined viewing directions (15) or the one or more predetermined solid angle regions, and a setup phase (202) occurring before, during or after the preparation phase (200), in which the predetermined location is covered with a three-dimensional display device (20) (S106) which is equipped on its outer surface with a plurality of individually controllable light-active elements (22). a model generation phase (204) occurring during or after the preparation phase (202), in which a display model (S108) is calculated in a control device (24) based on the images acquired in the preparation phase (200), with which a model view of the predetermined location, in particular including its surroundings, seen from the respective viewing direction (15), can be displayed for each of the predetermined one or more viewing directions (15) or for each viewing direction (15) within the one or more predetermined solid angles, by means of the light-active elements (22) of the display device (20), and after completion of the preparation phase (200) and the model generation phase (204) an application phase (206), in which a momentary viewing direction (15) is determined (S1 12), by means of the display model a model view of the predetermined location, in particular including its surroundings, related to the momentary viewing direction,determined (S1 16) and is displayed by means of the light-active elements (22) of the display device (20) (S1 18)., 2. Method according to claim 1, characterized in that during the application phase (206) an environment of the predetermined location is monitored by means of a monitoring device (26) and, upon detection of an external object (40), in particular a moving or movable object, a viewing direction (15) of the external object (40) to the location is determined as the instantaneous viewing direction (15) and, in particular, the instantaneous viewing direction (15) of a movement of the external object (40) is tracked.

3. Method according to claim 1 or 2, characterized in that, that the display model implements a predetermined or spontaneously assumed form of the display device (20) and is refined during the model generation phase (204) by taking test images of the representation of the predetermined location and its surroundings displayed by the display device (20) from one or more viewing directions (15), in particular by means of a drone (28) or by means of one or more external cameras (30), comparing the test images with a target representation and, if the test images deviate from the target representation, adjusting the display model so that deviations are minimized.

4. Method according to one of claims 1 to 3, characterized in that the representation of the predetermined location and its surroundings is adapted to changing light conditions, which are detected in particular by means of one or more light sensors (32) or cameras (30) arranged on or near the display device.

5. Method according to claim 4, characterized in that, to adapt the display, the intensity and / or color of the light output of the light-active elements (22) of the display device (20) is adapted to the intensity and / or color of the natural reflection.

6. Method according to one of claims 1 to 5, characterized in that the display model is a generative image generation model which is adapted to the surrounding background in an iterative process using external recordings of the display device at the predetermined location.

7. System (10) for displaying a viewing direction-dependent view for a predetermined location when viewed from one or more predetermined viewing directions (15) or one or more predetermined solid angle areas, comprising a three-dimensional display device (20) which is equipped on its outside with a plurality of individually controllable light-active elements (22), a control device (24) which is configured to control the plurality of light-active elements (22) on the outside of the display device (20), and one or more stationary or mobile, ground-based or airborne, image acquisition devices (28, 30) which are configured to transmit images to the control device via a data communication interface, wherein the system is configured to perform a method according to one of claims 1 to 7 or to be used in a method according to one of claims 1 to 7.

8. System (10) according to claim 7, further comprising a monitoring device which is data-connected to the control device (24) and is configured to recognize external objects (40) and to determine their direction of view (15) in relation to the display device (20) and to report this to the control device (24).

9. System (10) according to claim 8, characterized in that the monitoring device (26) comprises optical, acoustic, lidar, radar, radio direction finding, ultrasonic and / or motion sensors.

10. System (10) according to one of claims 7 to 9, characterized in that the system (10) further comprises one or more arranged in, on or near the display device (20) and The control device (24) comprises data-connected light sensors (32), light contrast sensors or cameras (30), wherein the control device (24) is configured to adapt the representation of the predetermined location and its surroundings to changing light conditions detected by the light sensor(s) (32) or cameras (30).

11. System (10) according to one of claims 7 to 10, characterized in that the light-active elements (22) are light-generating elements, in particular LEDs, LED assemblies, self-illuminating displays or optical fibers, or light-reflecting elements with variable optical properties, in particular E-Ink assemblies.

12. System (10) according to one of claims 7 to 11 , characterized in that the display device (20) is a preformed or a flexible, in particular textile, planar carrier of the light-active elements, which in particular has an irregular outer boundary.

13. System (10) according to one of claims 7 to 12, characterized in that the display device (20) comprises a planar film whose light transmission can be electrically controlled, and behind which a white area is placed.

14. System (10) according to one of claims 7 to 13, characterized in that the display device (20) has thermally active elements and / or a cooling system on its outside and / or inside.

Images