Method for creating projection on surface of real-world object

Abstract

The invention relates to a method for creating undistorted projections on the surface of real-world objects in the case of artistic lighting of buildings, structures and similar objects, and in the case of light markings at industrial facilities and on civil objects. The method comprises performing geodetic measurements of the objects and their spatial surroundings to create a vertical control network, performing photogrammetric measurements or laser scans of the real-world objects to be projected thereon and objects in their spatial surroundings, creating a 3D model of the real-world objects, during the rendering process, real-world objects are referenced to a given coordinate system, a projection zone is determined and calculated, the position of the projector is calculated, taking into account the position of an object in the space surrounding that blocks the path of the projection beam from the projector to the object on which it is to be projected. The method includes creating a model of a "source (projector)-screen" system using coordinates obtained in a 3D modeling process, transmitting a projected image onto a gobo picture, installing a projector according to coordinates of a virtual projector in the 3D model of the real-world object, placing the picture with the image into the projector, and projecting the image onto a surface of the real-world object.

Description

TECHNICAL FIELD

[0001] The present invention relates to a method for creating a distortion-free projection on the surface of a real object in the case of artistic lighting of architectural buildings, structures and similar objects, as well as in the case of light marking on industrial facilities and civil objects. BACKGROUND

[0002] DE 102005034990 A1 discloses a method for projecting an image onto the surface of a real object. The method involves using a plurality of projectors to project a plurality of digital images onto a projection surface (surfaces), wherein the projectors are tuned to different image planes, wherein "image elements" - pixels are simultaneously projected from different projectors into the area of the projection surface(s). With this display method, all areas of the projection surface illuminated by the projectors can be automatically adjusted to the best sharpness after calibration by an automatic focusing mechanism to correct the geometry and color, and to determine the sharpness value and the appropriate transformation when manipulating the intensity to be corrected. In this case, the calibration is a lengthy and complex process, and the image on the surface is not ultimately completely realistic.

[0003] RU 2433487 (published on November 10, 2011, Bulletin No. 31) discloses a method for projecting an image of a three-dimensional object onto the surface of a real object, which image is to be projected on the surface of the real object by measuring and / or photographing and / or laser scanning the real object, calculating the positioning of the projector, light source and viewer relative to the real object. In this method, a 3D model of the real and virtual objects is created and textured, and a virtual scene is constructed, including a virtual light source, a virtual projector, a 3D model of the real object (the position of the 3D model of the real object is the same as the position of the real object, light source and projector) and a 3D model of a virtual object. Programming of visualization of the virtual scene in real time from the point of view of the viewer's position, programming of real-time display of the obtained image on the surface of the model of the real object, programming of visualization of the resulting real-time scene from the point of view of the position of the projector, programming of the scene of the interactive part and behavior of the model. Then, the projector is arranged in the same real environment as the virtual scene, and the obtained image is displayed on the surface of the real object by means of the real projector. In some cases, due to inaccuracies in the initial measurements or in the calibration of the projector, it is necessary to adjust (reset) the projector in the real environment. After positioning the projector in the real environment, the projection image is additionally corrected by editing the 3D model of the real object to take into account the positioning of the projector. In cases where it is difficult to superimpose the final video image on the real object due to inaccuracies in the initial measurements or in the calibration of the projector, the projector is adjusted in the real environment by adjusting the projection image using 3D model editing tools.

[0004] The complexity of the methods described above stems from the adjustment of projector positioning and the duration of the determination process. This is due to the fact that their positioning is determined by a virtual 3D model of the object, and transferring the projector's position from the virtual model to the real object requires multiple iterations to search for the optimal location. Furthermore, the projected image is sometimes adjusted by editing the 3D model of the real object to account for the projector's placement. Digital projectors (see RU 2433487: Figures 3-5) do not allow for projecting images by avoiding locations with undesirable lighting (windows, galleries, holes, etc.). Video projectors are not designed for use in open spaces unless additional measures are taken to protect them from weather conditions—they are most often installed inside buildings or under canopies, which greatly limits the choice of their installation location.

[0005] Existing technology discloses methods for projecting images onto the surface of real objects using so-called gobo projectors—street projectors in the form of small LED devices—to project images of gobo slides onto any (preferably flat) surface, such as asphalt pavements, building walls, shopping mall floors, etc. The projector's gobo slides are typically thin discs of heat-resistant, dichroic glass with laser-engraved images, mounted in front of the light source (prior to the optics).

[0006] RU 2667346 (published September 18, 2018, Patent Publication No. 26) discloses a method for projecting an image onto the surface of a real object. This method involves determining the approximate location of a projection area and a gobo projector, calibrating the projector with a calibration gobo image, inserting the gobo image with a reference point, and projecting, thereby adjusting the projection size and focus. After setting the values ​​of the angular and focus, the distance from the gobo projector to the extreme point of the projection is measured. The projection of the calibration gobo is photographed from the gobo projector's mounting location and / or from the observation point of the projection. The distortion of the projection is calculated. A master template is created in a graphics editor based on the photograph of the calibrated gobo projection by removing all graphic information except for the graphic information in the projection area from the template image. The projection area is then distorted to ensure the possibility of linking the image to the reference point of the calibrated gobo projection. If necessary, the primary template is divided into a master template of the projected image and a mask template that blocks luminous flux from areas where light should not fall. Next, the master template is overlapped with the image used for projection, and if necessary, luminous flux cutoff areas are marked on the mask template. The master template with the image and the mask template with the luminous flux cutoff areas are distorted according to the original geometry and dimensions of the calibrated gobo. Next, a gobo image is created based on the master template with the image of the non-illuminated area for luminous flux cutoff. The gobo image is mounted in the gobo projector, and the gobo projector is guided and adjusted according to the previously made settings. The axis of the gobo with the image and the gobo mask relative to the luminous flux is adjusted, and the focus of the projected image is adjusted by shifting and tilting the lens.

[0007] The method described above for creating a gobo image is quite labor-intensive. It requires a lot of manual work to adjust the projector, perform necessary measurements and calculations, first create a master template, then create the master template and cutoff mask template, create and use a calibration gobo, and in some cases, a set of repetitive work is required to determine the new position of the projector and its adjustment. Summary of the Invention

[0008] The purpose of the method claimed is to ensure manufacturability by reducing the labor intensity of the process of creating a distortion-free projection on the surface of a real object.

[0009] The technical result of the claimed invention is to reduce the labor intensity of methods for creating distortion-free projections onto the surface of real objects by: creating a three-dimensional digital model (3D model) of the object and its spatial environment with given actual coordinates, without creating an additional virtual model of the object; performing post-field processing on the projection scene, taking into account the propagation of projection rays that prevent optical flow from the projector (multiple projectors) to the object and the object's position in the spatial environment; determining the optimal mounting position of the projector (multiple projectors) by modeling and comparing projection variables, without making further additional adjustments at the mounting location of the projection source; and creating a master image template and (if necessary) a mask template, the final gobo image.

[0010] Implementing the technical results by claiming protection involves: performing preliminary geodesic surveys of the real object and its spatial environment to create a vertical control network in which each point of the scanned object is bound to a coordinate system; performing a photogrammetric survey of the real object on which projection will be made, and its spatial environment, taking into account the location of objects that might obstruct projection, to subsequently determine the installation location of the projector; and creating a three-dimensional digital model (3D model) of the projected object in a given coordinate system, performing necessary calculations on it to obtain a distortion-free image, and determining the optimal installation location of the projector. A model of the "source (projector)-screen" system is created based on the coordinates obtained while working with the 3D model.

[0011] Depending on the composition of the object to be projected, photogrammetry or laser scanning is used to gather the necessary amount of information about the object and its spatial environment. Software products (e.g., ContextCapture or Metashape for photogrammetry, or Trimble Business Center for laser scanning) are used to link the information about the projected object to a given coordinate system. A three-dimensional digital model (3D model) is created in the specified coordinate system to determine the coordinates of the projector and projection plane in the adopted coordinate system, without creating an additional virtual model. Constructing a 3D model of a real object using photogrammetry (when creating a raster model based on pixels from a photographic image) or laser scanning (when creating a model based on a point cloud obtained by a laser beam reflected from the object's surface) is based on software such as ContextCapture, Metashape, or Trimble Business Center, which depends on the applied equipment and also involves identification based on geodesy.

[0012] Taking into account the use of ground control points and coordinates from the camera (GPS) center of photography, a digital 3D model is an accurate digital copy of a real object, which allows people to perform any necessary measurements of the object without having to repeatedly travel to the location of the real object.

[0013] For each scene of image projection at different locations where the projector is placed, virtual image projection is performed, lighting calculations are performed, distortion of the projected image is fixed to form a future distortion-free projection on the surface of a real object, computer processing of the obtained results is performed, and as a result, a view of the final image is created for further transmission to the gobo picture.

[0014] A distorted image of the future projection is created in a 3D digital model in a backfield manner. As a result, the projected image is obtained, and (if necessary) a light flux cutoff region is formed for further generation of gobo pictures and gobo masks based on, for example, borosilicate dichroic glass.

[0015] Based on the coordinates of the projector's installation location in the 3D model, simply mount the projector directly onto the real object, install the prepared images and (if necessary) gobo masks on it, and then adjust the device.

[0016] This method significantly reduces the labor intensity of creating distortion-free projections on the surface of real objects. This is because all operations are performed in a backfield manner without requiring physical access to the real object: virtual installation of the projector, determining the projection's location, modeling the luminous flux depending on the projector and its optics power, identifying obstacles in its path, handling possible changes in the projector's location, fixing distortion, determining necessary parameters, and processing all information. When working with 3D models in a backfield manner, no calibration gobo is required; all editing is done procedurally. Viewers are not confined to a specific location to view the projection and have unrestricted freedom of movement.

[0017] This method allows people to display a light spot on the projection surface (location) of a real object's 3D model based solely on the results of lighting calculations, according to the projector's selected (optimal) location, without having to travel to the real object's location. It also utilizes software products to handle the fixed distortion of the light spot on the projection surface (reverse distortion of the original image) by transmitting distortion parameters to the image of the future projection. The method creates a picture (e.g., made of borosilicate dichroic glass) based on the parameters of the virtual distortion of the original image to mount it in the projector, and precisely mounts the projector according to the coordinates of the virtual projector in the 3D model of the real object.

[0018] One variation of the claimed method of application is the use of active demarcation for both static images and images that move after the moving goods and the moving mechanism when an image is projected at a constant distance from the projector to the projection surface by means of a projector mounted on a moving mechanism. Detailed Implementation

[0019] The method is as follows. A vertical control network is created using geodetic equipment, in which each scan point of the object is bound to a coordinate system and used to bind the future digital 3D model to the real object. When creating a model based on a point cloud obtained using laser beams reflected from the object's surface, a raster model of the real object is created based on pixels from a photographic image or using laser scanning technology.

[0020] Software (such as ContextCapture, Metashape, and Trimble Business Center) is used to: create 3D models of objects, which are accurate digital copies of real objects and allow people to virtually measure objects virtually in a real coordinate system without having to go to the site; create digital models of spatial environments; edit real-world models and digital terrain models composed of hundreds of millions of triangles or point clouds; import, retouch, and export real-world models in various formats; and migrate data to the optimal format for processing.

[0021] For each visualization scene, a virtual projection of the image is performed, illumination calculations are conducted, distortion of the projected image is recorded, computer processing of the obtained results is performed, and a view of the final image is created for its further transmission to the gobo picture. A distorted image of the future projection is created in a 3D digital model in a backfield manner. As a result, the master image of the projection is obtained, and (if necessary) a luminous flux cutoff region is formed for subsequent generation of pictures and gobo masks based on, for example, borosilicate dichroic glass.

[0022] Modeling the projector mounting position in a 3D model depends on the projector's power and its optics mounted on surfaces inherently suited for projection within the spatial environment from the projected object to the given projection position, creating a virtual visualization of the luminous flux projection from the projector. Because the optimal options (no obstructions in the luminous flux path, the minimum possible tilt angle required for comfortable viewer image perception, and the projector's position closest to the projection location (within reasonable constraints)) have been determined, the projector mounting position is fixed and bound to the 3D model coordinates. To perform illumination calculations, beam trajectories are traced and their interactions with objects on those trajectories are calculated, and geometric optics problems are computed using software products such as ZEMAX and DIALUX packages, which allow modeling and analysis of various optical systems. A model of the "source (projector)-screen" system is created based on the coordinates obtained during 3D model creation. A uniform grid of points is created on the screen, its size corresponding to the image size. Based on the calculation results, two text files are created: one with the coordinates of the uniform grid in the gobo mask in the virtual projection, and the other with the coordinates of the distorted grid.

[0023] Install the projector directly on the real object at the coordinates in the 3D model, based on the projector's installation location. Place the prepared images and (if needed) a gobo mask on the projector, and then adjust the projector.

Claims

1. A digital method for creating a projected image on the surface of a real object, comprising: A real object is photographed or laser-scanned, and the projected image is to be projected onto the real object. Create a 3D model of the real object; Define and calculate the projection area; Calculate the positioning of the projector; The projected image is transmitted to the gobo picture; Install a projector on site; Install a Gobo picture with the projected image in each projector; The projected image is projected onto the surface of the real object; The method further includes: Perform geodetic surveys of the real objects and spatial environment to create a vertical control network; Perform photogrammetry or laser scanning on environmental objects in the spatial environment of the real object; and During rendering, the real-world object is linked to a given coordinate system; The positioning of the projector is calculated based on the position of at least one environmental object in the spatial environment that blocks the path of the projection ray from the projector to the real object. The coordinate system of the 3D model of the real object is used to create a model of the "source (projector)-screen" system; and The projector is loaded according to the coordinates of the projector in the 3D model of the real object.

2. The method of claim 1, further comprising installing a gobo mask in each of the projectors.