2d surface rendering from 2d images

Abstract

There is provided techniques for rendering a 2D surface from a set of 2D images with unknown scanning poses. The method is performed by a controller. The method comprises estimating a scene graph for the set of 2D images and, for each 2D image, a sensor pose from the set of 2D images. The sensor poses represent the scanning poses. The method comprises determining a set of view positions for the 2D images based on analysis of the scene graph and the sensor poses. The view positions are selected in accordance with a distance criterion with respect to the sensor pose and a connection criterion with respect to the scene graph. The method comprises generating a 2D model from the 2D images and the sensor poses. The 3D model is composed of voxels. The voxels comprise opacity information. The method comprises rendering the 2D surface based on raytracing in the 3D model from the view positions. One 2D surface is rendered per each view position. The opacity information defines how light is rendered on the 2D surface.

Description

[0001]2D SURFACE FROM 2D IMAGES TECHNICAL FIELD Embodiments presented herein relate to a method, a controller, a computer program, and a computer program product for rendering a two-dimensional (2D) surface from a set of 2D images with unknown scanning poses. BACKGROUND In general terms, a 3D model of a scene in a physical environment can be generated by scanning the physical environment. One time and cost-efficient approach is for a user to walk around in the physical environment, and to use a scanning device, such as a digital camera, to capture 2D images of the scene. The thus collected set of 2D images can then be used as input to a three-dimensional (3D) reconstruction algorithm which recovers the 3D scene geometry. Finally, the captured scene can be presented to the user by rendering the resulting 3D point cloud. In this respect, traditionally, visualization of 3D point clouds involves the use of a renderer. Some non-limiting examples of such renderers are Potree renderers and CloudCompare renderers. Such renderers allow for inspection, simple manipulation operations, and measurements to be made in the 3D point cloud. However, large 3D point clouds, as well as 3D point clouds generated from inside-out type scans (e.g., an indoor scan of a room in a building, or scans of multiple scans inside a building) are not easily inspected with Potree-type renderers. It is difficult to navigate in the indoor space of the 3D point cloud and downloading the entire 3D point cloud is inefficient and resource intensive, as well as requiring lots of data to be downloaded. In these scenarios, other renderers, such as skybox-type renderers, may be preferred. As a non-limiting example, skybox-type renderers are effective tools for viewing 3D scenes. Skybox-type renderers not only bring an immersive user experience, but are also particularly well suited for rendering of, and navigation in, indoor scenes. However, skybox-type renderers not only use 360-degree images but also requires view positions to be known. Neither are available from point clouds. This hinders skybox-type renderers to be used for rendering 3D point cloud. Hence, there is still a need for improved of 3D models generated from 2D images. SUMMARY An object of embodiments herein is to enable rendering of 3D models generated from 2D images where the above issues have been resolved, or at least mitigated or reduced. A particular object is to enable skybox-type renderers to be used in combination with 3D models generated from 2D images with unknown scanning poses. According to a first aspect there is presented a method for rendering a 2D surface from a set of ^^ 2D images ^ ^^௨^^௨ୀ^with unknown scanning poses. The method is performed by a controller. The method comprises estimating a scene graph ^^ for the set of 2D images ^ ^^௨^^௨ୀ^and, for each 2D image, a sensor pose ^ ^^௨^^௨ୀ^from the set of 2D images^^^௨^^௨ୀ^. The sensor poses represent the scanning poses. The method comprises determining a set of ^^ view positions ^ ^^^^^^ୀ^, where ^^ ^ ^^, for the 2D images ^ ^^௨^^௨ୀ^based on analysis of the scene graph ^^ and the sensor poses ^P௨^^௨ୀ^. The view positions ^ ^^^^^^ୀ^are selected in accordance with a distance criterion with respect to the sensor pose^^^௨^^௨ୀ^and a connection criterion with respect to the scene graph ^^. The method comprises generating a 2D model Ω from the 2D images ^ ^^௨^^௨ୀ^and the sensor poses ^P௨^^௨ୀ^. The 3D model Ω is composed of voxels. The voxels comprise opacity information. The method comprises rendering (S108) the 2D surface based on raytracing in the 3D model Ω from the view positions^^^^^^^ୀ^. One 2D surface is rendered per each view position ^ ^^^^^^ୀ^. The opacity information defines how light is rendered on the 2D surface. According to a second aspect there is presented a controller for rendering a 2D surface from a set of ^^ 2D images ^ ^^௨^^௨ୀ^with unknown scanning poses. The controller comprises processing circuitry. The processing circuitry is configured to cause the controller to estimate a scene graph ^^ for the set of 2D images ^ ^^௨^^௨ୀ^and,for each 2D image, a sensor pose^^^௨^^௨ୀ^ from the set of 2D images^^^௨^^௨ୀ^ . Thesensor poses represent the scanning poses. The processing circuitry is configured to cause the controller to determine a set of ^^ view positions ^ ^^^^^^ୀ^, where ^^ ^ ^^, for the 2D images^^^௨^^௨ୀ^based on analysis of the scene graph ^^ and the sensor poses^P௨^^௨ୀ^ . The view positions^^^^^^^ୀ^ are in accordance with a distancecriterion with respect to the sensor pose ^ ^^௨^^௨ୀ^and a connection criterion with respect to the scene graph ^^. The processing circuitry is configured to cause the controller to generate a 2D model Ω from the 2D images ^ ^^௨^^௨ୀ^and the sensor poses ^P௨^^௨ୀ^. The 3D model Ω is composed of voxels. The voxels comprise opacity information. The processing circuitry is configured to cause the controller to render the 2D surface based on raytracing in the 3D model Ω from the view positions^^^^^^^ୀ^. One 2D surface is rendered per each view position ^ ^^^^^^ୀ^. The opacity information defines how light is rendered on the 2D surface. According to a third aspect there is presented a controller for rendering a 2D surface from a set of ^^ 2D images ^ ^^௨^^௨ୀ^with unknown scanning poses. The controller comprises an estimate module configured to estimate a scene graph ^^ for the set of 2D images ^ ^^௨^^௨ୀ^and, for each 2D image, a sensor pose ^ ^^௨^^௨ୀ^from the set of 2D images ^ ^^௨^^௨ୀ^. The sensor poses represent the scanning poses. The controller comprises a determine module configured to determine a set of ^^ view positions ^ ^^^^^^ୀ^, where ^^ ^ ^^, for the 2D images ^ ^^௨^^௨ୀ^based on analysis of the scene graph the sensor poses ^P௨^^௨ୀ^. The view positions ^ ^^^^^^ୀ^are selected in accordance with a distance criterion with respect to the sensor pose ^ ^^௨^^௨ୀ^and a connection criterion with respect to the scene graph ^^. The controller comprises a generate module configured to generate a 2D model Ω from the 2D images ^ ^^௨^^௨ୀ^and the sensor poses ^P௨^^௨ୀ^. The 3D model Ω is composed of voxels. The voxels comprise opacity information. The controller comprises a render module configured to render the 2D surface based on raytracing in the 3D model Ω from the view positions ^ ^^^^^^ୀ^. One 2D surface is rendered per each view position ^ ^^^^^^ୀ^. The opacity information defines how light is rendered on the 2D surface. According to a fourth aspect there is presented a computer program for rendering a 2D surface from a set of 2D images with unknown scanning poses. The computer program comprises computer code which, when run on processing circuitry of a controller, causes the controller to perform actions. One action comprises the controller to estimate a scene graph ^^ for the set of 2D images ^ ^^௨^^௨ୀ^and, for each 2D image, a sensor pose ^ ^^௨^^௨ୀ^from the set of 2D images ^ ^^௨^^௨ୀ^. The sensor poses represent the scanning poses. One action comprises the controller to determine a setof ^^ view positions^^^^^^^ୀ^ , where ^^ ^ the 2D images^^^௨^^௨ୀ^ based on analysisof the scene graph ^^ and the sensor poses ^P௨^^௨ୀ^. The view positions ^ ^^^^^^ୀ^are selected in accordance with a distance criterion with respect to the sensor pose^^^௨^^௨ୀ^and a connection criterion with respect to the scene graph ^^. One action comprises the controller to generate a 2D model Ω from the 2D images^^^௨^^௨ୀ^and the sensor poses ^P௨^^௨ୀ^. The 3D model Ω is composed of voxels. The voxels comprise opacity information. One action comprises the controller to render the 2D surface based on raytracing in the 3D model Ω from the view positions ^ ^^^^^^ୀ^. One 2D surface is rendered per each view position ^ ^^^^^^ୀ^. The opacity information defines how light is rendered on the 2D surface. According to a fifth aspect there is presented a computer program product comprising a computer program according to the fourth aspect and a computer readable storage medium on which the computer program is stored. The computer readable storage medium could be a non-transitory computer readable storage medium. Advantageously, these aspects enable skybox-type renderers to be used in combination with 3D models generated from 2D images with unknown scanning poses. Advantageously, these aspects bridge the gap between 3D models generated from 2D images with unknown scanning poses and skybox-type renderers. This allows large 3D models to be exposed to the user by the efficient and easy to use skybox-type renderers. Advantageously, these aspects ensure that the rendered 2D surfaces are well- positioned to allow inside-out viewing of the 3D model, and that the views are of sufficient quality to serve for inspection, measurement, and presentation tasks. Other objectives, features and advantages of the enclosed embodiments will be apparent from the following detailed disclosure, from the attached dependent claims as well as from the drawings. Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a / an / the element, component, means, module, step, etc." are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, module, step, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated. BRIEF DESCRIPTION OF THE DRAWINGS The inventive concept is now described, by way of example, with reference to the accompanying drawings, in which: Fig.1 is a block diagram of a controller according to an embodiment; Fig.2 is a flowchart of methods according to embodiments; Fig.3 schematically illustrates a floor plan type view of a scene according to an embodiment; Fig.4 schematically illustrates an image rendering environment according to an embodiment; Fig.5 schematically illustrates a camera frustum according to an embodiment; Fig.6 is a schematic diagram showing structural units of a controller according to an embodiment; Fig.7 is a schematic diagram showing functional modules of a controller according to an embodiment; and Fig.8 shows one example of a computer program product comprising computer readable storage medium according to an embodiment. DETAILED DESCRIPTION The inventive concept will now be described more fully hereinafter with reference to the accompanying drawings, in which certain embodiments of the inventive concept are shown. This inventive concept may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the inventive concept to those skilled in the art. Like numbers to like elements throughout the description. Any step or feature illustrated by dashed lines should be regarded as optional. As noted above there is still a need for improved rendering of 3D models generated from 2D images. The embodiments disclosed herein therefore relate to techniques for rendering a 2D surface from a set of 2D images with unknown scanning poses. In order to obtain such techniques there is provided a controller 100, 600, 700, a method performed by the controller 100, 600, 700, a computer program product comprising code, for example in the form of a computer program, that when run on a controller 100, 600, 700, causes the controller 100, 600, 700 to perform the method. Fig. 1 is a block diagram of a controller 100 according to an embodiment. The controller 100 comprises a calculate poses block 110, a determine view positions block 120, a generate 3D model block 130, a synthesize images block 140, and a generate cubemaps block 150. The controller 100 in accordance with Fig.1 takes as input an unstructured set of ^^ 2D images ^J௨^^௨ୀ^, typically consisting of hundreds 2D images to a few thousand 2D images. The controller 100 in accordance with Fig.1 outputs a set of cubemaps Γ and view positions ^^. The number of view positions ^^ depends on the application and the scanned area, but it is typically in the range 10- 100. Each cubemap ^^^is associated with a position ^^^. This output can be used to render a 2D surface. In some examples, this is achieved by estimating a scene graph ^^ and sensor poses ^^ from the set of 2D images, determining view positions ^^, generating a 3D model Ω, synthesizing cubemap images ^^ and depth-maps ^^, and then generating the cubemaps Γ in accordance with the blocks 110:150 in Fig.1. Further details of the blocks 110:150 will be disclosed below in conjunction with the description of methods for rendering a 2D surface from a set of ^^ 2D images. Fig. 2 is a flowchart illustrating embodiments of methods for rendering a 2D surface from a set of ^^ 2D images ^ ^^௨^^௨ୀ^with unknown scanning poses. In some examples, the set of 2D images^^^௨^^௨ୀ^is a set of unstructured 2D images; i.e., the 2D images do not occur in the order they were captured and / or do not comprise any information identifying the order the 2D images were captured. The methods are performed by the controller 100, 600, 700. The are advantageously provided as computer programs 820. S102: The controller 100, 600, 700 estimates a scene graph ^^ for the set of 2D images ^ ^^௨^^௨ୀ^. The controller 100, 600, 700 estimates, for each 2D image, a sensor pose ^ ^^௨^^௨ୀ^from the set of 2D images ^ ^^௨^^௨ୀ^. The sensor poses represent the scanning poses. S104: The controller 100, 600, 700 determines a set of ^^ view positions ^ ^^^^^^ୀ^, where ^^ ^ ^^, for the 2D images ^ ^^௨^^௨ୀ^based on analysis of the scene graph ^^ andthe sensor poses^P௨^^௨ୀ^ . The view positions^^^^^^^ୀ^ are selected in accordance with adistance criterion with respect to the sensor pose ^ ^^௨^^௨ୀ^and a connection criterion with respect to the scene graph ^^. S106: The controller 100, 600, 700 generates a 3D model Ω from the 2D images ^ ^^௨^^௨ୀ^and the sensor poses ^P௨^^௨ୀ^. The 3D model Ω is composed of voxels. The voxels comprise opacity information. S108: The controller 100, 600, 700 renders the 2D surface based on raytracing in the 3D model Ω from the view positions ^ ^^^^^^ୀ^. One 2D surface is rendered per each view position^^^^^^^ୀ^. The opacity information defines how light is rendered on the 2D surface. Embodiments relating to further details of rendering a 2D surface from a set of 2D images with unknown scanning poses as performed by the controller 100, 600, 700 will now be disclosed. Continued reference is here made to Fig.1. In general terms, step S102 can be implemented by the block 110. Further aspects of how the controller 100, 600, 700 may estimate the sensor poses^^^௨^^௨ୀ^as in step S102 will be disclosed next. In some embodiments, the sensor poses ^ ^^௨^^௨ୀ^are estimated by applying structure- from-motion processing to the set of 2D images^^^௨^^௨ୀ^. Non-limiting examples of structure-from-motion algorithms are COLMAP and OpenMVG. In general terms, by applying a structure-from-motion algorithm, for each 2D image ^ ^^௨^^௨ୀ^a sensor poses ^ ^^௨^^௨ୀ^can be obtained as a six-dimensional vector capturing position and orientation in 3D space of the scanning That is, one six-dimensional vector capturing position and orientation in 3D space of the scanning device is obtained for each 2D image ^ ^^௨^^௨ୀ^. Further, in the process of estimating the sensor poses ^ ^^௨^^௨ୀ^, the structure-from-motion algorithm also calculates and outputs the scene graph S. The scene graph comprises nodes and edges, where the edges connect the nodes. The nodes represent the images and the edges represent connections, or relations, between the images. In general terms, step S104 can be implemented by the block 120. Further aspects of how the controller 100, 600, 700 may determine the set of ^^ view positions ^ ^^^^^^ୀ^as in step S104 will be disclosed next. Intermediate reference is here made to Fig.3. Fig.3 schematically illustrates a floor plan type view of a scene 300. The scene 300 is represented by two rooms in where two objects 330a, 330b are placed. The objects 330a, 330b have been scanned by a scanning device from different positions, hereinafter referred to as scanning device positions, or 3D positions 310. The different 3D positions 310 can, for example, be obtained by applying structure-from-motion processing to the set of 2D images ^ ^^௨^^௨ୀ^. In Fig.3 is also illustrated a set of ^^ ൌ 6 view positions ^ ^^^^^^ୀ^320. As can be seen, there are much fewer view positions 320 than 3D positions 310. One reason for this is that if the 3D positions 310 would have been used as view positions 320, there would typically be too many view positions 320 and typically at least some of the view positions 320 would be too close to the objects 330a, 330b. Therefore, a position selection-and-refinement approach can be used to select fewer, centrally located view positions ^ ^^^^^^ୀ^that sufficiently well can capture the scanned scene 300. According to a first example, the view positions ^ ^^^^^^ୀ^are selected based on the sensor poses ^ ^^௨^^௨ୀ^, and scene graph ^^. A first view position ^^^is selected among the sensor poses ^ ^^௨^^௨ୀ^, where the first view position ^^^has largest connectedness in the scene graph ^^. Therefore, in some embodiments, determining view position ^^^among the sensor poses^^^௨^^௨ୀ^comprises, according to the connection criterion, selecting the view position ^^^as the 3D position having highest connectivity in the scene graph ^^. Then, a first exclusion circle of radius ^^ (where, according to an example, ^^ ൌ 2m) is placed at the first view position ^^^. That is, in some embodiments, determining view position ^^^among the sensor poses^comprises, according to the distance criterion, defining an exclusion circle placed at the selected 3D position and within which circle no other view position can be located. The second view position ^^ଶis then selected as the 3D position outside the first exclusion circle that has largest connectedness in the scene graph ^^. Then, a second exclusion circle is placed on the second view position, and the process repeats until there are no more sensor poses ^ ^^௨^^௨ୀ^outside exclusion circles. That is, in some embodiments, the determining of location and the defining of an exclusion circle is repeated to determine further locations and defining further exclusion circles until there are no more sensor poses ^ ^^௨^^௨ୀ^outside any of the exclusion circles. According to a second example, the view positions ^ ^^^^^^ୀ^are selected using a clustering algorithm. That is, in some embodiments, determining view position ^^^among the sensor poses^^^௨^^௨ୀ^comprises applying a clustering algorithm to cluster the 3D positions into 3D position clusters. Non-limiting examples of clustering algorithm are k-means clustering and hierarchical clustering. The clustering algorithm is applied to the 3D positions of the sensor poses ^ ^^௨^^௨ୀ^. A two-step approach can be used to guarantee that the selected view ^ ^^^^^do not end ^ୀ^up inside one of the objects 330a, 330b in the scene 500 and that a large part of the scene 300 can be seen from each selected view position ^ ^^^^^^ୀ^. The method according to the first example is then repeated per cluster (i.e., for a respective subset of the sensor poses^^^௨^^௨ୀ^). That is, in some embodiments, the determining of location and the defining of an exclusion circle is performed for the sensor poses ^ ^^௨^^௨ୀ^per each of the 3D position clusters until there are no more sensor poses ^ ^^௨^^௨ୀ^outside any of the exclusion circles in each of the 3D position clusters. After selecting the view positions ^ ^^^^^^ୀ^, the altitude of all view positions ^ ^^^^^^ୀ^is set to an average level ^^̅, defined by the average elevation of the selected view positions ^ ^^^^^^ୀ^. That is, where each view position ^^^can be written in the form ^^^^ ^^, ^^, ^^ ൌ ^^̅^^^ୀ^. In other words, in some embodiments, each view position ^^^comprises an elevation value, and the elevation value for each of the view positions ^ ^^^^^^ୀ^is set to an average elevation ^^̅ of the view positions ^ ^^^^^^ୀ^. In general terms, step S106 can be by the block 130. Further aspects of how the controller 100, 600, 700 may generate the 3D model Ω from the 2D images^^^௨^^௨ୀ^ and the sensor poses^P௨^^௨ୀ^ as in step S106 will be disclosed next.Based on the set of 2D images ^ ^^௨^^௨ୀ^and their sensor poses ^ ^^௨^^௨ୀ^, a dense 3D model Ω of the scene 300 is learned. This 3D model Ω allows for generating novel- views in the scene 300. The 3D model Ω can be learned from the set of 2D images ^ ^^௨^^௨ୀ^and their sensor poses ^ ^^௨^^௨ୀ^using Neural Radiance Field (NeRF) processing or plenoptic voxels (Plenoxels) type approaches. In general terms, step S108 can be implemented by the block 140. Further aspects of how the controller 100, 600, 700 may renders the 2D surface as in step S108 will be disclosed next. In some embodiments, the 2D surface represents sides of a cubemap ^Γ^^^^ୀ^, with one cubemap Γ^per each view position ^^^. In general terms, the obtained view position ^ ^^^^^^ୀ^are not associated with any particular images in set of 2D images ^ ^^௨^^௨ୀ^, for example due to the change in elevation. Moreover, the field-of-view of the 2D images might not correspond to any cubemap side. The learned 3D model Ω is therefore used to generate 6 cubemap images ^^^(see, Fig.4) for each view position ^^^, creating a complete view of the scene as seen from view position ^^^. This can be achieved by re-projection of the 3D model Ω to the cubemap image based on each given view position ^^^. In general terms, generation of the cubemaps ^Γ^^^^ୀ^can be implemented by the block 150. Further aspects of how the cubemaps^Γ^^^^ୀ^can generated will be disclosed next. Together with cubemap images ^^^, the dense model Ω also generates depth-maps ^^^associated for the view position ^^^, with one depth-map per viewing direction, i.e., 6 in total per each view position ^^^. A cubemap ^^^ൌ ^ ^^^^ can be defined as a 360- degree composite image consisting of six depth-maps ^^^corresponding to the up, left, forward, right, behind, down directions, and hence where ^^^ൌ^^^^,௨^, ^^^,^, ^^^,ଽ^,, ^^^,^଼^, ^^^,.ଽ^, ^^^,ௗ^௪^^. For example, for view position ^^^, raytracingcan be performed for a respective sub-subset ^^^ᇱof voxels of the subset Π^. In general terms, the known distance from the ^^^to the surface (i.e., the distance between the camera at a given view position and a surface in the scene 300) can be used for this purpose. A cubemap Γ^associated with a particular view position ^^^can therefore, alternatively, be defined as Γ^ൌ ^ ^^^, ^^^^, where each side of each cubemap Γ^has its own assigned cube image and depth-map, generated from Ω. That is, where ^^^ൌ{ ^^^,௨^, ^^^,^, ^^^,ଽ^,, ^^^,^଼^, ^^^,ିଽ^, ^^^,ௗ^௪^}. With already calculated^^^^, ^^^^^^ୀ^the user can explore the digitalized 3D environment by means of a rendering environment, such as a skybox rendering environment, a Potree rendering environment, or a CloudCompare rendering environment. Therefore, in some examples, the 2D surface is rendered in a skybox rendering environment, a Potree rendering environment, or a CloudCompare rendering environment. In Fig.4 is illustrated an example image rendering environment 400 (Fig.4(a)), where one individual image ^^^,^୧୰ൌ ^ ^^^,ିଽ^, ^^^,^, ^^^,ାଽ^, ^^^,^଼^, ^^^,^୮, ^^^,^୭^୬^ of a 2D panoramic image 410 (Fig.4(b)) is rendered on each side of the image rendering environment 400. This is the type of image set that a skybox-rendered system uses to display all views. Thereby, with already calculated ^ ^^^, ^^^^^^ୀ^the user can explore the digitalized 3D environment by means of skybox viewer. The skybox rendering environment is represented by a set of ^^ virtual cubes and virtual cameras. The virtual camera is located at the center of the virtual cube. In this respect, a camera frustum (see, Fig.5) can be specified for each cubemap side. In particular, in some embodiments, the sub-subset Π^ᇱof voxels for view position t୩and a side ^^^,^of the cubemap Γ^is defined by a view frustum at the view position t୩and directed towards the side ^^^,^. The sides ^^^,^of the virtual cube are textured using the cubemaps. Each of the sides ^^^,^in the cubemap Γ^might represents a respective depth-map ^^^. The camera frustum 500 generally has a horizontal and a vertical field-of-view of 90 degrees. In a cubemap ^^, rotation matrices ^^^,^୧୰would be created for directions forward (or 0), right (or +90), behind (or 180), left (or -90), up, and down (see, Fig.4). In Fig.5 is illustrated a rendering environment in terms of a virtual cube 500 where a virtual camera 510 is placed at the center of the virtual cube 500, and where the virtual camera 510 has a camera frustum 520 marked in solid lines. Parts of two cube sides 530a, 530b are textured with a cubemap, and the virtual camera 510 sees a subset of the cubemap (shown by . Projecting that subset of the cubemap onto the virtual camera’s image plane creates a 2D view for the user. The user can switch to different views (such as different skyboxes). In a client-server system, when the user makes an attempt to navigate to the ^^-th skybox view, for example, the server transfers a set of images and depth maps Γ^ൌ ^ ^^^, ^^^^ that should be projected on the sides of ^^-th skybox. The client performs rendering operation and displays the scene on the screen. Fig. 6 schematically illustrates, in terms of a number of structural units, the components of a controller 600 according to an embodiment. Processing circuitry 610 is provided using any combination of one or more of a suitable central processing unit (CPU), multiprocessor, microcontroller, digital signal processor (DSP), etc., capable of executing software instructions stored in a computer program product 810 (as in Fig.8), e.g. in the form of a storage medium 630. The processing circuitry 610 may further be provided as at least one application specific integrated circuit (ASIC), or field programmable gate array (FPGA). Particularly, the processing circuitry 610 is configured to cause the controller 600 to perform a set of operations, or steps, as disclosed above. For example, the storage medium 630 may store the set of operations, and the processing circuitry 610 may be configured to retrieve the set of operations from the storage medium 630 to cause the controller 600 to perform the set of operations. The set of operations may be provided as a set of executable instructions. Thus the processing circuitry 610 is thereby arranged to execute methods as herein disclosed. The storage medium 630 may also comprise persistent storage, which, for example, can be any single one or combination of magnetic memory, optical memory, solid state memory or even remotely mounted memory. The controller 600 may further comprise a communications (comm.) interface 620 at least configured for communications with other entities, functions, nodes, and devices. As such the communications interface 620 may comprise one or more transmitters and receivers, comprising analogue and digital components. The processing circuitry 610 controls the general operation of the controller 600 e.g. by sending data and control signals to the communications interface 620 and the storage medium 630, by receiving data and reports from the communications interface 620, and by retrieving data and instructions from the storage medium Other components, as well as the related functionality, of the controller 600 are omitted in order not to obscure the concepts presented herein. Fig. 7 schematically illustrates, in terms of a number of functional modules, the components of a controller 700 according to an embodiment. The controller 700 of Fig.7 comprises a number of functional modules; an estimate module 710 configured to perform step S102, a determine module 720 configured to perform step S104, a generate module 730 configured to perform step S106, and a render module 740 configured to perform step S108. The controller 700 of Fig.7 may further comprise a number of optional functional modules, as represented by functional module 750. In general terms, each functional module 710:750 may in one embodiment be implemented only in hardware and in another embodiment with the help of software, i.e., the latter embodiment having computer program instructions stored on the storage medium 630 which when run on the processing circuitry makes the controller 600 perform the corresponding steps mentioned above in conjunction with Fig 7. It should also be mentioned that even though the modules correspond to parts of a computer program, they do not need to be separate modules therein, but the way in which they are implemented in software is dependent on the programming language used. Preferably, one or more or all functional modules 710:750 may be implemented by the processing circuitry 610, possibly in cooperation with the communications interface 620 and / or the storage medium 630. The processing circuitry 610 may thus be configured to from the storage medium 630 fetch instructions as provided by a functional module 710:750 and to execute these instructions, thereby performing any steps as disclosed herein. The controller 100, 600, 700 may be provided as a standalone device or as a part of at least one further device. A first portion of the instructions performed by the controller 100, 600, 700 may be executed in a first device, and a second portion of the of the instructions performed by the controller 100, 600, 700 may be executed in a second device; the herein disclosed embodiments are not limited to any particular number of devices on which the instructions performed by the controller 100, 600, 700 may be executed. Hence, the methods according to the herein disclosed embodiments are suitable to be performed by a controller 100, 600, 700 residing in a cloud computational environment. Therefore, although a single processing circuitry 610 is illustrated in Fig.6 the 610 may be distributed among a plurality of devices, or nodes. The same applies to the functional modules 710:750 of Fig.7 and the computer program 820 of Fig.8. Fig. 8 shows one example of a computer program product 810 comprising computer readable storage medium 830. On this computer readable storage medium 830, a computer program 820 can be stored, which computer program 820 can cause the processing circuitry 610 and thereto operatively coupled entities and devices, such as the communications interface 620 and the storage medium 630, to execute methods according to embodiments described herein. The computer program 820 and / or computer program product 810 may thus provide means for performing any steps as herein disclosed. In the example of Fig.8, the computer program product 810 is illustrated as an optical disc, such as a CD (compact disc) or a DVD (digital versatile disc) or a Blu-Ray disc. The computer program product 810 could also be embodied as a memory, such as a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or an electrically erasable programmable read-only memory (EEPROM) and more particularly as a non-volatile storage medium of a device in an external memory such as a USB (Universal Serial Bus) memory or a Flash memory, such as a compact Flash memory. Thus, while the computer program 820 is here schematically shown as a track on the depicted optical disk, the computer program 820 can be stored in any way which is suitable for the computer program product 810. The inventive concept has mainly been described above with reference to a few embodiments. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the inventive concept, as defined by the appended patent claims.

Claims

1. A method for rendering a two-dimensional, 2D, surface from a set of ^^ 2D images ^ ^^௨^^௨ୀ^with unknown scanning poses, the method being performed by a controller (100, 600, 700), the method comprising: estimating (S102) a scene graph ^^ for the set of 2D images^^^௨^^௨ୀ^and, for each 2D image, a sensor pose ^ ^^௨^^௨ୀ^from the set of 2D images ^ ^^௨^^௨ୀ^, wherein the sensor poses represent the scanning poses;determining (S104) a set of ^^ view positions^^^^^^^ୀ^(320), where ^^ ^ ^^, for the 2D images ^ ^^௨^^௨ୀ^based on analysis of the scene ^^ and the sensor poses^P௨^^௨ୀ^, wherein the view positions ^ ^^^^^^ୀ^are selected in accordance with a distance criterion with respect to the sensor pose ^ ^^௨^^௨ୀ^and a connection criterion with respect to the scene graph ^^; generating (S106) a three-dimensional, 3D, model Ω from the 2D images ^ ^^௨^^௨ୀ^and the sensor poses ^P௨^^௨ୀ^, wherein the 3D model Ω is composed of voxels, and wherein the voxels comprise opacity information; and rendering (S108) the 2D surface based on raytracing in the 3D model Ω from the view positions^^^^^^^ୀ^, wherein one 2D surface is rendered per each view position ^ ^^^^^^ୀ^, and wherein the opacity information defines how light is rendered on the 2D surface.

2. The method according to claim 1, wherein the set of 2D images ^ ^^௨^^௨ୀ^is a set of unstructured 2D images.

3. The method according to any claim 1 or 2, wherein the sensor poses^^^௨^^௨ୀ^are estimated by applying structure-from-motion processing to the set of 2D images ^ ^^௨^^௨ୀ^.

4. The method according to any preceding claim, wherein each of the sensor poses^^^௨^^௨ୀ^comprises a 3D position (310), and wherein determining view position ^^^among the sensor poses ^ ^^௨^^௨ୀ^comprises, according to the connection criterion,selecting the view position ^^^as the 3D having highest connectivity in the scene graph ^^.

5. The method according to claim 4, wherein determining view position ^^^among the sensor poses ^ ^^௨^^௨ୀ^comprises, according to the distance criterion, defining an exclusion circle placed at the selected 3D position and within which circle no other view position can be located.

6. The method according to claim 5, wherein said determining a location and said defining an exclusion circle is repeated to determine further locations and defining further exclusion circles until there are no more sensor poses ^ ^^௨^^௨ୀ^outside any of the exclusion circles.

7. The method according to claim 5, wherein each of the sensor poses^^^௨^^௨ୀ^comprises a 3D position, and wherein determining view position ^^^among the sensor poses^^^௨^^௨ୀ^comprises applying a clustering algorithm to cluster the 3D positions into 3D position clusters, and wherein said determining a location and said defining an exclusion circle is performed for the sensor poses ^ ^^௨^^௨ୀ^per each of the 3D position clusters until there are no more sensor poses ^ ^^௨^^௨ୀ^outside any of the exclusion circles in each of the 3D position clusters.

8. The method according to any preceding claim, wherein each view position ^^^comprises an elevation value, and wherein the elevation value for each of the view positions ^ ^^^^^^ୀ^is set to an average elevation of the view positions ^ ^^^^^^ୀ^.

9. The method according to any preceding claim, wherein the 2D surface represents sides of a cubemap Γ^.

10. The method according to claim 9, wherein for view position ^^^, raytracing is performed for a respective sub-subset ^^^ᇱof voxels of the subset Π^.

11. The method according to claim 10, wherein the sub-subset Π^ᇱof voxels for view position t୩and a side ^^^,^of the cubemap Γ^is defined by a view frustum at the view position t୩and directed towards the side ^^^,^.

12. The method according to any of 9 to 11, wherein each of the sides ^^^,^in the cubemap Γ^represents a respective depth-map ^^^.

13. The method according to any preceding claim, wherein the 2D surface is rendered in a skybox rendering environment, a Potree rendering environment, or a CloudCompare rendering environment.

14. A controller (100, 600) for rendering a two-dimensional, 2D, surface from a set of ^^ 2D images^^^௨^^௨ୀ^with unknown scanning poses, the controller (100, 600) comprising processing circuitry (610), the processing circuitry being configured to cause the controller (100, 600) to: estimate a scene graph ^^ for the set of 2D images^^^௨^^௨ୀ^and, for each 2D image, a sensor pose ^ ^^௨^^௨ୀ^from the set of 2D images ^ ^^௨^^௨ୀ^, wherein the sensor poses represent the scanning poses; determine a set of ^^ view positions^^^^^^^ୀ^(320), where ^^ ^ ^^, for the 2D images ^ ^^௨^^௨ୀ^based on analysis of the scene graph ^^ and the sensor poses ^P௨^^௨ୀ^, wherein the view positions ^ ^^^^^^ୀ^are selected in accordance with a distance criterion with respect to the sensor pose ^ ^^௨^^௨ୀ^and a connection criterion with respect to the scene graph ^^; generate a three-dimensional, 3D, model Ω from the 2D images^^^௨^^௨ୀ^and the sensor poses ^P௨^^௨ୀ^, wherein the 3D model Ω is composed of voxels, and wherein the voxels comprise opacity information; and render the 2D surface based on raytracing in the 3D model Ω from the viewpositions^^^^^^^ୀ^ , wherein one 2D surface is rendered per each view position^^^^^^^ୀ^ ,and wherein the opacity information defines how light is rendered on the 2D surface.

15. A controller (100, 700) for rendering a two-dimensional, 2D, surface from a set of ^^ 2D images^^^௨^^௨ୀ^with unknown scanning poses, the controller (100, 700) comprising:an estimate module (710) to estimate a scene graph ^^ for the set of 2D images ^ ^^௨^^௨ୀ^and, for each 2D image, a sensor pose ^ ^^௨^^௨ୀ^from the set of 2D images^^^௨^^௨ୀ^, wherein the sensor poses represent the scanning poses; a determine module (720) configured to determine a set of ^^ view positions ^ ^^^^^^ୀ^(320), where ^^ ^ ^^, for the 2D images ^ ^^௨^^௨ୀ^based on analysis of the scene graph ^^ and the sensor poses ^P௨^^௨ୀ^, wherein the view positions ^ ^^^^^^ୀ^are selected in accordance with a distance criterion with respect to the sensor pose^^^௨^^௨ୀ^and a connection criterion with respect to the scene graph ^^; a generate module (730) configured to generate a three-dimensional, 3D, model Ω from the 2D images ^ ^^௨^^௨ୀ^and the sensor poses ^P௨^^௨ୀ^, wherein the 3D model Ω is composed of voxels, and wherein the voxels comprise opacity information; and a render module (740) configured to render the 2D surface based on raytracing in the 3D model Ω from the view positions ^ ^^^^^^ୀ^, wherein one 2D surface is rendered per each view position^^^^^^^ୀ^, and wherein the opacity information defines how light is rendered on the 2D surface.

16. The controller (100, 600, 700) according to claim 14 or 15, further being configured to perform a method according to any of claims 2 to 13.

17. A computer program (820) for rendering a two-dimensional, 2D, surface from a set of ^^ 2D images^^^௨^^௨ୀ^with unknown scanning poses, the computer program comprising computer code which, when run on processing circuitry (610) of a controller (100, 600), causes the controller (100, 600) to: estimate (S102) a scene graph ^^ for the set of 2D images ^ ^^௨^^௨ୀ^and, for each 2Dimage, a sensor pose^^^௨^^௨ୀ^ from the set of 2D images^^^௨^^௨ୀ^ , wherein the sensorposes represent the scanning poses;determine (S104) a set of ^^ view positions ^ ^^^^^^ୀ^(320), where ^^ ^ ^^, for the 2D images^^^௨^^௨ୀ^based on analysis of the^^ and the sensor poses ^P௨^^௨ୀ^, wherein the view positions ^ ^^^^^^ୀ^are selected in accordance with a distance criterion with respect to the sensor pose ^ ^^௨^^௨ୀ^and a connection criterion with respect to the scene graph ^^;generate (S106) a three- 3D, model Ω from the 2D images^^^௨^^௨ୀ^and the sensor poses^P௨^^௨ୀ^, wherein the 3D model Ω is composed of voxels, and wherein the voxels comprise opacity information; and render (S108) the 2D surface based on raytracing in the 3D model Ω from the view positions ^ ^^^^^^ୀ^, wherein one 2D surface is rendered per each view position^^^^^^^ୀ^, and wherein the opacity information defines how light is rendered on the 2D surface.

18. A computer program product (810) comprising a computer program (820) according to claim 17, and a computer readable storage medium (830) on which the computer program is stored.

Images