Interactive re-scan solution based on change detection

Abstract

A method, apparatus for re-scanning a 3D scene in which changes occur is provided. A 3D model of a 3D scene is obtained, and a scanning device is positioned in the 3D scene. The scanning device begins to scan the 3D scene. A difference between the scanned portion of the 3D scene and the 3D model is detected and highlighted on a view displayed to the user. The user is provided with options to modify the 3D model by adding a new detected object, removing a detected missing object, or ignoring the change. The 3D model is updated accordingly and a new portion of the 3D scene is scanned.

Description

1. Technical Field

[0001] The present principles generally relate to the field of three-dimensional scanning and 3D reconstruction. In particular, the present principles relate to the detection of changes during rescanning or scan extension. This document is also understood in the context of formatting and playback of extended reality applications when rendered on an end-user device such as a mobile device or head-mounted display (HMD). 2. Background technology

[0002] This section is intended to introduce the reader to various aspects of the art, which may be related to various aspects of the present principles described and / or claimed below. This discussion is considered to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present principles. Therefore, it should be understood that these statements are to be read in this light, and not as admissions of prior art.

[0003] 3D scanning and 3D reconstruction are the processes of capturing the 3D shape and appearance of real-world surfaces by collecting and analyzing data corresponding to real-world objects or environments. The reconstructed 3D model of an object can be used in different situations. For example, the 3D model can be used for object / position detection / recognition, and for triggering actions such as displaying additional information (augmented reality). The 3D model can be used for object pose estimation / camera repositioning and is required to render real-virtual interactions (occlusion, bounces, shadows, ...). In mixed reality experiences, the 3D model can be used as new 3D content to populate a 3D asset database, or displayed in a business catalog or extended reality (XR) application.

[0004] The application areas are wide, such as e-shopping, gaming, education, cultural heritage, training, industrial maintenance, virtual tourism, where 3D scanning has become available to consumers due to mobile-based scanning solutions, while previous solutions were still reserved for professionals (complex multi-camera setups, fixed camera arrays, cumbersome user interaction, time consuming, cost...). Scanning remains a critical task and should be performed with care. Even when considering 3D scenes containing only non-self-moving objects, many of these objects are movable, and such scenes rarely remain static as soon as humans visit the real-world environment. For example, doors are opened and chairs are moved. Accurate 3D model reconstruction usually requires updates (in the case where a part of the environment has changed, a subsequent scan focusing on this part is required) and / or extension steps (in the case where a part of the environment has not been scanned during the initial scan).

[0005] The scanning solution provides a point cloud that is used to create a mesh of the scanned surface. The mesh consists of faces (usually triangles or quadrilaterals). Some mobile 3D scanning solutions feature an extension mode as follows: a previously scanned 3D mesh model is reloaded and, after repositioning, it can be completed. The completion process works as follows: some new faces are added, and some existing faces are retained, even if they correspond to removed objects. The user cannot decide or control the insertion or discarding of new faces, nor the retention or removal of existing faces. There is a lack of a solution in which changes are highlighted and can therefore be marked or annotated in order to bring added value to the reconstructed 3D model. 3. Summary of the invention

[0006] A simplified summary of the present principles is presented below to provide a basic understanding of some aspects of the present principles. This summary is not an extensive review of the present principles. It is not intended to identify the key or decisive elements of the present principles. The following summary merely presents some aspects of the present principles in a simplified form as a preface to the more detailed description provided below.

[0007] This principle relates to a method comprising:

[0008] - Get a 3D model of the 3D scene;

[0009] - Position the scanning device in the 3D scene;

[0010] - Scanning the 3D scene with a scanning device and detecting the difference between the 3D model and the scanned 3D scene;

[0011] - highlight differences in the view of the 3D scene; and

[0012] -Modify 3D models based on change detection patterns.

[0013] In an embodiment, a user is able to modify a change detection mode between scans of two portions of a 3D scene. The change detection mode belongs to a set of modes including: retaining detected removals and adding detected insertions; removing detected removals and ignoring detected insertions; retaining detected removals and ignoring detected insertions; and removing detected removals / adding detected insertions. In an embodiment, a manual mode allows a change detection option to be displayed to the user for retaining or ignoring detected removals and / or detected insertions, and wherein the 3D model is modified according to the user's selection of the option. In another embodiment, the method includes maintaining a database of removed objects of the 3D scene, and when a new object is detected, searching the database for the occurrence of the detected object. The scanning device may capture an RGB(D) frame of a portion of the 3D scene. Then, differences are detected by comparing the RGB(D) frame with the corresponding portion of the 3D model. Highlighting of the differences may be displayed on a live video frame or on a static RGB(D) frame, where the differences are fully visible.

[0014] The present principles also relate to a device comprising a scanning device and a display device and configured to implement the above method. 4. Description of the drawings

[0015] The present disclosure will be better understood, and other particular features and advantages will emerge, upon reading the following description, which refers to the accompanying drawings, in which:

[0016] - Figure 1 A method for interactively scanning a 3D scene based on change detection according to the present principles is illustrated;

[0017] - Figure 2 shows a reference view and two enhanced views after rescanning according to the present principles;

[0018] - Figure 3 It is shown that the present principles can be configured to implement Figure 1 An example architecture of a device for the described method. 5. Specific implementation methods

[0019] The present principle will be described more fully below with reference to the accompanying drawings, in which examples of the present principle are shown. However, the present principle can be implemented in a variety of alternative forms and should not be interpreted as being limited to the examples set forth herein. Therefore, although the present principle is susceptible to various modifications and alternative forms, its specific examples are shown by the examples in the accompanying drawings and will be described in detail herein. However, it should be understood that there is no intention to limit the present principle to the specific form disclosed, but on the contrary, the present disclosure will cover all modifications, equivalents and alternatives that fall within the spirit and scope of the present principle as defined by the claims.

[0020] The terms used herein are only used for the purpose of describing specific examples and are not intended to be a limitation of this principle. As used herein, the singular forms "one", "an" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will be further understood that when the terms "include", "include", "contain" and / or "contain" are used in this specification, the existence of the features, integers, steps, operations, elements and / or components is specified, but the existence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups is not excluded. In addition, when an element is referred to as "in response to" or "connected to" another element, it can directly respond to or be connected to another element, or there can be an intermediate element. In contrast, when an element is referred to as "directly responding to" or "directly connected to" another element, there is no intermediate element. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items, and can be abbreviated as " / ".

[0021] It will be understood that although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the teaching of the present principles, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

[0022] Although some of the figures include arrows on communication paths to illustrate a primary direction of communication, it is to be understood that communication can occur in the opposite direction to the depicted arrows.

[0023] Some examples are described with respect to block diagrams and operational flow charts, wherein each block represents a portion of a circuit element, module, or code, which includes one or more executable instructions for implementing a specified (one or more) logical function. It should also be noted that in other implementations, the (one or more) functions indicated in the blocks may not appear in the order indicated. For example, two blocks shown in succession may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order, depending on the functions involved.

[0024] References herein to "according to an example" or "in an example" mean that a particular feature, structure, or characteristic described in conjunction with the example can be included in at least one implementation of the present principles. The phrases "according to an example" or "in an example" appearing in various places in the specification are not necessarily all referring to the same example, nor are they necessarily separate or alternative examples that are mutually exclusive of other examples.

[0025] Reference signs appearing in the claims are by way of illustration only and shall have no limiting effect on the scope of the claims.The present examples and variations may be used in any combination or sub-combination although not explicitly described.

[0026] Figure 1 A method 10 for interactively scanning a 3D scene based on change detection according to the present principles is illustrated. The method is applicable to a system comprising a scanning device and a display device that provides a view of the scanned 3D scene to a user.

[0027] At step 11, the system is started. A reference 3D model of the 3D scene is obtained in association with the positioning data. Based on the positioning data and the reference 3D model, the scanning device is positioned relative to the coordinate system of the 3D scene. For example, the reference 3D model is a 3D mesh provided as an "obj" file (with or without texture), and the "obj" file is obtained by an existing mobile 3D scanning solution using LiDAR technology or through a photogrammetry pipeline. The associated positioning data are usually derived from the keyframes used for reconstruction, and their poses are related to the coordinate system in which the 3D mesh is registered and the set of feature points extracted from these keyframes and their descriptors. They usually appear in a format that depends directly on the selected reconstruction solution. The positioning of the scanning device consists in estimating the pose (position plus orientation) of the device relative to the pre-scanned environment ("model space"). For example, it can be calculated by extracting 2D feature points in the keyframes and matching them with 3D feature points calculated during reconstruction. When running an augmented reality (AR) application, the device is able to determine its own pose relative to the "world" (also called "world space") of the current AR session. This means that the transformation between the 3D model space and the world space is obtained by combining the poses of the two devices. The reference 3D mesh can also be the first 3D model dynamically constructed by the present scanning device implementing the present scanning method. In such a case, positioning is simple.

[0028] At step 12 of the method according to the present principles, a partial scan of the 3D scene is performed by the scanning device. For example, RGB(D) frames (i.e., color plus depth images) are captured by the (mobile) scanning device and sent to the change detection module together with their pose relative to the model space. Such a real-time pose can be directly calculated from the real-time pose of the camera in the world space and the transformation between the model space and the world space, without the need for positioning each new frame. In the case where the change detection module is not implemented on the scanning device but on a server (e.g., on a PC or in the cloud), the RGB(D) frames and the pose can be sent to the server in, for example, an HTTP request.

[0029] At step 13, local geometric changes between the reference 3D model and the partially scanned model are searched and detected. For example, a live RGB(D) frame is compared to a reference 3D mesh using a hybrid method based on projection and reprojection. Changes (insertions and removals) are first detected in 2D at the level of pixels of the RGB(D) frame and then in 3D model space as 3D regions (3D point groups or parametric 3D shapes, such as, for example, ellipsoids or parallelepipeds) by triangulation.

[0030] At step 14, the 3D area corresponding to the detected change is projected in the user view in 2D and displayed to the user (e.g., an outline, a specific color, or a translucent level for removal and for insertion). The 3D area corresponding to the detected change is rendered in the user view and displayed to the user in AR. For example, an ellipsoid, or an outline of a projected ellipsoid, or a 3D face included in the ellipsoid, or a wireframe is rendered with a specific color and translucent level for removal, and with a different given color for insertion. Animation effects can be applied by playing on the translucent level to illustrate the appearance of the inserted face and the disappearance of the removed face.

[0031] Figure 2 An example reference view 20 and two enhanced views 21 and 22 after a rescan are shown. In this example, the reference view 20 has been established at a reference viewpoint and the positioning data is referenced to that viewpoint. When a new (or the same) scanning device is introduced into the scene in a new pose, pictures 21 and 22 may be captured. In the scene represented by picture 20, there is a box on the floor. In the same scene captured later from a different viewpoint in the same 3D space, the box has disappeared. Technically, the faces of the 3D mesh representing the 3D scene have moved or have been canceled. In Figure 2 In the example of , the absence of the box is highlighted in the user's view as an ellipsoid (in picture 21) or a grid (in picture 22). Similar highlighted representations are possible for object (i.e., grid) additions. Many other representations are possible, as long as they are clear to the user. For example, while additions can be highlighted in green, removals can be highlighted in red. When selecting such a highlighted area of ​​their view, the user can confirm or weaken the default action (remove or add), as shown in picture 23. Many ways to indicate to the user in AR that the system has detected a removal are possible. For example, a 3D virtual element can be rendered in real time on top of the real environment observed from the current user's viewpoint. In picture 21, it is a semi-transparent ellipsoid. In picture 22, a wireframe of the face of the reference 3D grid is included in the change area. In another example not shown, a portion of the reference 3D texture grid included in the change area is displayed in a semi-transparent manner and recolored in red.

[0032] exist Figure 1 At step 15, the user makes and inputs a decision based on the highlighted area displayed on his view. Each detected 3D changed area is made interactive by associating a collider. When the user selects an area, they are asked how the graphical user interface (GUI) must behave. For example, for a detected removal, two buttons may appear for retaining the 3D faces included in the 3D area or for deleting them from the reference grid (as in Figure 23). Once the buttons are clicked, both buttons may disappear and the desired action is performed. For a newly inserted object, the user may be similarly asked whether the object must be ignored or added to the reference grid. In a variant, the entire reference grid may be searched in order to find potential occurrences of an object located at another position (meaning that the object has changed position). The output of the search may be a set of possible 3D areas in the reference grid that may correspond to the "new" object, which may be presented to the user for selection and / or verification. The search may be limited to the set of removed objects. If available in the data accompanying the reference grid, the user is shown a previous RGB keyframe that may have included the same object before it had changed position for selection and / or verification. In another variation, if the object is moved, the motion between the previous and current positions may be determined by automatic estimation of its 3D transform (e.g., based on 3D feature point matching) and presented to the user for confirmation (e.g., rotation along a particular axis, translation, etc.).

[0033] The case of moving objects may also involve situations such as drawers that have been opened or closed (showing or hiding the contents inside), doors that have been opened or closed (showing or hiding the room behind), objects that have fallen over... In the case of active scanning, such complex situations are particularly interesting in order to obtain a more complete understanding of the object. The user can easily indicate that the "removed" front side and the "new" back side of the door do belong to the same object (similarly for the front side and the inside of the drawer), while automatic matching may fail. In a variant, maintenance of a database of new and removed objects can be implemented. When a removal or insertion is detected, the corresponding 3D change area (3D shape, view,...) can be stored together with (at least) its last known position in order to maintain a database of "known" objects. Whenever a new object is detected, the database can be searched in order to find potential occurrences of the object in the database.

[0034] At step 16, the 3D model is updated based on the detected and highlighted changes, and based on the user's decision. For detected removals, if the user decides to keep the 3D faces included in the detected 3D region, no updates are performed. If the user decides to delete a 3D face, each face and all of its vertices in the 3D region are removed from the mesh. For detected insertions, if the user decides to ignore the 3D faces included in the detected 3D region, no updates are performed. If the user decides to add a 3D face, each face and all of its vertices in the 3D region are added to the mesh. If the object is detected as "moving", then in addition to scanning, a previous version of the mesh can be used for updating. If the object is brand new, the user can be notified that the object should be scanned with accurate accuracy. Once the update is performed, the region highlighting is turned off, and another region can be selected. Once a decision has been made for each region, a new real-time RGB(D) frame can be considered and sent to change detection.

[0035] In another embodiment, the history of all detected changes and user decisions is maintained in metadata related to the associated faces and is embedded in the 3D mesh or in the scene description and can be exported in an output format. Considering the .obj file format, the polygonal geometry of the mesh can be segmented into different objects, for example using the "g" groupName or "o" objectName grouping keywords. Therefore, for a group of faces detected as insertions, and regardless of whether the user decides to add it or ignore it, a new group is created and added to the file. Similarly, for removals, and regardless of whether the user decides to keep it or remove it, the corresponding faces are cut from the main mesh and pasted into the new group. The groupName and objectName descriptions or additional keywords can contain information about the detection type (insert or remove), user decision (keep, remove, add, ignore), username or time.

[0036] In this embodiment, object detection is applied to RGB(D) frames, for example using a deep neural network, making it possible to name object classes based on detected insertions. Similarly, if the reference grid has been processed jointly with semantic segmentation, semantic metadata is available and it is possible to name object classes based on detected removals. For detected insertions of class A objects, the entire reference grid can be searched to find potential occurrences of the same class A object at another location (meaning that the object has changed location). The output of the search can be a set of possible 3D regions in the reference grid that may correspond to "new" objects, which can be presented to the user for selection or verification. The search may be limited to the set of removed class A objects. Semantics can also be used to help estimate the 3D motion of objects by reducing the number of degrees of freedom: typically, doors rotate around a vertical axis and drawers translate horizontally on an axis perpendicular to the plane of the front face. When the detected regions are displayed to the user, the names can be displayed as text or pronounced by speech synthesis (e.g., "A new chair has been inserted. Do you want to add it to the model?"). Object names may also be saved in the history and exported in the output files (e.g. using the "o insertedChair" grouping key that may be included in the .obj file). Furthermore, if a scene graph representation is available and contains information about the spatial relations between objects contained in the scene, the detection may be further characterized, e.g. to resolve ambiguity in case of several instances of the same object, e.g. "A new chair has been inserted between the table and the fireplace".

[0037] In another embodiment, the required user interaction or verification is limited. In this embodiment, the following predefined automatic modes are presented to the user via the GUI, for example as radio buttons:

[0038] 1.ALL / MAX: keep detected removals / add detected insertions => minimize empty (i.e. unoccupied) space => build / update STABLE / SECURED empty space and corresponding (adjacent) surfaces

[0039] 2.MIN: remove detected removals / ignore detected insertions => maximize empty space => build / update STABLE / UNMOVABLE environment as stable surface set

[0040] 3.OLD: keep detected removals / ignore detected insertions => build / update old surface set

[0041] 4.NEW: Remove detected removals / add detected insertions => Build / update new surface set

[0042] The options are mutually exclusive. The user must select exactly one option ("Manual" is the fifth mode selected by default). In other words, clicking an unselected radio button will deselect whatever other button was previously selected in the list. As long as a mode other than "Manual" is selected, the system does not require specific user decisions to process each change region: the grid updates corresponding to the selected mode are automatically applied to all detected removals and insertions until that mode is deselected.

[0043] In another embodiment, the changes are displayed on top of a static image corresponding to the last RGB frame that has been conducted to detect the changes, rather than on top of the live video stream. It allows the user to focus on the subsequent decision without having to keep their device camera pointed at a specific 3D location, and without having to exit the program because the live video stream is displayed again as soon as the last user decision is made.

[0044] In another embodiment, to avoid displaying changes as soon as they are detected, the display is postponed while other possible changes are detected and a list of changes is maintained. Displaying changes to the user begins only after a given amount of time (e.g., after N minutes of scanning) or on demand. In this embodiment, since the user may have moved between the detection time and the display time, the detected changes may instead be displayed on top of a stored RGB frame that best shows the changes rather than on top of a real-time video stream.

[0045] Figure 3 An example architecture of a device 30 is shown, which may be configured to implement Figure 1 Described method. Figure 3 The devices of the architecture are linked to other devices via their buses 31 and / or via I / O interfaces 36 .

[0046] The device 30 comprises the following elements linked together by a data and address bus 31:

[0047] - a microprocessor 32 (or CPU), which is for example a DSP (or digital signal processor);

[0048] - ROM (or read-only memory) 33;

[0049] - RAM (or random access memory) 34;

[0050] - storage interface 35;

[0051] - an I / O interface 36 for receiving data to be transmitted from an application; and

[0052] -power supply( Figure 3 not shown), such as batteries.

[0053] According to the example, the power supply is external to the device. In each of the memories mentioned, the word "register" used in the specification may correspond to an area of ​​small capacity (a few bits) or a very large area (such as an entire program or a large amount of received or decoded data). ROM 33 includes at least programs and parameters. ROM 33 can store algorithms and instructions to perform technology according to the present principles. When turned on, CPU 32 uploads the program to RAM and executes the corresponding instructions.

[0054] RAM 34 includes in registers a program executed by CPU 32 and uploaded after device 30 is turned on, input data in registers, intermediate data in different states of a method in registers, and other variables in registers used to execute the method.

[0055] The implementation described herein can be implemented in, for example, a method or process, a device, a computer program product, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (e.g., discussed only as a method or device), the implementation of the discussed features can also be implemented in other forms (e.g., programs). The device can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a device (such as, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device). The processor also includes a communication device, such as, for example, a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate the communication of information between end users.

[0056] The device 30 is linked, for example, via a bus 31, to a set of sensors 37 and a set of rendering devices 38. The sensors 37 may be, for example, cameras, microphones, temperature sensors, inertial measurement units, GPS, humidity sensors, IR or UV light sensors, or wind sensors. The rendering devices 38 may be, for example, displays, speakers, vibrators, heaters, fans, etc.

[0057] According to an example, the device 30 is configured to implement Figure 1 The method of the present principles is described and belongs to the group consisting of:

[0058] -mobile device;

[0059] -communication equipment;

[0060] -Gaming equipment;

[0061] - Tablet (or tablet computer);

[0062] - Laptop;

[0063] -Still picture camera;

[0064] -Camcorder.

[0065] The implementation described herein can be implemented in, for example, a method or process, a device, a computer program product, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (e.g., discussed only as a method or device), the implementation of the discussed features can also be implemented in other forms (e.g., programs). The device can be implemented in, for example, appropriate hardware, software, and firmware. The method can be implemented in, for example, a device (such as, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device). The processor also includes a communication device, such as, for example, a smart phone, a tablet, a computer, a mobile phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate the communication of information between end users.

[0066] The implementation of the various processes and features described herein can be implemented in a variety of different equipment or applications (particularly, for example, equipment or applications associated with data encoding, data decoding, view generation, texture processing, and other processing of images and related texture information and / or depth information). Examples of such equipment include encoders, decoders, post-processors that process outputs from decoders, pre-processors that provide inputs to encoders, video encoders, video decoders, video codecs, web servers, set-top boxes, laptop computers, personal computers, cellular phones, PDAs, and other communication devices. It should be clear that the equipment can be mobile and even installed in a mobile vehicle.

[0067] In addition, the method can be implemented by instructions executed by a processor, and such instructions (and / or data values ​​produced by the implementation) can be stored on a processor-readable medium, such as, for example, an integrated circuit, a software carrier, or other storage device, such as, for example, a hard disk, a compact disk ("CD"), an optical disk (such as, for example, a DVD, commonly referred to as a digital versatile disk or digital video disk), a random access memory ("RAM"), or a read-only memory ("ROM"). The instructions can form an application program that is tangibly implemented on a processor-readable medium. The instructions can be, for example, in hardware, firmware, software, or a combination. The instructions can be found in, for example, an operating system, a separate application, or a combination of the two. Therefore, a processor can be characterized as both a device configured to perform a process and a device that includes a processor-readable medium (such as a storage device) having instructions for performing the process. In addition, in addition to or in lieu of instructions, the processor-readable medium can store data values ​​produced by the implementation.

[0068] As will be apparent to one skilled in the art, implementations may generate a variety of signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry as data the rules for writing or reading the grammar of the described embodiments, or to carry as data the actual grammar values ​​written by the described embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0069] Many implementations have been described. However, it will be appreciated that various modifications may be made. For example, elements of different implementations may be combined, supplemented, modified, or removed to produce other implementations. In addition, it will be appreciated by those of ordinary skill in the art that other structures and processes may replace those disclosed, and that the resulting implementations will perform at least substantially the same (one or more) functions in at least substantially the same (one or more) manners to achieve at least substantially the same (one or more) results as the disclosed implementations. Therefore, the present application contemplates these and other implementations.

Claims

1. A method comprising: - Get a 3D model of the 3D scene; - Position the scanning device in the 3D scene; - Scanning the 3D scene with a scanning device and detecting the difference between the 3D model and the scanned 3D scene; - Highlight differences in the view of the 3D scene; as well as -Modify 3D models based on change detection patterns.

2. The method of claim 1 , wherein a user is able to modify the change detection mode between scans of two portions of the 3D scene, and wherein the change detection mode belongs to a set of modes comprising: -keep detected removals and add detected insertions; - remove detected removals and ignore detected insertions; - keep detected removals and ignore detected insertions; and - Remove detected removals / Add detected insertions.

3. The method of claim 2, wherein the set of change detection modes comprises a manual mode in which change detection options are displayed to a user for retaining or ignoring detected removals and / or detected insertions, and wherein the 3D model is modified according to a user selection of an option.

4. A method according to claims 1 to 3, comprising maintaining a database of removed objects of the 3D scene and, when a new object is detected, searching the database for occurrences of the detected object.

5. The method according to claims 1 to 4, wherein the scanning device captures an RGB(D) frame of a portion of the 3D scene, and the difference is detected by comparing the RGB(D) frame with the corresponding portion of the 3D model.

6. The method of claim 5, wherein the highlighting of the differences is displayed on a static RGB(D) frame, wherein the differences are fully visible.

7. The method of claims 1 to 6, wherein the 3D model is a 3D mesh, and wherein modifying the 3D model comprises modifying a face of the 3D mesh.

8. A method according to claims 1 to 7, wherein highlighting of differences is performed within a period of time after the differences are detected.

9. A device comprising a scanning device and a display device, and configured to: - Get a 3D model of the 3D scene; - Position the scanning device in the 3D scene; - Scanning the 3D scene with a scanning device and detecting the difference between the 3D model and the scanned 3D scene; - highlighting differences in the view of the 3D scene displayed on the display device; and -Modify 3D models based on change detection patterns.

10. The apparatus of claim 9, wherein a user is able to modify the change detection mode between scans of two portions of the 3D scene, and wherein the change detection mode belongs to a set of modes comprising: -keep detected removals and add detected insertions; - remove detected removals and ignore detected insertions; - keep detected removals and ignore detected insertions; and - Remove detected removals / Add detected insertions.

11. The apparatus of claim 10, wherein the set of change detection modes comprises a manual mode in which change detection options are displayed to a user for retaining or ignoring detected removals and / or detected insertions, and wherein the 3D model is modified according to a user selection of an option.

12. The device according to claims 9 to 11, wherein the device is further configured to maintain a database of removed objects of the 3D scene and when a new object is detected, search the database for an occurrence of the detected object.

13. A device according to claims 9 to 12, wherein the scanning device captures an RGB(D) frame of a portion of the 3D scene, and the difference is detected by comparing the RGB(D) frame with the corresponding portion of the 3D model.

14. The apparatus of claim 13, wherein the highlighting of the differences is displayed on a static RGB(D) frame, wherein the differences are fully visible.

15. Apparatus according to claims 9 to 14, wherein the 3D model is a 3D mesh, and wherein modifying the 3D model is modifying a face of the 3D mesh.

16. Apparatus as claimed in claims 9 to 15, wherein highlighting of differences is performed within a period of time after the differences are detected.

Images