Local optical flow as an alternative to planar scanning volume

Abstract

The present invention provides an apparatus and method for image processing. In one embodiment, the method includes accessing a plurality of images captured by at least one reference camera, wherein the images represent a plurality of views corresponding to the same scene. A plurality of planar scan volume (PSV) slices are then generated from the images, and for each slice, a flow graph is calculated from at least reference camera calibration parameters, and using the flow graph, a first slice of the planar scan volume is generated.

Description

Technical Field

[0001] This disclosure generally relates to light field transmission and processing, including depth estimation and view synthesis, and more specifically to techniques and systems using neural networks. Background Technology

[0002] Many devices and systems allow scenes to be captured by generating image and / or video data of the scene. For example, a conventional camera can be used to capture images of a scene for various purposes. One such use is to provide a 3D reconstruction of the scene geometry. Another is to render a virtual view of the scene.

[0003] Light field cameras and all-light cameras provide more information than conventional cameras. This is because the camera can acquire light field data and capture the light field emanating from the scene. This is the intensity of light in the scene. One type of light field camera uses an array of microlenses placed in front of what would otherwise be a conventional image sensor to sense intensity, color, and direction information. This allows for the display of several images from a single scene, providing far more information than a conventional camera.

[0004] In recent years, cameras, especially light field cameras, have been used in an increasing number of fields related to deep or neural networks. Deep or neural networks (DNNs) are artificial neural networks (ANNs) with multiple layers between the input and output layers. DNNs find the correct mathematical operations to transform the input into the output, whether the relationship is linear or non-linear. Unlike most traditional machine learning algorithms, these deep learning networks perform automatic feature extraction without human intervention. In some cases, neural networks can be used to perform depth estimation and view synthesis, among other tasks. Given the vast amounts of data stored by neural network-based systems, such systems can generate high-quality 3D reconstruction and view synthesis results. While neural networks (e.g., deep learning networks) have proven to be very versatile and accurate in a wide variety of tasks, such networks require high memory bandwidth and high computational costs.

[0005] Regardless of its purpose (view synthesis, depth estimation, etc.), light field processing requires knowledge of the underlying ray geometry. The acquisition device must be calibrated, but directly feeding the algorithm with a set of calibration parameters is inconvenient. One problem is the diversity of existing devices and required calibration models (all-light cameras versus camera setups, distortion polynomials, etc.), leading to heterogeneous (and potentially computationally complex) processing. A common way to restore computational uniformity in algorithms, especially when dealing with convolutional neural networks (CNNs), involves pre-converting the light field image into a planar scan volume (PSV). However, planar scan volumes are redundant and result in significant memory consumption. Therefore, improved techniques for image transmission and processing are needed, particularly those provided by neural or deep networks. Summary of the Invention

[0006] Additional features and advantages are achieved through similar techniques, and other embodiments and aspects are described in detail herein and are considered part of the claimed invention. For a better understanding of the invention, which has advantages and features, refer to the specification and drawings.

[0007] An apparatus and method for image processing are provided. In one embodiment, the method includes accessing a plurality of images captured by at least one reference camera, wherein the images represent a plurality of views corresponding to the same scene. A plurality of planar scan volume (PSV) slices are then generated from the images, and for each slice, a flow graph is calculated from at least reference camera calibration parameters, and using the flow graph, a first slice of the planar scan volume is generated.

[0008] In another embodiment, a method and apparatus are provided, wherein the apparatus has a processor configured to: obtain a first planar scan volume (PSV) slice and its associated flow graph, and to determine camera calibration parameters associated with the first PSV, and to generate at least a previous PSV slice or a next PSV slice based on the first PSV slice and the camera parameters. Attached Figure Description

[0009] The teachings of this disclosure can be readily understood by considering the following specific embodiments in conjunction with the accompanying drawings, wherein:

[0010] Figure 1 This is a block diagram of a light field preprocessing block that is processed into a planar scan volume (PSV) slice according to one embodiment;

[0011] Figure 2 It is a diagram of an input image distorted onto another camera based on one implementation, showing its depth in a coordinate system;

[0012] Figure 3 It is a diagram of the components of a reference camera and another camera, and the resulting PSV, according to one embodiment;

[0013] Figure 4 This is an illustration of an exemplary PSV geometry;

[0014] Figure 5 It is a flowchart of an implementation plan;

[0015] Figure 6 It is a diagram of a device that can be used in conjunction with one or more specific implementation schemes;

[0016] Figure 7 and Figure 8 This is an alternative embodiment of the apparatus that can be used in conjunction with one or more embodiments of the encoder and decoder shown respectively;

[0017] Figure 9 This is a flowchart illustration of another implementation scheme; and

[0018] Figure 10 This is a diagram of an exemplary symbol table.

[0019] It should be understood that the accompanying drawings are intended to illustrate the concept of the invention and are not necessarily intended to illustrate the only possible configuration of the invention. For ease of understanding, the same reference numerals have been used where possible to denote the same elements common in the drawings. Detailed Implementation

[0020] Figure 1 This is an example of a light field array. Figure 1 In the diagram, the number 110 provides an array of light fields for n views, which also includes calibration parameters. Array 110 is then set up as n planar scan volumes, as shown at 120. In effect, light field samples from different view / sub-aperture images are aligned to the same 2D coordinate system in the planar scan volumes, as shown at 120. This resampling greatly simplifies subsequent processing; at this point, the calibration parameters do not need to be considered. However, it should be noted that this simplification is not without its limitations: the planar scan volumes exhibit more redundancy than the original light fields from which they originated. This is achieved through preprocessing, but once this step is performed, the data is ready for processing, as shown at 130. This is one way to restore computational uniformity in algorithms, particularly when dealing with convolutional neural networks (CNNs), which involve pre-converting the light field images into planar scan volumes (PSVs).

[0021] Figure 2 Examples of warping an image onto data acquired by another camera are provided. Resampling the input image into a planar scan volume involves warping the image onto the reference camera using a series of planes at a fixed depth in the reference camera's coordinate system (see [link to documentation]). Figure 2 Depth values ​​are typically expressed as 1 / z in [z]. min ;z max The sample is uniformly sampled within the range.

[0022] See again Figure 2The input image 210 is warped onto another camera 220, representing the depth of its coordinate system 230. As depicted, if no pixels from the input image are mapped onto it, a portion of the resulting PSV slice can remain unknown. This is why planar scan volumes are typically represented with 4 channels: RGBM, where RGB represents red, green, and blue, and M represents a binary occupancy mask (valid / unknown). In this sense, a planar scan volume is therefore often referred to as a 4-dimensional tensor: [H, W, D, 4], where W and H represent the width and height of the reference camera in pixels, and D represents the number of slices in the volume (i.e., the number of fixed-depth planes considered).

[0023] Figure 10 Provided to facilitate discussion, such as combination Figure 1 and Figure 2 A symbol table for the mathematical and geometric concepts discussed.

[0024] In many optical field applications, the following discussion can help in understanding the parameters involved.

[0025] In this example, the vector is considered to be composed of Defined, z ≠ 0. The perspective projection of a vector is written as follows:

[0026]

[0027] There exists a camera that has K as its intrinsic matrix:

[0028]

[0029] in:

[0030] • f represents the pinhole-sensor distance (often mistakenly referred to as "focal length"), expressed in pixels;

[0031] · The pixel coordinates (“principal point”) represent the projection of the pinhole onto the sensor.

[0032] • α represents the aspect ratio of a pixel;

[0033] γ: Skew coefficient, in pixels.

[0034] Let P be the camera's pose matrix. It takes pictures of the camera's position and orientation in the reference ("world") coordinate system.

[0035]

[0036] Where R is the rotation matrix:

[0037]

[0038] And T is the translation vector:

[0039]

[0040] The external matrix of the camera is defined as:

[0041]

[0042] Finally This represents a distortion operator that models the geometric distortion of a camera.

[0043] From the coordinate system (CS), we can obtain the calculation of the other. Let's consider an example with 3D points. Let X... world and X cam These are its coordinates in the world CS and the camera CS, respectively. Then:

[0044] and

[0045] The distorted image projection from this coordinate system to the pixel can then be defined as:

[0046] in

[0047] Without any distortion, this will become

[0048]

[0049] Therefore, 3D reconstruction from pixels to coordinate systems will start from pixels. Transform it into 3D space, representing the depth z in the camera's center (CS):

[0050] It has distortion

[0051] in

[0052] No distortion

[0053]

[0054] See now Figure 4 The information provided can be used as an example, such as a reference camera, to help understand the concept by illustrating a schematic PSV geometry. In this example, the reference camera is taken as a standard pinhole camera defined by its intrinsic matrix K and its pose matrix P. Now consider an actual camera, also calibrated as a standard pinhole camera, whose intrinsic and extrinsic matrices are denoted by K, respectively. c and Q c Suppose we want to convert the image I recorded by an actual camera... cThis is transformed into a planar scan volume, and therefore we assume that we consider a depth plane D, located at depths z1, ..., z in the coordinate system of the reference camera. D The d-th slice (1≤d≤D) of the planar scan volume is defined by the following resampling:

[0055]

[0056] Where the interpolated pixel position Determined by the following:

[0057]

[0058] Unfortunately, the main drawback of planar scanning volumes is their size, and there are associated issues with sufficient memory and cost. The light field image is copied along the z-axis, requiring a significant amount of memory allocation. This is critical because the size of the PSV increases cubically with image resolution. In fact, an effective CNN typically requires sub-pixel parallax (at most 1 pixel parallax) between two consecutive slices of the PSV. If the reference camera's resolution doubles in both width and height, the number of slices must also double, and the total size of each PSV multiplied by 8.

[0059] This order of magnitude is also important. For example, consider a scene with objects ranging from 1m to infinity, such as an outdoor selfie. A 5cm baseline is very small for immersive applications. Let's also maintain a 1-pixel parallax between two PSV slices. Consider a 2K reference camera (2048×1088 resolution). In a standard field of view, this camera should exhibit a focal length f≈2000 pixels. This results in:

[0060]

[0061] Storing in single-precision floating-point format, a 100-slice PSV requires 100×1024×2048×4×4=3.5 gigabytes (meaning 7GB for two views, 14GB for four views, and so on). To avoid this, light field images and PSVs are typically tiled into smaller patches during training (e.g., 30×30 in [1]), however, the amount of data stored in memory for gradient backpropagation makes it difficult to scale up the number of views and image resolution.

[0062] In one implementation, an alternative approach would involve processing each set of corresponding d slices (i.e., those corresponding to the same depth) sequentially from back to front or from front to back. However, in most light field applications, processing the third dimension is crucial for extracting relevant features and performing the desired tasks, thus this alternative approach has drawbacks.

[0063] Figure 3An example based on one implementation is provided. In this implementation, a displacement component is appended to each slice's RGBM component. The stream from the current slice to the next slice is captured. Therefore, PSVs can be processed slice by slice, significantly reducing memory requirements, which enables larger batch sizes and / or higher spatial-angular resolution (pixels and views). Assuming the PSV is sampled uniformly at 1 / z, the stream is naturally bidirectional; this means it is suitable for both the previous slice and the next slice. In other words, consider the displacement (stream) component of each slice of the PSV. If the decoder implementation requires planar scanning volume optical field processing, this can occur after the transmission of the coded optical field.

[0064] See again Figure 3 This illustrates the concept that consecutive slices in a PSV become redundant. This redundancy can be captured by the flow from one slice to another. Figure 3 The image shows a reference camera 310 and another camera 320, and the resulting PSV is shown by dashed lines.

[0065] like Figure 3 The stream shown in the example is obtained directly from the following camera parameters:

[0066] Consider a reference camera calibrated as a standard pinhole camera and thus determined by its intrinsic matrix K. In the following text, the "reference" coordinate system will refer to the coordinate system of this reference camera. Now consider another camera with an attitude P = (RT) in the reference coordinate system. Consider the planar scan volume of this second camera on the reference camera. Now, if T is represented as... Then the pixels in the z-slice And its match in another slice z' The displacement between them is determined by the following:

[0067]

[0068] Consider z'(1 / z+Δ) -1 That is, Δ is a slice shift of 1 / z. The shift is given by the following:

[0069]

[0070] Where K 2×3 This represents the first two lines of K.

[0071] It can be observed that if t z =0, then the displacement no longer depends on the pixel position but only on the 1 / z shift:

[0072]

[0073] Otherwise, if t zIf ≠0, then we can define The polar line of the second camera in the reference camera: and:

[0074]

[0075] Such an example is provided in the illustrative PSV geometry and the application of the above formulas.

[0076] Pixels in the z-slice corresponding to a shift Δ of 1 / z The displacement is determined by the following:

[0077]

[0078] In this case, the displacement is an odd number Δ: changing the sign of the 1 / z shift only changes the sign of the displacement.

[0079]

[0080] Therefore, this flow can be used bidirectionally for both the slice before backward distortion and the slice after forward distortion. In one implementation, the optical field flow can be calculated at the decoder side simply by transmitting the calibration parameter set. Note that the only required parameter is the relative position of the other camera in the coordinate system of the reference camera: T c refcs =Q·(T) c 1) t The intrinsic parameters of the reference camera are also considered. The z and Δ independent variables are determined by the user / decoder. In one implementation, the stream can be directly fed into an algorithm, such as a convolutional neural network, along with the RGB(M) plane. It can also be used to recover the next slice and the previous slice, so that the RGB(M) slice triples can be used for subsequent processing.

[0081] In one implementation, assuming that only the transmitted slice is selected to fill it as much as possible, the stream can be used to recover the entire planar scan volume from that single transmitted slice. In the case of a parallel camera, this means the maximum z-value; however, for a convergence device, the depth of the convergence plane should be preferred. Subsequently, sequential reconstruction can be defined as:

[0082] For each new slice:

[0083] First, calculate the flow graph.

[0084] Then, the previous RGB(M) slice is backward twisted to obtain a new slice.

[0085] In an alternative implementation, since the flow is spatially very smooth, it can also be encoded as a subpixel motion vector map for some reason, or its decoding will be preferred for its simple mathematical reconstruction for some other reason, where calibration parameters cannot be transmitted. Furthermore, assuming the decoder reconstructs a PSV uniformly sampled at 1 / z, the current flow graph can be used to positively distort itself and generate the next flow graph.

[0086] Figure 5 This is a flowchart illustrating an implementation plan. Figure 5 A method is provided in which, as provided in step 510, multiple images captured by a camera are accessed, representing multiple views corresponding to the same scene. In step 520, multiple PSV slices from the images are generated. In step 530, for each slice, a flow graph is calculated from one or more parameters of the camera. In step 540, at least one of the next slice and the previous slice of the planar scan volume is generated.

[0087] Figures 6 to 9 Different means are provided that can be used to adapt functionality to one or more implementation schemes accordingly. For example, Figure 6 A general overview of an encoding and decoding system according to one or more embodiments is illustrated schematically. Figure 6The system is configured to perform one or more functions and may have a preprocessing module 6300 to prepare received content (including one or more images or videos) for encoding by an encoding device 6400. The preprocessing module 6300 may perform multiple image acquisition, merging multiple acquired images in a common space, acquiring omnidirectional video in a specific format, and other functions that allow preparation of a format more suitable for encoding. Another embodiment may combine multiple images into a common space with a point cloud representation. The encoding device 6400 encapsulates the content in a form suitable for transmission and / or storage for recovery by a compatible decoding device 6700. Generally, although not strictly required, the encoding device 6400 provides a degree of compression, thereby allowing for a more efficient representation of the common space (i.e., using less memory for storage and / or using less bandwidth required for transmission). In the case of mapping a 3D sphere onto a 2D frame, the 2D frame is actually an image that can be encoded by any of multiple image (or video) codecs. In the case of a common space with a point cloud representation, the encoding device 6400 can provide point cloud compression, as is known, for example, through octree decomposition. After encoding, the data is sent to network interface 6500, which can typically be implemented in any network interface, such as a gateway. The data can then be transmitted via communication network 6500 (such as the Internet), but any other network is foreseeable. Data received via network interface 6600 can then be implemented in the gateway within the device. After reception, the data is sent to decoding device 6700. The decoded data is then processed by device 6800, which can also communicate with sensors or user input data. Decoder 6700 and device 6800 can be integrated into a single device (e.g., smartphone, game console, STB, tablet, computer, etc.). In another embodiment, rendering device 6900 can also be incorporated.

[0088] Figure 7 and Figure 8 An exemplary architecture including an encoder and decoder structure of transmitter / receiver 700 / 800 (as shown) according to an exemplary and non-limiting embodiment is provided, which is configured to encode / decode an all-optical image in a stream.

[0089] exist Figure 7In this device, transmitter 700 may include one or more processors 7100 (e.g., CPU, GPU, and / or DSP (Digital Signal Processor)) and internal memory 7130 (e.g., RAM, ROM, and / or EPROM). Transmitter 700 also includes one or more communication interfaces 7110, each adapted to display output information and / or allow user input of commands and / or data (e.g., keyboard, mouse, touchpad, webcam); and a power supply 7120 that may be located externally to transmitter 700. Transmitter 700 may also include one or more network interfaces (not shown). Encoder module 7140 represents a module that may be included in the device to perform encoding functions. Alternatively, encoder module 7140 may be implemented as a separate element of transmitter 700 or may be incorporated within processor 7100 as a combination of hardware and software known to those skilled in the art.

[0090] All-optical images can be obtained from a source. Depending on the implementation, the source can be, but is not limited to: local memory, such as video memory, RAM, flash memory, hard disk; storage interface, such as an interface with a mass storage device, ROM, optical disk, or magnetic media; communication interface, such as a wired interface (e.g., bus interface, WAN interface, LAN interface) or a wireless interface (e.g., IEEE 802.11 interface or Bluetooth interface); and image capture circuitry (e.g., a sensor such as, for example, a CCD (or charge-coupled device) or CMOS (or complementary metal-oxide-semiconductor)).

[0091] Depending on the implementation, the stream can be sent to a target. For example, the stream can be stored in remote or local storage, such as video storage or RAM, hard disk. In one variation, the stream can be sent to a storage interface, such as an interface with a mass storage device, ROM, flash memory, optical disc, or magnetic media, and / or transmitted via a communication interface, such as a point-to-point link, communication bus, point-to-multipoint link, or broadcast network interface.

[0092] According to an exemplary and non-limiting embodiment, the transmitter 700 also includes a computer program stored in the memory 7130. The computer program includes instructions that, when executed by the transmitter 700, particularly by the processor 7100, enable the transmitter 700 to perform a reference... Figures 1-5The method described. According to an alternative embodiment, the computer program is stored externally to the transmitter 700 on a non-transitory digital data carrier, such as an external storage medium, like an HDD, CD-ROM, DVD, read-only and / or DVD drive and / or DVD read / write drive, all of which are known in the art. The transmitter 700 therefore includes a mechanism for reading the computer program. Additionally, the transmitter 700 can access one or more Universal Serial Bus (USB) type storage devices (e.g., "Memory Stick") via a corresponding USB port (not shown).

[0093] According to exemplary and non-limiting embodiments, transmitter 100 may be, but is not limited to: mobile device; communication device; gaming device; tablet computer (or tablet PC); laptop computer; still image camera; video camera; encoding chip; still image server; and video server (e.g., broadcast server, video-on-demand server, or web server).

[0094] Figure 8 This describes an exemplary architecture of receiver 800 according to an exemplary and non-limiting embodiment, which is configured to decode an all-optical image from a stream.

[0095] Receiver 800 includes one or more processors 8100 (which may include, for example, a CPU, GPU, and / or DSP (Digital Signal Processor)) and internal memory 8130 (e.g., RAM, ROM, and / or EPROM). Receiver 800 includes one or more communication interfaces 8110, each adapted to display output information and / or allow user input of commands and / or data (e.g., keyboard, mouse, touchpad, webcam); and a power supply 8120 that may be located externally to receiver 800. Receiver 800 may also include one or more network interfaces (not shown). Decoder module 8240 represents a module that may be included in the device to perform decoding functions. Alternatively, decoder module 8140 may be implemented as a separate element of receiver 800 or may be incorporated within processor 8100 as a combination of hardware and software known to those skilled in the art. This stream can be obtained from a source. Depending on the implementation, the source may be, but is not limited to: local memory, such as video memory, RAM, flash memory, hard disk; storage interface, such as an interface with mass storage device, ROM, optical disk, or magnetic media; communication interface, such as a wired interface (e.g., bus interface, WAN interface, LAN interface) or a wireless interface (e.g., IEEE 802.11 interface or Bluetooth interface); and image capture circuitry (e.g., sensors such as CCD (or charge-coupled device) or CMOS (or complementary metal-oxide-semiconductor)). Depending on the implementation, the decoded all-optical image may be transmitted to a target, such as a display device. For example, the decoded all-optical image may be stored in remote or local memory, such as video memory, RAM, or hard disk. In one variation, the decoded all-optical image may be transmitted to a storage interface, such as an interface with mass storage device, ROM, flash memory, optical disk, or magnetic media, and / or transmitted via a communication interface, such as a point-to-point link, communication bus, point-to-multipoint link, or broadcast network interface.

[0096] According to an exemplary and non-limiting embodiment, receiver 800 also includes a computer program stored in memory 8130. The computer program includes instructions that, when executed by receiver 800, particularly by processor 8100, enable the receiver to perform reference... Figures 1 to 5 and Figure 9The method described. According to another embodiment, the computer program can be stored externally to the receiver 800 on a non-transitory digital data carrier, such as an external storage medium, such as an HDD, CD-ROM, DVD, read-only and / or DVD drive and / or DVD read / write drive, all of which are known in the art. The receiver 800 therefore includes a mechanism for reading the computer program. Additionally, the receiver 800 can access one or more Universal Serial Bus (USB) type storage devices (e.g., "Memory Stick") via a corresponding USB port (not shown).

[0097] Figure 9 Another flowchart is provided according to another method, such as that offered by an alternative implementation scheme. Figure 9 In step 910, a first PSV slice and its associated flow graph are generated. In step 920, camera parameters associated with the first PSV are determined. In step 930, at least a previous PSV slice or a next PSV slice is generated based on the first PSV slice and the camera parameters.

[0098] Several specific embodiments have been described. However, it should be understood that many modifications can be made. For example, elements of different embodiments can be combined, supplemented, modified, or removed to produce other embodiments. Furthermore, those skilled in the art will understand that other structures and processes can be replaced with those disclosed, and the resulting embodiments will perform at least substantially the same function in at least substantially the same manner to achieve at least substantially the same results as the disclosed embodiments. Therefore, these and other embodiments are contemplated by and within the scope of this disclosure.

Claims

1. A method, the method comprising: Access multiple images captured by one or more cameras; The images therein represent multiple views corresponding to the same scene; Multiple planar scan volume PSV slices are generated from the image, and for each slice, a displacement map is calculated from the calibration parameters of the one or more cameras; wherein the PSV slice corresponding to a reference camera is located in a continuous depth plane in the coordinate system of the reference camera; and The displacement map is used to generate at least one of the next slice and the previous slice of the planar scan volume; Each slice of the PSV is provided with a displacement component, and the displacement component is appended to each slice that captures the displacement from the current slice to the next slice.

2. The method of claim 1, wherein the displacement map for the next slice is different.

3. The method of claim 1, wherein the displacement maps used for the preceding slice are different.

4. The method of claim 1, wherein the displacement component occurs after the transmission of content-related encoded optical field data has been generated and received.

5. The method of claim 1, wherein the image is captured by a light field camera.

6. The method of claim 1, wherein generating the PSV slice further comprises: Provides depth information for each of the selected views at multiple depths; Pixels that match the depth provided by the selected view; as well as Determine the depth of the requested pixel of the selected view and the color of each pixel of the selected view at the determined depth.

7. An apparatus, said apparatus comprising: At least one processor, said at least one processor being configured to: Access multiple images captured by one or more cameras; The images therein represent multiple views corresponding to the same scene; Multiple planar scan volume PSV slices are generated from the image, and for each slice, a displacement map is calculated from the calibration parameters of the one or more cameras; wherein the PSV slice corresponding to a reference camera is located in a continuous depth plane in the coordinate system of the reference camera; and Generate a corresponding displacement map from a given slice for at least one of the next slice and the previous slice of the planar scan volume; Each slice of the PSV is provided with a displacement component, and the displacement component is appended to each slice that captures the displacement from the current slice to the next slice.

8. The device of claim 7, wherein the corresponding displacement maps for the next slice are different.

9. The apparatus of claim 7, wherein the corresponding displacement maps for the preceding slice are different.

10. The device of claim 7, wherein the displacement component occurs after the transmission of content-related encoded optical field data has been generated and received.

11. The device of claim 7, wherein the image is captured by a light field camera.

12. The apparatus of claim 7, wherein generating the PSV slice further comprises: Provides depth information for each of the selected views at multiple depths; Pixels that match the depth provided by the selected view; as well as Determine the depth of the requested pixel of the selected view and the color of each pixel of the selected view at the determined depth.

13. A method, the method comprising: Determine the camera calibration parameters associated with the first planar scanning volume PSV; The first slice of the PSV and its associated displacement map are obtained based on the determined camera calibration parameters; as well as Generate at least one previous slice or one next slice based on the first slice of the PSV and the camera calibration parameters; Each slice of the PSV is provided with a displacement component, and the displacement component is appended to each slice that captures the displacement from the current slice to the next slice.

Images