Training Method of Image Fusion Model, Image Generation Method and Their Devices

Abstract

The present disclosure relates to a training method for an image fusion model, an image generation method, and an apparatus thereof. A training method for an image fusion model based on a neural network is provided, comprising: receiving M input images of a specific scene, where M is an integer greater than or equal to 3; generating a three-dimensional global grid of the scene based on the M input images; selecting one input image from the M input images as a reference image; using M-1 non-reference images from the M input images to generate n mosaic images for the perspective of the reference image, where n is an integer greater than or equal to 2 and n is less than or equal to M-1; inputting the three-dimensional global grid and the n mosaic images as training images into a fusion model to generate a predicted image with the same perspective as the reference image; calculating an error between the predicted image and the reference image using a cost function; and adjusting a fusion weight of the image fusion model using the error to reduce the error.

Description

Technical Field

[0001] The present disclosure generally relates to a training method for an image fusion model, an image generation method, and an apparatus and medium thereof. Background Art

[0002] There is an increasing demand for real-time, realistic, easy-to-capture, free-viewing, and interactively navigated 3D content. Given that images of a scene from multiple perspectives (or viewpoints) have been acquired, it is desirable to easily acquire images from perspectives different from the multiple perspectives. Summary of the Invention

[0003] A brief overview of the present disclosure is provided below to provide a basic understanding of some aspects of the present disclosure. However, it should be understood that this overview is not an exhaustive overview of the present disclosure. It is not intended to identify key or important parts of the present disclosure, nor is it intended to limit the scope of the present disclosure. Its purpose is simply to present certain concepts of the present disclosure in a simplified form as a prelude to the more detailed description that will be given later.

[0004] According to one aspect of the present disclosure, a training method for a neural network-based image fusion model is provided, comprising: receiving M input images of a specific scene, where M is an integer greater than or equal to 3; generating a three-dimensional global grid of the scene based on the M input images; selecting one of the M input images as a reference image; using M-1 non-reference images among the M input images to generate n mosaic images for the perspective of the reference image, where n is an integer greater than or equal to 2 and n is less than or equal to M-1; inputting the three-dimensional global grid and the n mosaic images as training images into the fusion model to generate a predicted image with the same perspective as the reference image; calculating the error between the predicted image and the reference image using a cost function; and adjusting the fusion weights of the image fusion model using the error to reduce the error.

[0005] According to one aspect of the present disclosure, there is provided an image generation method, comprising: receiving L input images of a specific scene, where L is an integer greater than or equal to 2; generating a three-dimensional global grid of the scene based on the L input images; selecting a new perspective different from that of the L input images; generating n mosaic images for the new perspective using the L input images, where n is an integer greater than or equal to 2 and n is less than or equal to L; and inputting the three-dimensional global grid and the n mosaic images into an image fusion model obtained according to the above method to generate a predicted image of the new perspective.

[0006] According to another aspect of the present disclosure, a training device for a neural network-based image fusion model is provided, comprising: a memory on which instructions are stored; and a processor configured to execute the instructions stored on the memory to perform the above-mentioned training method for the neural network-based image fusion model.

[0007] According to another aspect of the present disclosure, an image generating apparatus is provided, including: a memory having instructions stored thereon; and a processor configured to execute the instructions stored on the memory to perform the above-mentioned image generating method.

[0008] According to another aspect of the present disclosure, a computer-readable storage medium is provided, which includes computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors execute the method according to any of the above aspects of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0010] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0011] Figure 1 An example of a scenario to which the present disclosure is to be applied is shown.

[0012] Figure 2 An example of an image and its 3D global mesh is shown.

[0013] Figure 3 An example of a mosaic image generation process is shown.

[0014] Figure 4 is a schematic diagram for explaining mosaic generation of different grids of a mosaic image.

[0015] Figure 5 is a schematic diagram for explaining mosaic generation of different grids of a mosaic image.

[0016] Figure 6 An example process of a training method for a neural network-based image fusion model according to the present disclosure is shown.

[0017] Figure 7 An example of a convolutional neural network architecture according to the present disclosure is shown.

[0018] Figure 8 An example of the prediction process for a new-view image is shown.

[0019] Figure 9An example of the training and fusion process of a neural network-based image fusion model according to an embodiment of the present disclosure is shown.

[0020] Figure 10 An exemplary configuration of a computing device that can implement embodiments according to the present disclosure is shown. DETAILED DESCRIPTION

[0021] The following detailed description is made with reference to the accompanying drawings, and the following detailed description is provided to assist in a comprehensive understanding of the various exemplary embodiments of the present disclosure. The following description includes various details to assist in understanding, but these details are considered to be examples only and are not intended to limit the present disclosure, which is defined by the appended claims and their equivalents. The words and phrases used in the following description are intended only to enable a clear and consistent understanding of the present disclosure. In addition, descriptions of well-known structures, functions, and configurations may be omitted for clarity and brevity. Those skilled in the art will recognize that various changes and modifications may be made to the examples described herein without departing from the spirit and scope of the present disclosure.

[0022] In scenarios such as 3D live broadcast of sports events, 3D real-time video surveillance, and 3D tourism, image fusion is often required. Taking live video broadcast as an example, Figure 1 As shown, after the cameras around the football field capture real-time video, it is expected to fuse the three-dimensional video images according to the client's perspective and send them to the client.

[0023] To this end, the present disclosure proposes an image fusion technology. Figure 1 In the application scenario shown, through the fusion technology disclosed in the present invention, watching a football game is like being in the game. The audience can watch the game from any angle and run in the virtual scene. The images they see vary depending on their position in the three-dimensional space.

[0024] According to an embodiment of the present disclosure, a training method for a neural network-based image fusion model includes: receiving M input images of a specific scene, where M is an integer greater than or equal to 3; generating a three-dimensional global grid of the scene based on the M input images; selecting one input image from the M input images as a reference image; using M-1 non-reference images from the M input images to generate n mosaic images for the perspective of the reference image, where n is an integer greater than or equal to 2 and n is less than or equal to M-1; inputting the three-dimensional global grid and n mosaic images as training images into the image fusion model to generate a predicted image with the same perspective as the reference image; calculating the error between the predicted image and the reference image using a cost function; and using the error to adjust the fusion weight of the image fusion model to reduce the error.

[0025] In one embodiment, the above method may further include the steps of iteratively generating a predicted image, calculating an error, and adjusting a fusion weight until the error is less than a predetermined value or the number of iterations reaches a predetermined number.

[0026] In the step of receiving M input images of a specific scene, after collecting a group of images, the present disclosure determines the spatial and geometric relationship (SFM) of the target through camera movement and generates a three-dimensional global mesh. Figure 2 An example of an image and its 3D global mesh is shown. As an alternative, when generating the 3D global mesh, the image depth can also be calculated by multi-view stereo vision (MVS) and a local depth map can be established.

[0027] In order to generate an image of a new perspective from an image taken from an existing perspective, the present invention Figure 3 In addition to the three-dimensional global grid shown, n mosaic images need to be generated. n is an integer greater than or equal to 2 and n is less than or equal to M-1.

[0028] In one embodiment, the step of generating n mosaic images for the perspective of the reference image may include: for each grid in the three-dimensional global grid: calculating the weights of the M-1 non-reference images on the grid; selecting n non-reference images with higher weights; obtaining warped projections of the n non-reference images on the grid; and using the warped projections of each grid to generate the n mosaic images, wherein the pixels at each grid in a first mosaic image among the n mosaic images are obtained by warping and projecting the pixels corresponding to the grid of the non-reference image with the highest weight on the grid, the pixels at each grid in a second mosaic image among the n mosaic images are obtained by warping and projecting the pixels corresponding to the grid of the non-reference image with the second highest weight on the grid, and so on for the third mosaic image up to the nth mosaic image.

[0029] Figure 3 An example of a mosaic image generation process is shown. Figure 4 is a schematic diagram for explaining mosaic generation of different grids of a mosaic image.

[0030] In the following description, n is taken as 4 (ie, four mosaic images are obtained) as an example. However, n can be set to any value greater than 1.

[0031] exist Figure 4 In , if we have already obtained five images from camera 1, camera 2, camera 3, camera 4 and camera 5, we need to calculate the four mosaic images with the highest weights of the new view x.

[0032] Triangular mesh m (denoted as t m ) is one of the many grids in the stereo 3D grid established by the above process. As an example, the cosine value of the angle between the normal of the lens of the camera that obtained the specific image and the normal of the grid in the 3D global grid can be used as the weight of the specific image on the grid. For example, the cosine value of the angle between the normal of the lens of camera c and the normal of t m The cosine of the angle between the normals is used as the weight W cm , and select the mosaic with the highest priority (weight) based on this weight. However, other parameters can also be used as weight W cm .

[0033] by Figure 4 Taking the triangular mesh m as an example, when the cosine of the angle is used as the weight, the weight of camera 1 in mesh m> the weight of camera 2 in mesh m> the weight of camera 3 in mesh m> the weight of camera 4 in mesh m> the weight of camera 5 in mesh m, that is, W 1m >W 2m >W 3m >W 4m >W 5m . Taking the triangle mesh n as an example, W 3n >W 4n >W 5n >W 2n >W 1n .

[0034] In the new view direction x, if the pixel of the image taken by camera c is projected into the area of grid m as p cm , combined with the calculated weights, we can get four mosaic images with the highest weights. The projections of grid m and grid n on these four mosaic images are as follows: Figure 5 As shown. The m triangle mesh and n triangle mesh of the mosaic image of the first priority (i.e., the aforementioned first mosaic image) are respectively projected by pixel distortion of the triangle meshes corresponding to camera 1 and camera 3; the m triangle mesh and n triangle mesh of the mosaic image of the second priority (i.e., the aforementioned second mosaic image) are respectively projected by pixel distortion of the triangle meshes corresponding to camera 2 and camera 4, and so on.

[0035] The above process generates a global 3D grid of 3D images and four high-priority mosaic images, totaling five images. These five images are then used as input to train a neural network-based image fusion model. In this disclosure, the image fusion model can be based on any neural network or any combination of neural networks, including feedforward neural networks, recurrent neural networks, convolutional neural networks, deep belief networks, generative adversarial networks, and so on. Below, we will use a convolutional neural network (CNN) as an example.

[0036] The training process is as follows Figure 6 As shown. First, a global grid is generated from M images of a specific scene at one time, and the global grid is used multiple times in the subsequent training. Then, M-1 images are selected from the M images to generate four mosaic images. These four mosaic images and the global grid are input into the forward neural network together to obtain the predicted image of the new perspective. The neural network contains image fusion weights. Initially, these image fusion weights can have arbitrary initial values. After obtaining the predicted image of the new perspective, the training loss is calculated based on the difference between the predicted image and the reference image. Then, the image fusion weights of the neural network model are adjusted through the reverse neural network (or back propagation).

[0037] The above process only gives the training process of deep neural network. The following will describe a specific convolutional neural network architecture example, such as Figure 7 As shown, the architecture includes an image contraction path on the left and an image expansion path on the right. Note that, as mentioned above, the present disclosure can adopt any neural network other than a convolutional neural network.

[0038] exist Figure 7 In the example shown, the contraction path can adopt a typical convolutional neural network architecture. For example, the following operations can be performed:

[0039] (1) Perform a 3x3 convolution operation on the image, followed by a ReLU operation.

[0040] (2) Repeat the first step based on the result of step (1).

[0041] (3) Downsampling is further achieved through a maximum pooling operation with a stride of 2 and a window size of 2x2.

[0042] (4) Return to step (1) and perform similar operations multiple times.

[0043] Figure 7 The expansion path for the right image in [1] can follow similar steps as the contraction path, but with an increased number of feature channels. The upsampling of the expansion path consists of the following steps:

[0044] (1) An up-convolution operation with a window size of 2x2, which reduces the number of channels.

[0045] (2) Perform the first 3x3 convolution operation, followed by the ReLU operation.

[0046] (3) Repeat step (2).

[0047] (4) Return to step (1) and perform similar operations multiple times.

[0048] Since each convolution operation loses some edge pixels, for example, an input image with a pixel count of 572x572 has an output resolution of 388x388 after the above processing, and 92 pixels are cropped on all sides of the image. Needless to say, the above pixel resolution and number of cropped pixels are merely examples, and the present disclosure may employ any other suitable resolution and number of cropped pixels.

[0049] The new perspective image prediction process of the feedforward neural network is as follows Figure 8 As shown, the relevant parameters are Figure 7 Take the image size shown as an example (the input image size is 572x572 and the output image size is 388x388):

[0050] (1) After the input image is processed by the aforementioned method of the present disclosure, a global grid image and four mosaic images with the highest priority (weight) are obtained, a total of five images.

[0051] (2) These five images are then input into a CNN neural network (i.e., a neural network-based image fusion model), which contains image fusion weights.

[0052] (3) Perform pixel-level weighted sum on the five input images. The details are as follows:

[0053] a. The value of row i and column j of frame m of image fusion weight is r mij , where 0 <m<6,(0,0)<(i,j)<(388,388)

[0054] b. The value of row i and column j in frame m of the cropped image is c mij , where 0 <m<6,(0,0)<(i,j)<(388,388)

[0055] c. The pixel value of the i-th row and j-th column of the predicted image is

[0056]

[0057] Wherein, k=n+1.

[0058] (4) Finally, the image prediction result of the new perspective is obtained.

[0059] The 3D image training process and the deep fusion process are as follows Figure 9 As shown. The dotted box area in the upper left corner is Figure 8 The new perspective image prediction process described in the preceding text obtains the prediction result through a neural network (e.g., a feed-forward neural network). For a complete training process of M input images for a specific scene, the complete training and fusion process is as follows: Figure 9 As shown:

[0060] (1) First, build a global grid from M images

[0061] (2) Select one image as the original reference image and use the perspective of this image as the new perspective

[0062] (3) Generate, for example, 4 mosaic images with the highest priority from the remaining M-1 images

[0063] (4) Taking the global grid from step (1) and the four mosaic images from step (3) as input, Figure 8 The image prediction process shown, for example, using a convolutional neural network, generates a predicted image of the new perspective in step (2). Compared with the five input images, the predicted image is cropped with a predetermined number of pixels on all sides.

[0064] (5) At the same time, the predetermined number of pixels of step (4) are cropped from the four sides of the original reference image of step (2) to obtain a cropped reference image.

[0065] (6) Input the image prediction result of step (4) and the cropped reference image of step (5) into the cost function to obtain the error.

[0066] (7) Through the inverse weight adjustment process of the convolutional neural network, the neural network weights are adjusted to reduce the prediction error.

[0067] Obviously, as mentioned before, an iterative process can be performed on the above steps to continuously adjust the neural network weights to continuously reduce the error.

[0068] As described above, in the case where the neural network is a convolutional neural network, the predicted image and the reference image may be cropped by a predetermined number of pixels on all sides compared to the input image, and the error may be calculated using the cropped predicted image and reference image.

[0069] In one embodiment, the above training method may further include training the neural network-based image fusion model multiple times by changing the image serving as the reference image among the M input images, or using different M input images.

[0070] In addition, the present disclosure may also include an image generation method, which includes: receiving L input images of a specific scene, where L is an integer greater than or equal to 2; generating a three-dimensional global grid of the scene based on the L input images; selecting a new perspective different from the perspective of the L input images; using the L input images to generate n mosaic images for the new perspective, where n is an integer greater than or equal to 2 and n is less than or equal to L; inputting the three-dimensional global grid and the n mosaic images into an image fusion model obtained according to the method as described above to generate a predicted image of the new perspective.

[0071] Figure 10 An exemplary configuration of a computing device 1200 capable of implementing embodiments according to the present disclosure is shown.

[0072] Computing device 1200 is an example of a hardware device to which the above-described aspects of the present disclosure can be applied. Computing device 1200 can be any machine configured to perform processing and / or computation. Computing device 1200 can be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal data assistant (PDA), a smartphone, an in-vehicle computer, or a combination thereof.

[0073] like Figure 10 As shown, computing device 1200 may include one or more components that can connect to or communicate with bus 1202 via one or more interfaces. Bus 1202 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus. Computing device 1200 may include, for example, one or more processors 1204, one or more input devices 1206, and one or more output devices 1208. The one or more processors 1204 may be any type of processor and may include, but is not limited to, one or more general-purpose processors or special-purpose processors (such as dedicated processing chips). Processor 1204 may be configured to implement the training method or image generation method described above. Input device 1206 may be any type of input device capable of inputting information into the computing device and may include, but is not limited to, a mouse, keyboard, touch screen, microphone, and / or remote control. Output device 1208 may be any type of device capable of presenting information and may include, but is not limited to, a display, speakers, video / audio output terminals, vibrators, and / or printers.

[0074] The computing device 1200 may also include or be connected to a non-transitory storage device 1214, which may be any non-transitory storage device that can implement data storage and may include, but is not limited to, a disk drive, an optical storage device, a solid-state memory, a floppy disk, a flexible disk, a hard disk, a magnetic tape or any other magnetic medium, a compact disk or any other optical medium, a cache memory and / or any other storage chip or module, and / or any other medium from which a computer can read data, instructions and / or code. The computing device 1200 may also include a random access memory (RAM) 1210 and a read-only memory (ROM) 1212. The ROM 1212 may store programs, utilities, or processes to be executed in a non-volatile manner. The RAM 1210 may provide volatile data storage and store instructions related to the operation of the computing device 1200. The computing device 1200 may also include a network / bus interface 1216 coupled to a data link 1218. The network / bus interface 1216 may be any type of device or system capable of enabling communication with an external device and / or network, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication device, and / or a chipset (such as Bluetooth TM equipment, 802.11 equipment, WiFi equipment, WiMax equipment, cellular communication facilities, etc.).

[0075] The present disclosure may be implemented as any combination of an apparatus, a system, an integrated circuit, and a computer program on a non-transitory computer-readable medium. One or more processors may be implemented as an integrated circuit (IC), an application-specific integrated circuit (ASIC), or a large-scale integrated circuit (LSI), a system LSI, a super LSI, or a ultra-LSI component that performs some or all of the functions described in the present disclosure.

[0076] The present disclosure includes the use of software, applications, computer programs, or algorithms. The software, applications, computer programs, or algorithms can be stored on a non-transitory computer-readable medium to enable a computer, such as one or more processors, to perform the steps described above and in the accompanying figures. For example, one or more memories can store the software or algorithm in the form of executable instructions, and one or more processors can execute a set of instructions for the software or algorithm to provide various functions according to the embodiments described in the present disclosure.

[0077] Software and computer programs (which may also be referred to as programs, software applications, applications, components, or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural language, an object-oriented programming language, a functional programming language, a logic programming language, or assembly language or machine language. The term "computer-readable medium" refers to any computer program product, apparatus, or device for providing machine instructions or data to a programmable data processor, such as magnetic disks, optical disks, solid-state storage devices, memories, and programmable logic devices (PLDs), including computer-readable media that receive machine instructions as computer-readable signals.

[0078] For example, computer-readable media may include dynamic random access memory (DRAM), random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store the desired computer-readable program code in the form of instructions or data structures and can be accessed by a general-purpose or special-purpose computer or a general-purpose or special-purpose processor. As used herein, disk or disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks generally reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of the above are also included within the scope of computer-readable media.

[0079] This disclosure provides a holographic image generation device based on, for example, a convolutional neural network. By reprojecting a set of input images from different perspectives to a new perspective, a weighted mosaic list from each perspective is created. The best candidate set for each pixel is then selected and fed into a CNN model for fusion. Multiple images from different scenes serve as a training dataset. For a given scene, one image serves as a reference view, and the CNN neural network is used to generate the reference image by fusing other images, thereby achieving training.

[0080] The subject matter of the present disclosure is provided as an example of devices, systems, methods, and programs for performing the features described in the present disclosure. However, in addition to the features described above, other features or variations are also contemplated. It is contemplated that any emerging technology may be used to implement the components and functions of the present disclosure in place of any of the technologies described above.

[0081] In addition, the above description provides examples and does not limit the scope, applicability or configuration set forth in the claims. Without departing from the spirit and scope of this disclosure, the function and arrangement of the elements discussed can be changed. Various embodiments can appropriately omit, replace or add various processes or components. For example, features described with respect to certain embodiments can be combined in other embodiments.

[0082] In addition, in the description of the present disclosure, the terms "first", "second", "third", etc. are used only for descriptive purposes and cannot be understood as indicating or implying relative importance and order.

[0083] Similarly, while operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed to achieve desired results. In certain circumstances, multitasking and parallel processing can be advantageous.

Claims

1. A training method for an image fusion model based on a neural network, comprising: Receive M input images of a specific scene, where M is an integer greater than or equal to 3; generating a three-dimensional global mesh of the scene based on the M input images; Selecting one of the M input images as a reference image; Using M-1 non-reference images from among the M input images, generating n mosaic images for the perspective of the reference image, comprising: for each grid in the three-dimensional global grid: calculating the weights of the M-1 non-reference images on the grid; selecting n non-reference images with higher weights; obtaining warped projections of the n non-reference images on the grid; and using the warped projections of each grid to generate the n mosaic images, wherein pixels at each grid in a first mosaic image among the n mosaic images are obtained by warping and projecting pixels corresponding to the grid of the non-reference image with the highest weight on the grid, pixels at each grid in a second mosaic image among the n mosaic images are obtained by warping and projecting pixels corresponding to the grid of the non-reference image with the second highest weight on the grid, and so on, wherein n is an integer greater than or equal to 2 and n is less than or equal to M-1; Inputting the three-dimensional global grid and n mosaic images as training images into the image fusion model to generate a predicted image with the same viewing angle as the reference image; Calculating an error between the predicted image and the reference image using a cost function; and The error is used to adjust the fusion weight of the fusion model to reduce the error.

2. The training method according to claim 1, wherein: The cosine value of the angle between the normal of the lens of the camera that obtains the non-reference image and the normal of the grid in the three-dimensional global grid is used as the weight of the non-reference image on the grid.

3. The training method according to claim 1, wherein: The neural network is a convolutional neural network, and Compared with the input image, the predicted image and the reference image are cropped by a predetermined number of pixels on all sides, and the error is calculated using the cropped predicted image and reference image.

4. The training method according to claim 3, wherein: The predicted image is expressed by the following formula: Among them, ij Represents the pixel value of the i-th row and j-th column of the predicted image, k=n+1, c mij represents the pixel value of the i-th row and j-th column of the m-th image in the three-dimensional global grid and the n mosaic images after cropping the predetermined number of pixels, represents the fusion weight corresponding to the i-th row and j-th column of the cropped 3D global grid and the m-th image in the n mosaic images.

5. The training method according to claim 1, wherein: The steps of generating a predicted image, calculating an error, and adjusting a fusion weight are iterated until the error is less than a predetermined value or the number of iterations reaches a predetermined number.

6. A method for generating an image, comprising: Receive L input images of a specific scene, where L is an integer greater than or equal to 2; generating a three-dimensional global mesh of the scene based on the L input images; selecting a new perspective different from the perspectives of the L input images; Generate n mosaic images for the new perspective using the L input images, where n is an integer greater than or equal to 2 and n is less than or equal to L; The three-dimensional global grid and n mosaic images are input into an image fusion model obtained by the method according to any one of claims 1 to 5 to generate a predicted image of the new perspective.

7. A training device for an image fusion model based on a neural network, comprising: a memory having instructions stored therein; as well as A processor configured to execute instructions stored on the memory to perform the method according to any one of claims 1 to 5.

8. An image generating device, comprising: a memory having instructions stored therein; as well as A processor configured to execute instructions stored on the memory to perform the method according to claim 6.

9. A computer-readable storage medium comprising computer-executable instructions, which, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 6.

Images