Stereoscopic image reconstruction method, system, storage medium and computer program product

Abstract

The present invention discloses a stereoscopic image reconstruction method, system, storage medium and computer program product, respectively extracting a low-resolution left image I left and the right image I right The initial features of left and F right ; The initial feature F left and F right As the input of the cross-view module, the output F CVB and #imgabs0# will output F CVB and #imgabs1# are respectively input into the first feature extraction module and the second feature extraction module to obtain the first channel separation result and the second channel separation result respectively; wherein, the first feature extraction module and the second feature extraction module have the same structure; the first channel separation result is aggregated with F CVB , get the left image I left The reconstructed image; aggregate the second channel separation result and #imgabs2# to get the right image I right The present invention can effectively perform super-resolution reconstruction on binocular images, and the reconstructed images are clear and natural with rich texture details.

Description

Technical Field

[0001] The present invention relates to the field of image reconstruction, and in particular to a stereoscopic image reconstruction method, system, storage medium and computer program product. Background Art

[0002] In recent years, artificial intelligence has become increasingly mature in theory and technology, bringing many conveniences to people's daily lives. With the rise of the 3D industry, the field of stereoscopic image processing has developed very rapidly, such as using deep learning to generate novel views of a scene from known views to deal with view synthesis problems and using convolutional neural networks to perform neural style transfer on stereo images, which has strongly promoted the development of 3D movies and VR. In addition, the widespread use of dual cameras on mobile devices has also promoted the research of stereoscopic images. However, due to the limitations of imaging equipment and network transmission, the generated stereoscopic images are usually of low resolution and difficult to apply to actual situations. Stereoscopic image super-resolution aims to restore high-resolution images from low-resolution images in 3D scenes, which is of great significance to 3D applications such as telemedicine and video analysis. For example, it can help doctors clearly observe the details of medical images and increase the possibility of disease diagnosis and cure; it can help relevant video surveillance viewers see the details of images, assist in solving cases, and improve public safety.

[0003] Single image super-resolution based on convolutional neural networks benefits from the powerful feature extraction and high representation capabilities of convolutional neural networks, which greatly surpasses traditional super-resolution methods. However, for stereo image pairs, there are two different views of the same scene, and the information of these two views complements each other, but these single image super-resolution methods cannot utilize this information. Reference-based image super-resolution seems to be helpful, but it requires high-resolution images as references. However, the left and right images in stereo vision are both low-resolution, so it cannot be applied to stereo image super-resolution. Video super-resolution processes temporally continuous frames, but stereo image pairs are two images with different views at the same time. Therefore, video super-resolution is not suitable for stereo image super-resolution either.

[0004] In order to utilize the left and right view information in stereo scenes, some stereo image super-resolution methods have been proposed. By jointly training a two-stage network, the disparity prior is learned from the stereo image dataset, and the disparity prior of binocular vision is used to achieve sub-pixel precision spatial super-resolution of the image. The epipolar constraint and attention mechanism are combined to calculate the feature similarity along the epipolar direction to obtain the stereo image correspondence information. A self-disparity attention mechanism is introduced to simultaneously aggregate the information of the self-image and the corresponding stereo image, so as to reconstruct a high-quality stereo super-resolution image pair. Although these methods have achieved good results, they are heavily dependent on the disparity prior. Therefore, these methods are based on the assumption that the contribution of one view to the other view is only along the epipolar direction (epiped is a special term in binocular images, see https: / / blog.csdn.net / weixin_29790771 / article / details / 112180342). However, unlike depth estimation or other stereo image processing tasks that require the calculation of disparity along the epipolar direction, the purpose of stereo image super-resolution is to utilize the complementary information between the left and right views. In fact, in addition to the epipolar direction, information in other locations of one view also helps to reconstruct the information of another view. Therefore, existing methods ignore information other than epipolar lines, which greatly limits their performance.

[0005] There are studies on restoring the left image in a binocular image in the prior art, but the prior art does not consider the correlation between the left and right images and the influence of the correlation on the reconstruction performance. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a stereoscopic image reconstruction method, system, storage medium and computer program product to improve the image reconstruction performance in view of the deficiencies in the prior art.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is: a stereoscopic image reconstruction method, comprising the following steps:

[0008] S1, extract the low-resolution left image I respectively left and the right image I right The initial features of left and F right ;

[0009] S2, the initial feature F left and F right As the input of the cross-view module, the output F CVB and

[0010] S3, output F CVB and Inputting the first feature extraction module and the second feature extraction module respectively, obtaining the first channel separation result and the second channel separation result respectively; wherein the first feature extraction module and the second feature extraction module have the same structure;

[0011] S4, the first channel separation results of polymerization and F CVB , get the left image I left The reconstructed image of the second channel is aggregated with Get the right image I right The reconstructed image.

[0012] In order to reconstruct the left image in the binocular image, the present invention utilizes the feature information in the right perspective; at the same time, when reconstructing the right image, the present invention utilizes the feature information in the left perspective, and adopts the parameter weight sharing method to fully utilize the semantic information extracted by the model in the reconstruction process of the left and right images. This semantic information can effectively promote the reconstruction effect, thereby further improving the performance of the overall model. Compared with super-resolution reconstruction of only one side of the image, reconstructing both sides of the image at the same time can more effectively utilize the semantic information between the two side images, and the parameter sharing method is also conducive to reducing the number of parameters and improving the efficiency of the model.

[0013] The specific implementation process of step S1 includes: left Perform a convolution operation, input the result of the convolution operation into two series-connected residual modules to obtain the initial features of the left image; right Perform a convolution operation, and input the result of the convolution operation into two series-connected residual modules to obtain the initial features of the image on the right. The initial feature extraction uses two superimposed residual modules, which can effectively extract the semantic information of the input image and fully retain the content features of the original image. The operation process is simple to calculate, fast to run, and has high computational efficiency.

[0014] In the present invention, F CVB The specific acquisition process includes:

[0015] 1) The initial feature F left Input the first residual module and convert F right Input the second residual module and get Q e and K e ; for Q e and K e After size transformation, we can get θ e and φ e ; For the initial feature F left Perform convolution processing to obtain the result after convolution processing γ e ; The initial feature F leftInput the third residual module and convert F right Input the fourth residual module and get Q g and K g ; for Q g and K g After size transformation, we can get θ g and φ g ; The initial feature F right Input the fifth residual module and get V g , for V g Perform pooling and transposition operations to obtain γ g ;

[0016] 2) Using θ e and φ e Calculate the similarity matrix of the right view to the left view in the epipolar direction

[0017]

[0018] 3) Cascade and γ e , and obtain the characteristic F in the epipolar direction e ;in,

[0019] The global feature F is calculated using the following formula: g : G(i,j) is the element in the i-th row and j-th column of G; M g (i,j) is M g The element in the i-th row and j-th column of ; τ is the set threshold;

[0020] 4) Calculate the output F using the following formula: CVB : F CVB =F e +F g .

[0021] The present invention simultaneously calculates the correlation of left and right perspective images along the epipolar direction and the global correlation. When calculating the global correlation, a pooling operation is used to expand the receptive field of the feature map, while also reducing the amount of calculation, thereby greatly improving the running speed.

[0022] In order to further effectively improve the range of the receptive field and extract useful information from the feature map, in step S3, the first channel separation result F cat The calculation formula is: cat =conv(H cat (F SPM1 ,F SPM2,...,F SPML )); Among them, F PPB =σ(conv(conv(F c / 4 )+F up ))·F';F'=cat(F'1,F'2,F'3,…,F L ); F2=F'2=Split2(LRelu(conv(F1))), F3=F'3=Split3(LRelu(conv(F2))), F L =conv(F L-1 ), t=1,2,……,L-1, L is the number of feature extractions; F SPM1 =F CVB ; LRelu(·) and conv(·) represent activation and convolution operations respectively; Split1(·), Split2(·) and Split3(·) are operations for channel separation according to channel dimensions; F c / 4 is the result obtained after F' passes through a convolution layer with a convolution kernel size of 1×1; F up To F pooling The result obtained after performing convolution and upsampling operations in sequence; F pooling =pooling(conv s (conv1(F'))), pooling(·) represents the pooling operation; conv1(·) and conv s (·) represent the convolutional layer with a kernel size of 1×1 and the strided convolutional layer respectively; H cat (·) indicates a cascade operation.

[0023] In step S4, the left image I left The reconstructed image The expression is: H up (·) represents the upsampling operation. This process can effectively integrate cross-view features with deep features to improve the reconstruction effect.

[0024] As an inventive concept, the present invention also provides a stereoscopic image reconstruction system, characterized in that it includes a computer device; the computer device is configured or programmed to execute the steps of the stereoscopic image reconstruction method of the present invention.

[0025] As an inventive concept, the present invention also provides a computer-readable storage medium, characterized in that it includes a program running in a computer device; the program is configured or programmed to execute the steps of the stereoscopic image reconstruction method of the present invention.

[0026] As an inventive concept, the present invention also provides a computer program product, including a computer program / instruction; characterized in that the computer program / instruction implements the steps of the stereoscopic image reconstruction method of the present invention when executed by a processor.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. The present invention can effectively perform super-resolution reconstruction on binocular images. The reconstructed images are clear and natural with rich texture details.

[0029] 2. The present invention has a fast running speed, a small number of parameters, and consumes less computing resources, which greatly increases the possibility of its large-scale use;

[0030] 3. By using the method of the present invention, low-resolution images can be used in the process of information transmission; by using the method of the present invention to restore the resolution of the image at the user end, the requirements for network bandwidth can be effectively reduced, and the pressure on data storage can be reduced;

[0031] 4. The present invention can effectively promote the application of super-resolution in binocular vision and stereo images. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a schematic diagram of the method of an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram of the pyramid sampling principle of an embodiment of the present invention;

[0034] Figure 3 This is a structural diagram of the space perception module of the present invention;

[0035] Figure 4 This is a structural diagram of the pixel sensing module of the present invention. DETAILED DESCRIPTION

[0036] The present invention proposes a cross-view capture network (CVCnet) for stereoscopic image super-resolution by using global context and local features extracted from two views. Specifically, the present invention designs a cross-view block to capture various feature embeddings from stereoscopic views. In addition, a cascaded spatial perception module is proposed to redistribute each position in the feature map according to its weight, so as to extract features more efficiently. Experiments show that the CVCnet proposed in the present invention outperforms the latest image super-resolution methods and can make the reconstructed image clearer and more detailed.

[0037] The overall structure of the model of the embodiment of the present invention is as follows Figure 1As shown in the figure, the model consists of an initial feature extraction module (IFE), a cross-view module (CVB), and a spatial perception module (SPM). CVCnet takes a low-resolution stereo image pair as input and performs super-resolution reconstruction on the left image. Figure 1 middle, I left and I right Represent the low-resolution images on the left and right sides of the input respectively. The first step of the network extracts multi-channel features through the initial feature extraction module, which can be expressed as:

[0038] F left =H IFE (I left ) (1)

[0039] F right =H IFE (I right ) (2)

[0040] Among them, F left and F right They represent the input features extracted from the input stereo image pair. IFE (·) represents the initial feature extraction module, which contains a convolutional layer and two consecutive residual modules. At the same time, the initial feature extraction on the left and right sides share the same weights. left and F right Finally, they are used as input to the cross-view block, where global and local feature correlations can be effectively learned to enhance the feature representation of pairs of images from similar modalities. This step can be expressed as:

[0041] F CVB =H CVB (F left ,F right ) (3)

[0042] Among them, H CVB (·) indicates the cross-view module, F CVB is the feature extracted by this module. Then F CVB It is further fed into the cascaded spatial perception module to effectively extract features:

[0043]

[0044] Among them, F SPM1 , F SPM2 , ..., F SPMn They represent the features extracted from the cascade space. SPM1 (·), H SPM2 (·), ..., H SPMn (·) represents the corresponding spatial perception module. Then FSPM1 , F SPM2 , ..., F SPMn To cascade:

[0045] F cat =conv(H cat (F SPM1 ,F SPM2 ,...,F SPMn )) (5)

[0046] Among them, H cat (·) indicates a cascade operation, and conv(·) indicates a convolution with a kernel size of 1×1. cat represents the cascaded features from SPM. Finally, using F cat Aggregate with CVB features to reconstruct the corresponding SR image:

[0047]

[0048] in represents the reconstructed left image, H up (·) represents the upsampling operation. We use pixel shuffling to upsample the features to the required size. cat and F CVB The addition is performed by cascading.

[0049] The specific steps of the embodiment of the present invention are as follows:

[0050] Step 1: First, the low-resolution left image I left and the right image I right Send it to the initial feature extraction module (IFE) to get F left and F right , F left and F right Respectively represent the input features of the left and right images extracted from the input stereo image pair. IFE consists of a convolutional layer and two consecutive residual modules. At the same time, the initial feature extraction of the left and right sides shares the same weights.

[0051] Step 2: In the epipolar direction, firstly, the input feature F left and F right The residual module is used for conversion to obtain Q e and K e Here, Q e and K e are all tensors of size H×W. The number of images processed in a batch in the tensor is B, the number of channels of the feature map is C; H is the height of the feature map; W is the width of the feature map. In order to calculate the similarity matrix, Q e and K eThe size of is re-transformed to W×C. By transforming and multiplying θ e and φ e , we can get the similarity matrix of the right view to the left view in the epipolar direction:

[0052]

[0053] in represents the similarity matrix of the right view to the left view in the epipolar direction. Similarly, can be obtained in the same way. e Re-transformed to γ e , γ e The size of is B'×W×C and is multiplied with the similarity matrix:

[0054]

[0055] Then, cascade and γ e , and obtain the characteristic F in the epipolar direction e .

[0056] In the embodiment of the present invention, the initial feature F left Input the first residual module and convert F right Input the second residual module, the output of the first residual module and the output of the second residual module are convolved to obtain Q e and K e ; for Q e and K e Transpose and get θ e (size is W×C) and φ e (size is W×C); for the initial feature F left Perform convolution processing to obtain V e , V e After transposing, we get γ e (size is W×C); the initial feature F left Input the third residual module and convert F right The fourth residual module is input, and the output of the third residual module and the output of the fourth residual module are convolved to obtain Q g and K g ; for Q g Perform a transpose operation on K g Perform pooling and transposition operations in sequence to obtain φ g (size is C×N) and θ g (size is C×Z); the initial feature F rightInput the fifth residual module, and perform convolution operation on the output of the fifth residual module to obtain V g , for V g Perform pooling and transposition operations in sequence to obtain γ g (size is C×Z);

[0057] Step 3. In order to mine the global context features between the left and right views (i.e., the right image and the left image), in the process of obtaining global features, Q g Transformed into φ g ,φ g The size of is C×N. For K g , using pyramid sampling to generate features θ g , whose size is B×C×Z. Figure 2 As shown in Figure 2, convolution kernel sizes of 12, 15, 18, and 21 are used for K convolutions. g Perform feature sampling.

[0058] The global similarity can be calculated as:

[0059]

[0060] Among them, M g Indicates the global similarity of the left time to the right view. In order to avoid introducing too much redundant information in the left image, the parts with lower values ​​in the similarity matrix are filtered:

[0061]

[0062] Where G represents a mask with a value of 0 or 1. If the value of the similarity matrix at (i, j) is greater than the set threshold τ, its value is set to 1, otherwise it is set to 0. Then, perform matrix dot multiplication on the mask and the similarity matrix:

[0063]

[0064] Among them, G·M g represents the matrix after filtering, which retains the regions with high similarity in the global similarity matrix. Through this design, the proposed global feature F g It contains the complementary information of the left image to the right image. G(i,j) is the element in the i-th row and j-th column of G; M g (i,j) is M g The element in the i-th row and j-th column of e and F g To add:

[0065] F CVB =F e +Fg (12)

[0066] F CVB It also contains epipolar information and global information between images across the field of view.

[0067] Similarly, in order to obtain the total contribution of the right view to the left view, calculate Process and calculation of F CVB The process is similar, calculating When the input of each residual module is equal to the calculated F CVB The input is different, the calculation When the initial feature F right Input the first residual module and convert F left Input the second residual module and get and right and After size transformation, we can get and For the initial feature F right Perform convolution processing to obtain the result after convolution processing The initial feature F right Input the third residual module and convert F left Input the fourth residual module and get and right and After size transformation, we can get and The initial feature F left Input the fifth residual module and get right Perform pooling and transposition operations to obtain Subsequent calculation process and F CVB The calculation process is similar.

[0068] Step 4. Figure 3 As shown in Figure 1, for the input of SPM, a convolutional layer with a kernel size of 3×3 and LRelu are used to extract input features. After this step, the channel segmentation method is applied to the extracted features, and these features are divided into two parts according to the channel dimension. One of them is sent to the next feature extraction module, and the other is retained for feature fusion. This process can be expressed as:

[0069]

[0070] Where LRelu(·) and conv(·) represent activation and convolution operations respectively. Split1(·), Split2(·) and Split3(·) are operations for channel separation based on channel dimensions. F1, F2 and F3 represent the features sent to the next feature extraction module. F1, F'2, F'3 and F4 are cascaded as the input of the pixel perception module:

[0071] F'=cat(F'1, F'2, F'3, F4) (14)

[0072] Then, F' is enhanced by a pixel-aware module.

[0073] The structure of the pixel perception module is as follows Figure 4 As shown. For the input feature F', a convolution layer with a convolution kernel size of 1×1 is used to reduce the number of channels and obtain F c / 4 , in order to reduce the consumption of computing resources. Then a gradual approach is adopted to gradually expand the receptive field of the feature. The strided convolution and pooling layers are used to compress the input features. This operation can be expressed as:

[0074] F pooling =pooling(conv s (conv1(F'))) (15)

[0075] Among them, conv1(·) and conv s (·) respectively represent convolution with kernel size of 1×1 and strided convolution, and pooling(·) represents pooling operation. Then, a series of convolution kernel activation operations with kernel size of 3×3 are applied to the upsampling layer. pooling And get F up Then, F c / 4 With F up Add and calculate the weight of each pixel through the sigmoid function, and multiply the weight to the corresponding feature map:

[0076]

[0077] Among them, conv(·) represents the convolution with a kernel size of 1×1, σ(·) represents the activation function, and F PPB Represents the output of the pixel-aware module.

[0078] Get F PPB Afterwards, F PPB It is fed into a convolution with a kernel size of 1×1 and gets F' PPB , F' PPB and Add them together to get the output of the i-th spatial perception module:

[0079]

[0080]

[0081] As can be seen from the above table, the PSNR (dB) (peak signal-to-noise ratio) and SSIM (structural loss) values ​​of the method of the present invention on the KITTI2015, KITTI2012, Middlebury and FLICKR1024 datasets are greater than those of the prior art, indicating that the method of the present invention greatly improves the image reconstruction effect compared with the prior art.

[0082] References

[0083] [1] C. Dong, CC Loy, K. He, and X. Tang, "Image super-resolution using deepconvolutional networks," IEEE Trans.Pattern Anal.Mach.Intell., vol.38, no.2, pp.295–307, Feb.2016.

[0084] [2] J.Kim, JKLee, and KMLee, “Accurate image super-resolution using very deep convolutional networks,” in Proc.IEEE Conf.Comput.Vision PatternRecognit., 2016, pp. 1646–1654.

[0085] [3]G.Huang, Z.Liu, L.Van Der Maaten, and KQWeinberger, “Denselyconnected convolutional networks,” in Proc.IEEE Conf.Comput.VisionPatternRecognit., 2017, pp. 4700–4708.

[0086] [4]Y.Tai,J.Yang,and X.Liu,“Image super-resolution via deeprecursiveresidual network,”in Proc.IEEE Conf.Comput.Vision Pattern Recognit.,2017,pp.3147–3155.

[0087] [5]N.Ahn,B.Kang,and K.A.Sohn,“Fast,accurate,and lightweightsuper-resolution with cascading residual network,”in Proc.Eur.Conf.Comput.Vision,2018,pp.252–268.

[0088] [6]Z.Li,J.Yang,Z.Liu,X.Yang,G.Jeon,and W.Wu,“Feedback networkforimage super-resolution,”in Proc.IEEE Conf.Comput.Vis.PatternRecognit.,2019,pp. 3862–3871.

[0089] [7]D.S.Jeon,S.Baek,I.Choi,and M.H.Kim,“Enhancing thespatialresolution of stereo images using a parallax prior,”in Proc.IEEEConf.Comput.Vision Pattern Recognit., 2018,pp.1721–1730.

[0090] [8]L.Wang,Y.Guo,Y.Wang,Z.Liang,Z.Lin,J.Yang,and W.An,“Parallaxattention for unsupervised stereo correspondence learning,”IEEE Trans.PatternAnal.Mach.Intell.,to be published.DOI:10.1109 / TPAMI.2020.3026899.

[0091] [9]X.Zhu,K.Guo,H.Fang,L.Chen,S.Ren and B.Hu,"Cross View Capture forStereo Image Super-Resolution,"in IEEE Transactions on Multimedia,doi:10.1109 / TMM.2021.3092571.

Claims

1. A stereoscopic image reconstruction method, characterized in that: The following steps are involved: S1, extract the low-resolution left image I respectively left and the right image I right The initial features of left and F right ; S2, the initial feature F left and F right As the input of the cross-view module, the output F CVB and F CVB The specific acquisition process includes: 1) The initial feature F left Input the first residual module and convert F right Input the second residual module and get Q e and K e ; for Q e and K e After size transformation, we can get φ e and θ e ; For the initial feature F left Perform convolution processing to obtain the result after convolution processing γ e ; The initial feature F left Input the third residual module and convert F right Input the fourth residual module and get Q g and K g ; for Q g and K g After size transformation, we can get θ g and φ g ; The initial feature F right Input the fifth residual module and get V g , for V g Perform pooling and transposition operations to obtain γ g ; 2) Using θ e and φ e Calculate the similarity matrix of the right view to the left view in the epipolar direction 3) Cascade and γ e , and obtain the characteristic F in the epipolar direction e ;in, The global feature F is calculated using the following formula: g : G(i,j) is the element in the i-th row and j-th column of G; M g (i,j) is M g The element in the i-th row and j-th column; τ is the set threshold; 4) Calculate the output F using the following formula: CVB :F CVB =F e +F g ; S3, output F CVB and Inputting the first feature extraction module and the second feature extraction module respectively, obtaining the first channel separation result and the second channel separation result respectively; wherein the first feature extraction module and the second feature extraction module have the same structure; S4, the first channel separation results of polymerization and F CVB , get the left image I left The reconstructed image of the second channel is aggregated with Get the right image I right The reconstructed image.

2. The stereoscopic image reconstruction method according to claim 1, characterized in that: The specific implementation process of step S1 includes: left Perform a convolution operation, input the result of the convolution operation into two series-connected residual modules to obtain the initial features of the left image; right A convolution operation is performed, and the result of the convolution operation is input into two series-connected residual modules to obtain the initial features of the image on the right.

3. The stereoscopic image reconstruction method according to claim 1, characterized in that: In step S3, the first channel separation result F cat The calculation formula is: cat =conv(H cat (F SPM1 ,F SPM2 ,...,F SPML )); Among them, F PPB =σ(conv(conv(F c / 4 )+F up ))·F';F'=cat(F1',F2',F3',…,F L ); F2=F2'=Split2(LRelu(conv(F1))),F3=F3′=Split3(LRelu(conv(F2))),F L =conv(F L-1 ), t=1,2,……,L-1, L is the number of feature extractions; F SPM1 =F CVB ; LRelu(·) and conv(·) represent activation and convolution operations respectively; Split1(·), Split2(·) and Split3(·) are operations for channel separation according to channel dimensions; F c / 4 is the result obtained after F' passes through a convolution layer with a convolution kernel size of 1×1; F up To F pooling The result obtained after performing convolution and upsampling operations in sequence; F pooling =pooling(conv s (conv1(F'))), pooling(·) represents the pooling operation; conv1(·) and conv s (·) represent the convolutional layer with a kernel size of 1×1 and the strided convolutional layer respectively; H cat (·) indicates a cascade operation.

4. The stereoscopic image reconstruction method according to claim 3, characterized in that: In step S4, the left image I left The reconstructed image The expression is: H up (·) indicates an upsampling operation.

5. A stereoscopic image reconstruction system, characterized in that: The invention comprises a computer device; the computer device is configured or programmed to execute the steps of the method according to any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that: The invention comprises a program running in a computer device; the program is configured or programmed to execute the steps of the method according to any one of claims 1 to 4.

7. A computer program product comprising a computer program / instructions; characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images