Method and device for composing images

By using inertial data for homograph transformations and projecting images onto a single plane, the method efficiently composes panoramic images on mobile devices, overcoming the limitations of limited computing power and enabling high-quality, real-time image composition for applications like vehicle inspections.

WO2024241265A1PCT designated stage expired Publication Date: 2024-11-28PERCEPTOLAB SRL +1
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
PCT/IB2024/055027
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-25
Filing Date
2024-05-23
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

The complexity of image composition for panoramic images, particularly on mobile devices with limited computing power, results in long processing times and low-resolution images, making it difficult to achieve high-quality image composition in real-time, especially when dealing with large objects or multiple composite images.

Method used

A method utilizing inertial data from an image acquisition device to perform homograph transformations and project images onto a single image projection plane, reducing computational load and enabling efficient image composition on mobile devices, such as smartphones, by approximating keypoint parameters and simplifying alignment procedures.

Benefits of technology

This approach allows for the creation of high-resolution panoramic images with reduced processing time and increased efficiency, suitable for detailed documentation tasks like vehicle inspections, providing objective and standardized image documentation without the need for post-processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure IB2024055027_28112024_PF_FP_ABST
    Figure IB2024055027_28112024_PF_FP_ABST
Patent Text Reader

Abstract

A method for image composition includes : acquiring (100) a sequence of input images (I0,..., IM) along an acquisition path (D) using an image acquisition device (4) equipped with an inertial measurement unit (5), wherein the acquisition path (D) is defined by rotating the image acquisition device (4); associating (105) each input image (I0,..., IM) with respective inertial data provided by the inertial measurement unit (5) and corresponding to respective acquisition conditions; connecting (110) adjacent input images (IJ-1, IJ) in pairs using homograph transformations defined on the basis of the inertial data associated with the input images (IJ-1, IJ); defining an image projection plane (PI); projecting (120) the input images (I0,..., IM) onto the image projection plane (PI), obtaining projected images (Io',..., IM'); and composing (125) an output image (IOUT) from the projected images (I0',..., IM').
Need to check novelty before this filing date? Find Prior Art

Description

[0001] METHOD AND DEVICE FOR COMPOSING IMAGES

[0002] Cross-Reference to Related Applications

[0003] This Patent Application claims priority from Italian Patent Application No. 102023000010641 filed on May 25, 2023, the entire disclosure of which is incorporated herein by reference .

[0004] Technical field

[0005] The present invention relates to a method and device for composing images.

[0006] Background

[0007] It is well known that the acquisition of panoramic images first requires the acquisition of a series of partial images or input images that are then composed and merged. The composition process is rather complex and requires processing capacity that is not always available on mobile devices, such as smartphones or tablets. The possibility of using mobile devices, which would be extremely useful and versatile precisely because of their convenience, is therefore limited, for example, by rather long processing times or low-resolution images. When there is a need for images in high resolution formats, which are in themselves available on mobile devices, the available computing power may not be sufficient and at the very least may not allow images of the desired amplitude to be composed in real time. In some cases, e.g. when a large number of composite images have to be produced, the power reserve of the portable device can also be significantly affected.

[0008] Image composition involves computationally onerous processing steps, especially in order to precisely determine the point of view corresponding to each image in the acquired series and the distortions (warps) that have to be applied accordingly in order to obtain a coherent final image. In practice, it is necessary to determine how the change in viewpoint between successive input images and possibly other shooting parameters affect the projection of the scene onto the sensor and, consequently, the dimensional and positional relationships of the image elements. By identifying a coordinate transformation between successive images that approximates the actual change in viewpoint, it is possible to compensate for distortion effects and compose the input images in a consistent manner. However, the search and application of the coordinate transformation requires the solution of a complex optimisation problem with a large number of variables. If the available computing power is limited, as is often the case with portable devices, the processing complexity may not come at an acceptable cost in terms of time or the quality of the result.

[0009] Summary

[0010] The purpose of the present invention is therefore to provide a method and device for capturing panoramic images that will overcome or at least mitigate the described limitations.

[0011] According to the present invention, a method for composing images is performed, comprising: acquiring a sequence of input images along an acquisition path using an image acquisition device equipped with an inertial measurement unit, wherein the acquisition path is defined by rotating the image acquisition device; associating each input image with respective inertial data provided by the inertial measurement unit and corresponding to respective acquisition conditions; connecting adjacent input images in pairs using homograph transformations defined on the basis of the inertial data associated with the input images; defining an image projection plan; projecting the input images onto the image projection plane, obtaining projected images; and composing an output image from the projected images.

[0012] The use of inertial data associated with the images and the projection of the input images onto an image projection plane drastically reduces the computational load required. In particular, image connection and alignment procedures would require heavy optimisation from the identified keypoints. However, on the one hand, although noisy, inertial data make approximations of the parameters defining the homograph transformations immediately available and, on the other hand, the approximation of the projection onto a single image projection plane reduces the number of operations required. Consequently, the method according to the invention is much more suitable for direct implementation on mobile devices, in which computing power is relatively modest, and can also be flexibly exploited in the field directly by users without the need for post-processing steps.

[0013] In particular, the method and portable device described are suitable for use in tasks where detailed images of relatively large objects may be useful, for example for control and documentation purposes. One case in point is vehicle rental. A critical aspect in rental procedures concerns the accuracy in ascertaining the condition of the vehicles before and after rental. Normally, vehicles are visually inspected upon delivery and upon return and any external defects, as well as the condition of the tyres, are noted on a card. However, inspection and reporting can only be done in a rather cursory manner and are often open to dispute. Thanks to the method and the portable device according to the invention, it is possible to obtain objective, accurate and easy-to-file documentation for possible subsequent inspections very quickly. As already mentioned, in fact, the efficiency of the method allows it to be implemented on devices with not particularly high computing power and now available in practically any situation, such as smartphones. It is also possible to exploit the higher resolutions of the image sensors normally integrated into portable devices to produce images with very high levels of detail.

[0014] In addition, having only one overall image offers further advantages. Firstly, more contextual information of any damage that might not be clear to an outside observer from single, very close-up images is made available. Secondly, the collection of images can be standardised, greatly simplifying the review work, both by a human operator and, above all, automatically.

[0015] According to the present invention, a portable device is also provided comprising: a processing unit; an image acquisition device controlled by the processing unit; and an inertial measurement unit; wherein the processing unit is configured to: cause acquisition of a sequence of input images along an acquisition path using the image acquisition device, wherein the acquisition path is defined by rotating the image acquisition device; associate each input image with respective inertial data provided by the inertial measurement unit and corresponding to respective acquisition conditions; connect adjacent input images in pairs using homograph transformations defined on the basis of the inertial data associated with the input images; define an image projection plan; project the input images onto the image projection plane, obtaining projected images; and compose an output image from the projected images.

[0016] Brief description of the figures

[0017] The present invention will now be described with reference to the attached drawings, which show a non-limiting embodiment thereof, wherein:

[0018] - Figure 1 is a schematic representation of a device for the acquisition of panoramic images in accordance with an embodiment of the present invention;

[0019] - Figure 2 is a simplified flow chart of a method for capturing panoramic images in accordance with an embodiment of the present invention;

[0020] - Figure 3 illustrates, in simplified plan view from above, the execution of a step of the method according to the present invention;

[0021] - Figure 4 is a schematic representation of the spatial distribution of input images acquired along a path according to the method according to the present invention; - Figure 5 is a more detailed flow chart relating to a step of the method according to the present invention;

[0022] - Figure 6 is a schematic representation of an image transformation used in the method according to the present invention; and

[0023] - Figure 7 is a more detailed flow chart of further steps of the method according to the present invention.

[0024] Description of embodiments

[0025] With reference to Figure 1, a portable device configured for image acquisition is indicated by reference number 1 and, in one embodiment, may be a smartphone or tablet. The portable device comprises a processing unit 2, a memory 3, a camera 4 and an Inertial Measurement Unit (IMU) 5. The processing unit 2 controls the camera 4 and the inertial measurement unit 5 and is coupled in communication to provide commands and receive data. For example, the processing unit 2 is configured to receive inertial data from the inertial measurement unit, to command image acquisition and to associate respective inertial data with the acquired images.

[0026] The memory 3 contains instructions which, when executed by processing unit 2, cause the processing unit 2 to execute a method for image composition as described below. In particular, the processing unit 2, executing the instructions contained in memory 3, implements a feature finder 10, a feature matcher 11, a matching validator 12, a parameter estimator 15, a parameter refiner 16, a pitch corrector 17, a planar warper 18 and an image composer 19. The operations carried out by the processing unit 2 and its components will be described in detail below and are specifically selected and optimised to acquire and compose images of essentially planar objects at a viewing angle of no more than 150° from a distance generally comprised between 1 and 5 metres (e.g. the flank of a car).

[0027] Referring to Figure 2, in an initial step (block 100) a sequence of input images I0, •••, IMis acquired through the portable device 1 during a scan at M+1 positions 0, ..., M along an acquisition path D, i.e. an arc in a determined horizontal plane around the user's own position. Advantageously, the acquisition can be guided by reference elements displayed on the screen of the portable device 1, such as bands and / or field boundaries, to facilitate and as far as possible standardise the capture task even when performed manually and without media.

[0028] At this stage, the processing unit 2 activates the camera 4 and prompts the user to carry out the scan by moving the shooting point along the acquisition path D, which covers a programmed arc of total amplitude αT, e.g. 120°. Ideally, the input images I0, •••, IMare acquired by rotating the camera around a single vertical axis, without translational movements, as shown in Figure 3; more realistically, the user's movement also includes translational components predominantly in a horizontal plane. During scanning, the processing unit 2 commands the acquisition of sequential images based on the data provided by the inertial measurement unit 5 and the field amplitude cij framed by the camera 4 and associated with each input image I0, •••,IM(Figures 3 and 4). In practice, the input images I0, ...,IMare automatically acquired during scanning as a function of the rotation of the camera 4, so that there is partial overlap between adjacent input images I0, •••,IM. "Adjacent images" herein and in the following means pairs of input images that are in consecutive positions along the acquisition path D. The definition "adjacent images" includes, in particular, pairs of images that are partially overlapping and pairs of images that are not fully aligned with each other.

[0029] Each input image is associated with respective inertial data provided by the inertial measurement unit 5 and indicative of an angular position of the camera 4 at the time of acquisition (block 105). The angular position can be defined with respect to a reference angular position or to an angular position of the previously acquired input image.

[0030] The processing unit 2 then performs a procedure of connecting adjacent input images I0, •••,IM(block 110), a procedure of aligning the input images I0, ...,IM(block 115) and a procedure of projecting the input images I0, •••, IM(block 120) onto an image projection plane determined in the alignment procedure, as explained in detail below. Finally (block 125), the processing unit 2 composes the input images I0, ...,IMinto a single output image IOUT.

[0031] The connection procedure is carried out by the feature finder 10, the feature matcher 11 and matching validator 12 according to the procedure shown in Figure 5. The input images are acquired and processed sequentially and at each iteration a generic current input image IJis connected to the previous input image IJ-1.

[0032] After selecting the current input image IJand the previous input image IJ-1, which are adjacent to each other, the feature finder 10 provides local feature descriptors that identify initial image keypoints KP0in the current input image IJ(block 150). A local feature descriptor defines a representation of the image information contained in a neighbourhood of each point. The initial image keypoints KP0are selected based on the values of the corresponding local feature descriptors. Examples of algorithms that can be applied to generate local feature descriptors are Oriented FAST and Rotated BRIEF (ORB), Binary Robust Invariant Scalable Keypoints (BRISK).

[0033] In addition (block 155), the feature finder 10 applies, independently to each current input image IJ, an Adaptive

[0034] Non-Maximal Suppression (ANNS) algorithm that has the effect of ensuring a homogeneous distribution of initial image keypoints KP0across the entire image field. This prevents the initial image keypoints KP0being grouped in clusters or redundant in image portions, which benefits efficiency.

[0035] Next (block 160), the feature matcher 11 defines potential matches between the initial image points KP0of the current input image IJand the image keypoints of the previous input image IJ-1. Potential matches are determined, taking into account eligibility criteria, based on a comparison of the local feature descriptors of image keypoints of the current input image IJand the local feature descriptors of the initial image keypoints KP0of the previous input image IJ-1. The image keypoints of the current input image IJwith potential matches in adjacent input images are selected, while the others are eliminated.

[0036] The image keypoints selected for the current input image IJ, indicated briefly for convenience by the notation KPJ, are then temporarily associated with corresponding image keypoints KPJ-1of the previous input image IJ-1. In this way, the feature matcher 11 defines pairs of image keypoints corresponding to [KPJ, KPJ-1], [KP2J, KP2J-1], ..., [KPQJ, KPQJ- i], where Q is the number of selected pairs.

[0037] The potential matches are then validated by the matching validator 12, which defines a homograph transformation between the current input image IJand the corresponding previous input image IJ-1(block 165). The homograph transformation is an estimate of the transformation between the current input image IJand the previous input image IJ-1and is determined based on the features of the optical sensor and inertial data associated with the current input image IJand the previous input image IJ-1, respectively, with the assumption that adjacent images are linked by pure rotation.

[0038] In one embodiment, the homograph transformation between two images is defined based on a 3x3 global rotation matrix, which can generally be obtained by separately determining the differences between the Euler angles corresponding to the two images in question and concatenating the partial rotation matrices relating to each axis. For example, the partial rotation matrices X, Y and Z respectively around x, z axes of a fixed coordinate system can be described as where θX, θYand θzare the Euler angles, and the generic global rotation matrix R is determined by composing the partial rotation matrices

[0039] R=XYZ (2)

[0040] The homograph transformation between points PI1, PI2 corresponding to two generic images II, 12 is therefore defined by with

[0041] In equations (3) and (4), KI and K2 are the intrinsic matrices of image II and image 12, respectively, and are provided by the camera 4. Intrinsic matrices account for how the scene captured is projected onto the camera 4 sensor as a function of parameters that characterise the sensor-lens system.

[0042] From equations (l)-(4), the homograph transformation between a generic point PJ-1of the previous input image IJ-1and the corresponding point PJof the current input image IJis provided by

[0043] For simplicity, in the following, the homograph transformations will be identified with the respective associated matrices

[0044] HJ-1, J.

[0045] The matching validator 12 (block 170) applies the homograph transformation to the image keypoints KPJ-1of the previous input image IJ-1that have potential matches with the current input image IJas defined in block 160. In practice (see also Figure 6), the matching validator 12 determines projections KP'J-1of the image keypoints KPJ-1of the previous input image IJ-1into the current input image IJsuch as

[0046] KP'j-1= Hj-1,jKPj-1(6)

[0047] Then (block 175) the matching validator 12 selects a pair comprising an image keypoint KPJ,Tof the current image IJand the projection KP'J-1,Tof the corresponding image keypoint KPJ-1,Tof the previous input image IJ-1(T = 1, 2,

[0048] Q) and applies a validation criterion (block 180). For example, the validation criterion may be based on a measurement of the Euclidean distance in the current input image IJbetween the image keypoint KPJ,Tof the current image IJand the projection KP'J-1,Tof the corresponding image keypoint KPJ-1,Tof the previous input image IJ-1. In particular, the validation criterion can be based on a comparison between the measured Euclidean distance and a threshold distance DTH1:

[0049] ||KPJ,T— KP'j_1T||< DTH1(7)

[0050] If the validation criterion is satisfied, the correspondence between the image keypoint KPJ,Tof the current image IJand the projection KP'J-1,Tof the corresponding image keypoint KPJ-1,Tof the preceding input image IJ-1is confirmed and the corresponding pair of points KPJ,T, KPJ-1,Tis preserved (block 185); otherwise, the matching under consideration is not validated and the image keypoint KPJ,T of the current image IJand the image keypoint KPJ-1,Tof the previous input image IJ-1are removed from the list of matching points (block 190).

[0051] The procedure of selecting a pair and applying the validation criterion is applied iteratively (blocks 195, 175) until the pairs are exhausted and the connection procedure ends (blocks 195, 197).

[0052] In one embodiment, the same procedure is performed bidirectionally and is also applied based on the inverse transformation from the current image IJto the previous image IJ-1, i.e. by calculating the projections KP'J,Tof the image keypoints KPJof the current input image IJinto the previous input image IJ-1.

[0053] In particular, for the inverse transformation, the matching validator 12 determines the homograph transformation between a generic point PJof the current input image IJand a corresponding point PJ-1of the previous input image IJ-1from and projections KP'j of the image keypoints KPJof the current input image IJinto the previous input image IJ-1such as

[0054] The validation of the pairs [KPJ-1,T, KPJ,T] is carried out on the basis of the validation criterion based on the comparison of the Euclidean distance in the previous input image IJ-1and a threshold distance DH2:

[0055] The input images of each group are then aligned and combined to form panoramic images. For this purpose (Figure 7), the parameter estimator 15 initially determines (block 200) a spanning tree of the set of input images I0, ...I,M, e.g. from a central image, and the distances between the root and the leaves. Next, the parameter estimator 15 determines an initial estimate of the parameters of the homograph transformations of each individual input image IJof the group with respect to the root and defines an initial position of the image projection plane PI (block 210). The initial estimation is performed by starting at the root of the newly calculated spanning tree and concatenating the relative transformations between pairs of adjacent input images along the path from the root to the current input image IJ(the relative transformations were previously calculated from the inertial data of each input image IJ). The initial position of the image projection plane PI can be defined arbitrarily according to design preferences.

[0056] Then, the parameter refiner 16 calculates a refinement of the distance de of the image projection plane PI (Figure 3) from the estimated image parameters using a global input image alignment method such as the bundle adjustment method, which allows for multivariable minimisation in order to obtain a three-dimensional reconstruction of the entire camera-plane system (block 220). In practice, the distance of the image projection plane is modified by methods, such as gradient descent, to minimise a figure of merit. According to the invention, the figure of merit is calculated on the basis of the approximation that the features represented by the image keypoints KPJlie substantially on the same image projection plane PI or at a short distance therefrom. In practice, the parameter refiner 16 searches for an optimising value dc* of the distance de of the image projection plane PI (Figure 3) from the shooting point that globally minimises the re-projection error of selected image keypoints in the connected adjacent pairs of images.

[0057] To include the constraint defined above, the optimising value dc* can be defined by the relation where fJand fJ-1are the focal distances of the input

[0058] (adjacent)images IJand IJ-1respectively, Xj and XJ-1are the three-dimensional projections of the image keypoints KPJ,Tand KPJ-1,Tof the input images IJand IJ-1.

[0059] The three-dimensional projection XJof the generic image point KPJis determined based on the intrinsic matrix KJof the input image IJand the rotation matrix Rj associated with the input image J, using the distance parameter de as follows and cxJ, cyJare the coordinates of the principal point of the input image IJ.

[0060] The summation of equation (11) is extended to all pairs of image keypoints KPJvalidated in all the input images I0,IMprovided to the parameter refiner 16 for global image parameter refinement.

[0061] Optimisation is carried out for distance de only. On the one hand, the optimisation of the distance de of the image projection plane PI allows the procedure to converge for images of planar objects even when, as in the case of the flank of a car, the image keypoints KPJare few in number and the images may contain artefacts, e.g. due to possible reflections. On the other hand, the computational burden is enormously reduced, to the extent that it is possible to process high-resolution images even using portable devices, such as smartphones, that have a relatively low processing capacity.

[0062] The search for the optimising solution can be conducted iteratively from the rotation estimated by the parameter estimator 15, e.g. by means of the Levenberg-Marquardt algorithm. Then, the pitch corrector 17 applies corrections to the images, in particular to compensate for distortions due to the unavoidable tilt ("pitch") of the camera 4 with respect to the vertical plane during the acquisition of the input images I0, •••,IMand the planar warper 18 projects the input images I0, •••,IMonto the image projection plane.

[0063] Compensation of the inclination with respect to the vertical plane (block 230) is performed using the angle of inclination provided by the inertial measurement unit 5. In practice, the input images I0, •••,IMare placed on a vertical plane parallel to the image projection plane PI, improving rendering.

[0064] The planar warper 18 then performs a planar projection of the input images I0, •••,IMonto the image projection plane PI placed at the optimising distance dc* previously determined in the connection step (block 240). For input images I0, •••,IMacquired on arcs not exceeding 150°, the approximation is acceptable and does not significantly affect the final result. In addition, the planar projection operation allows subsequent composition steps to be performed without the need to optimise the rotations associated with each input image I0, •••,IM, greatly reducing the computational burden.

[0065] The planar warper 18 thus generates projected images I0', •••,IM' which the image composer 19 combines after white balance, exposure compensation and suture feature search in regions of overlapping adjacent images (block 250). This leads to obtaining an output image IOUT.

[0066] It is finally clear that modifications and variants can be made to the disclosed method and device, without departing from the scope of the present innovation, as defined in the enclosed claims.

Claims

CLAIMS1. A method for composing images, comprising: acquiring (100) a sequence of input images (I0, ...,IM) along an acquisition path (D) using an image acquisition device (4) equipped with an inertial measurement unit (5), wherein the acquisition path (D) is defined by rotating the image acquisition device (4); associating (105) each input image (I0, ...,IM) with respective inertial data provided by the inertial measurement unit (5) and corresponding to respective acquisition conditions; connecting (110) adjacent input images (IJ-1, IJ) in pairs using homograph transformations defined on the basis of the inertial data associated with the input images (IJ-1, IJ); defining an image projection plane (PI); projecting (120) the input images (I0, ...,IM) onto the image projection plane (PI), obtaining projected images (I0', ...,IM'); and composing (125) an output image (IOUT) from the projected images (I0', ...,IM').

2. The method according to claim 1, wherein connecting comprises iteratively repeating: selecting a pair of adjacent input images (IJ, IJ-1) along the acquisition path (D) among the acquired inputimages (I0, ...,IM); identifying (150) image keypoints (KPJ, KPJ-1), the image keypoints (KPJ, KPJ-1) including first image keypoints (KPJ) and second image keypoints (KPJ-1) potentially corresponding to a first input image (IJ) and to a second input image (IJ- i) of the pair of adjacent input images (IJ, IJ-1), respectively; executing a validation procedure of the potentially corresponding first image keypoints (KPJ) and second image keypoints (KPJ-1).

3. The method according to claim 2, wherein executing the validation procedure comprises: defining (165) the respective homograph transformation between the first input image (IJ) and the second input image (IJ-1) on the basis of the inertial data of the first input image (IJ) and of the second input image (IJ-1); applying (170) the homograph transformation the second image keypoints (KPJ-1) identified in the second input image (IJ-1), obtaining projected image points (KP'J-1); comparing (175, 180) the projected image points (KP'J-1) with respective potentially corresponding first image keypoints (KPJ) of the first input image (IJ); and validating (185) the potential correspondences between pairs of first image keypoints (KPJ) and second image keypoints (KPJ-1) when the comparison meets a validationcriterion and not validating (190) the potential correspondences between pairs of first image keypoints (KPJ) and second image keypoints (KPJ-1) when the comparison does not meet the validation criterion.

4. The method according to claim 3, wherein executing the validation procedure comprises: defining (165) the respective homograph transformation between the second input image (IJ-1) and the first input image (IJ) on the basis of the inertial data of the first input image (IJ) and of the second input image (IJ-1); applying (170) the homograph transformation to the first image keypoints (KPJ) identified in the potentially corresponding first input image (IJ), obtaining further projected image points (KP'J); comparing (175, 180) the further projected image points (KP'j) with respective second image keypoints (KPJ-1) of the second input image (IJ-1); and validating (185) the potential correspondences between pairs of first image keypoints (KPJ) and second image keypoints (KPJ-1) when the comparison meets a validation criterion and not validating (190) the potential correspondences between pairs of first image keypoints (KPJ) and second image keypoints (KPJ-1) when the comparison does not meet the validation criterion.

5. The method according to claim 4, wherein comparing(175, 180) comprises measuring a distancebetween the projected image points (KP'J-1) and the respective first image keypoints (KPJ) of the potentially corresponding first input image (IJ) and wherein executing the validation procedure comprises validating (185) the potential correspondences if the distance is lower thana threshold distance (DTH) and not validating (190) the potential correspondences if the distanceis greater than the threshold distance (DTH).

6. The method according to any one of claims 2 to 5, comprising searching an optimizing value (dc*) of a distance(de) of the image projection plane (PI) form the image acquisition device (4) that globally minimizes a reprojection error of the image keypoints (KPJ, KPJ-1) selected in the pairs of connected adjacent input images (IJ,IJ-I).

7. The method according to claim 6, wherein the optimizing value dc* is given bywhere fj and fJ-1are focal distances corresponding to two adjacent input images IJand IJ-1of the input images (I0, ..., IM) and Xj and XJ-1are tridimensional projections of the image keypoints KPJ,Tand KPJ-1,Tof the input images IJandIJ-1.

8. The method according to claim 7, wherein thetridimensional projections (XJ) of points of the input image IJare determined by:whereis an intrinsic matrix of the input image IJ, Rj is a rotation matrix associated with the input image IJ, CxJ, cyJare coordinates of the principal point of the input image IJand9. The method according to any one of the preceding claims, comprising compensating for an inclination of the image acquisition device (4) with respect to a vertical plane in the input images (I0, ...,IM) on the basis of the associated inertial data.

10. The method according to any one of the preceding claims, wherein the acquisition path (D) is defined by rotating the image acquisition device (4) over an arc not greater than 150°.

11. A portable device, comprising: a processing unit (2); an image acquisition device (4) controlled by the processing unit (2); and an inertial measurement unit (5); wherein the processing unit (2) is configured to:cause acquisition of a sequence of input images(I0,IM) along an acquisition path (D) using the image acquisition device (4), wherein the acquisition path (D) is defined by rotating the image acquisition device (4); associate (105) each input image (I0, ...,IM) with respective inertial data provided by the inertial measurement unit (5) and corresponding to respective acquisition conditions; connect (110) adjacent input images (IJ-1, IJ) in pairs using homograph transformations defined on the basis of the inertial data associated with the input images (IJ-1, IJ); define an image projection plane (PI); project (120) the input images (I0, ...,IM) onto the image projection plane (PI), obtaining projected images (I0', ...,IM'); and compose (125) an output image (IOUT) from the projected images (I0', ...,IM').

12. Computer program comprising instructions that, when the program is executed by a computer, cause the computer to carry out the method according to any one of claims da 1 a 10.

Citation Information

Patent Citations

  • Method and system for image generation

    US20210004933A1