Image stitching method and device, electronic equipment and storage medium

Abstract

The application provides an image splicing method and device, electronic equipment and storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring at least two to-be-spliced images of a target scene; acquiring a target splicing distance parameter corresponding to the target scene; splicing the at least two to-be-spliced images based on the target splicing distance parameter to obtain a panoramic image; the target splicing distance parameter is obtained based on feature matching of at least one seam image and reference images corresponding to each seam image; and the seam image is a seam image in a reference panoramic image of the target scene. The application can determine the target splicing distance parameter corresponding to the target scene based on feature matching of at least one seam image and reference images corresponding to each seam image, and then automatically splice the to-be-spliced images of the target scene based on the target splicing distance parameter, so that the automatic correction of the splicing gap in the panoramic image is realized, and the quality of the panoramic image is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image stitching method, apparatus, electronic device, and storage medium. Background Technology

[0002] The integrated camera system combines bullet and PTZ cameras for panoramic stitching. The bullet camera, with its wide field of view, forms a global image of the monitored area as a reference. The PTZ camera is then controlled in a linked manner to output a local reference image, thereby optimizing the monitoring effect.

[0003] In related technologies, panoramic stitching cameras typically use multiple fixed-focus cameras to acquire images, and then stitch the images acquired by all the fixed-focus cameras together to obtain a panoramic image with a wider range. However, due to limitations in stitching algorithms, camera field of view, and environmental factors, the stitched panoramic image often exhibits stitching gaps. Objects appear discontinuously in the images at these gaps, which negatively impacts the overall quality of the panoramic image. Therefore, there is an urgent need to propose a new image stitching method to improve the quality of panoramic images. Summary of the Invention

[0004] This invention provides an image stitching method, apparatus, electronic device, and storage medium to solve the defect in the prior art where the imaging of objects is discontinuous at the stitching gaps, which to some extent affects the quality of panoramic images.

[0005] This invention provides an image stitching method, comprising the following steps.

[0006] Obtain at least two images to be stitched together for the target scene; Obtain the target stitching distance parameter corresponding to the target scene; the target stitching distance parameter is obtained by feature matching based on at least one stitching seam image and a reference image corresponding to each stitching seam image, wherein the stitching seam image is a stitching seam image in a reference panoramic image of the target scene; Based on the target stitching distance parameter, the at least two images to be stitched are stitched together to obtain a panoramic image.

[0007] According to an image stitching method provided by the present invention, the method further includes: For each of the stitched images, feature matching is performed between the stitched image and the corresponding reference image to obtain second and third feature points in the stitched image that match each first feature point in the corresponding reference image; Based on the positions of each of the second feature points and each of the third feature points, a direction adjustment parameter is determined, which is used to characterize the adjustment direction of the splicing distance parameter; Based on the positions of each of the second feature points and each of the third feature points, the target stitching distance adjustment parameters are determined. Based on the direction adjustment parameters and the target stitching distance adjustment parameters, the initial stitching distance parameters are adjusted to obtain reference stitching distance parameters; The gaps in the stitched image are corrected based on the reference stitching distance parameter to obtain a corrected stitched image; Based on the corrected stitch image, the reference stitching distance parameter is readjusted to obtain the target stitching distance parameter.

[0008] According to an image stitching method provided by the present invention, determining the orientation adjustment parameters based on the positions of each second feature point and each third feature point includes: For each of the first feature points, the initial direction adjustment parameters corresponding to the first feature point are determined based on the coordinates of the second feature point matched by the first feature point in the vertical direction of the splicing direction and the coordinates of the third feature point matched by the first feature point in the vertical direction of the splicing direction. Count the number of adjustments made with the same initial direction; The initial direction adjustment parameter corresponding to the maximum quantity is determined as the direction adjustment parameter.

[0009] According to an image stitching method provided by the present invention, determining the orientation adjustment parameters based on the positions of each second feature point and each third feature point includes: For each of the first feature points, based on the position of the second feature point corresponding to the first feature point and the position of the third feature point corresponding to the first feature point, the feature point distance and / or the position difference in the vertical direction of the splicing direction are determined; If the number of position differences greater than a preset value is greater than a first preset number, and / or the number of feature point distances greater than a preset distance is greater than a second preset number, the direction adjustment parameter is determined based on the position of each second feature point and the position of each third feature point.

[0010] According to an image stitching method provided by the present invention, the target stitching distance adjustment parameter includes a first stitching distance adjustment parameter and a second stitching distance adjustment parameter; The step of readjusting the reference stitching distance parameter based on the corrected stitching image includes: Based on the first change in the average value of the first pixel difference in the stitching direction in the corrected stitched image relative to the average value of the second pixel difference in the stitching direction in the uncorrected stitched image, a new first stitching distance adjustment parameter is determined. The new first stitching distance adjustment parameter is positively correlated with the first change. The first average pixel difference is determined based on the coordinates of the second feature point in the stitching direction and the coordinates of the third feature point in the stitching direction in the corrected stitched image. Based on the second change in the first average distance corresponding to the corrected stitched image relative to the second average distance corresponding to the uncorrected stitched image, a new second stitching distance adjustment parameter is determined. The new second stitching distance adjustment parameter is positively correlated with the second change. The first average distance is determined based on the position of the second feature point and the position of the third feature point in the corrected stitched image. Based on the new first stitching distance adjustment parameter and the new second stitching distance adjustment parameter, the reference stitching distance parameter is readjusted.

[0011] According to an image stitching method provided by the present invention, the step of stitching at least two images to be stitched together based on the target stitching distance parameter to obtain a panoramic image includes: The panoramic image is obtained by stitching together at least two images to be stitched based on the target stitching distance parameter using a panoramic stitching processor.

[0012] The present invention also provides an image stitching device, comprising: The first acquisition unit is used to acquire at least two images to be stitched together for the target scene; The second acquisition unit is used to acquire the target stitching distance parameter corresponding to the target scene; the target stitching distance parameter is obtained by feature matching based on at least one stitching image and a reference image corresponding to each stitching image, wherein the stitching image is a stitching image in a reference panoramic image of the target scene; A stitching unit is used to stitch together at least two images to be stitched based on the target stitching distance parameter to obtain a panoramic image.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image stitching method described above.

[0014] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image stitching method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the image stitching method as described above.

[0016] The image stitching method, apparatus, electronic device, and storage medium provided by this invention stitch at least two images of a target scene together based on a target stitching distance parameter to obtain a panoramic image. The target stitching distance parameter is obtained by feature matching between at least one stitching seam image and a corresponding reference image for each stitching seam image. The stitching seam image is a stitching seam image within a reference panoramic image of the target scene. This invention can pre-determine the target stitching distance parameter for the target scene based on feature matching between at least one stitching seam image and the corresponding reference image for each stitching seam image. Then, it automatically stitches the images of the target scene based on this target stitching distance parameter, enabling automatic correction of stitching gaps in the panoramic image and thus improving the quality of the panoramic image. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts illustrating the image stitching method provided in this embodiment of the invention.

[0019] Figure 2 This is the second flowchart illustrating the image stitching method provided in this embodiment of the invention.

[0020] Figure 3 This is the third flowchart of the image stitching method provided in the embodiments of the present invention.

[0021] Figure 4 This is one of the schematic diagrams of the reference image and stitched image provided in the embodiments of the present invention.

[0022] Figure 5 This is a second schematic diagram of the reference image and stitched image provided in the embodiments of the present invention.

[0023] Figure 6 This is a real-world image of the reference panoramic image provided in the embodiments of the present invention.

[0024] Figure 7 This is a schematic diagram of a reference panoramic image provided in an embodiment of the present invention.

[0025] Figure 8This is a schematic diagram of the image stitching system provided in an embodiment of the present invention.

[0026] Figure 9 This is a schematic diagram of the image stitching device provided in an embodiment of the present invention.

[0027] Figure 10 This is a schematic diagram of the physical structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] The following is combined Figures 1-7 The present invention describes an image stitching method. The entity executing this image stitching method can be a panoramic camera device, an electronic device such as a computer or server connected to the panoramic camera device, or an image stitching device installed in the electronic device. This image stitching device can be implemented through software, hardware, or a combination of both.

[0030] Figure 1 This is one of the flowcharts illustrating the image stitching method provided in this embodiment of the invention, such as... Figure 1 As shown, the image stitching method includes the following steps: Step 101: Obtain at least two images to be stitched together for the target scene.

[0031] The target scene can be any scene that needs to be filmed, such as a parking lot or a scenic spot.

[0032] For example, a panoramic camera device uses at least two fixed-focus cameras to capture the target scene and obtain at least two images to be stitched together. The number of fixed-focus cameras is usually 2, 4, 6 or 8, etc. Different fixed-focus cameras can acquire images with different field of view.

[0033] It should be noted that when the panoramic camera device uses at least two fixed-focus cameras to capture the target scene, it can cover the entire target scene or only a part of the target scene. The specific method of capture can be determined based on the needs, and this invention does not limit it.

[0034] Step 102: Obtain the target stitching distance parameter corresponding to the target scene; the target stitching distance parameter is obtained by feature matching based on at least one stitching image and a reference image corresponding to each stitching image, wherein the stitching image is a stitching image in a reference panoramic image of the target scene.

[0035] The reference image can be an image captured using a standard camera or a detail camera. Detail cameras typically use a lens to capture the reference image, also known as a detail image. The captured reference image should have sufficient resolution and sharpness for subsequent image processing. The following explanation uses the capture of a reference image using a detail camera as an example.

[0036] For example, a reference panoramic image of the target scene is pre-acquired, and each stitching seam in the reference panoramic image is detected to obtain a stitching seam image corresponding to each stitching seam. A detail camera is then controlled to rotate to the position of that stitching seam image and capture the image, obtaining a reference image corresponding to the stitching seam image. The reference image and the stitching seam image in the reference panoramic image of the target scene correspond one-to-one. Feature matching is performed between the stitching image and the corresponding reference image. Based on the feature matching result, the target stitching distance parameter corresponding to the target scene is determined, and the target stitching distance parameter is stored in relation to the target scene. When at least two images to be stitched for the target scene are acquired, the pre-stored target stitching distance parameter corresponding to the target scene is obtained.

[0037] Step 103: Stitch the at least two images to be stitched together based on the target stitching distance parameter to obtain a panoramic image.

[0038] Optionally, the panoramic image is obtained by stitching the at least two images to be stitched together using a panoramic stitching processor based on the target stitching distance parameter.

[0039] For example, when the target stitching distance parameter corresponding to the target scene is obtained, the Any View Stitching Processor (AVSP) stitches at least two images to be stitched based on the target stitching distance parameter. That is, the stitching gap of at least two images to be stitched is corrected by the target stitching distance parameter, and finally a panoramic image of the target scene is obtained.

[0040] It should be noted that other stitching algorithms in related technologies can also be used to stitch at least two images to be stitched together based on the target stitching distance parameter, which will not be elaborated here.

[0041] The image stitching method provided by this invention stitches at least two images of a target scene together based on a target stitching distance parameter to obtain a panoramic image. The target stitching distance parameter is obtained by feature matching between at least one stitching seam image and a corresponding reference image for each stitching seam image. The stitching seam image is the stitching seam image in the reference panoramic image of the target scene. This invention can pre-determine the target stitching distance parameter corresponding to the target scene based on feature matching between at least one stitching seam image and the corresponding reference image for each stitching seam image. Then, it automatically stitches the images of the target scene based on this target stitching distance parameter, enabling automatic correction of stitching gaps in the panoramic image, thereby improving the quality of the panoramic image. Furthermore, the entire stitching process requires no manual intervention and can be completed automatically. In addition, each scene corresponds to its own stitching distance parameter, allowing the image stitching method of this invention to adapt to the image stitching needs of different scenes, exhibiting strong versatility and flexibility.

[0042] In one embodiment, Figure 2 This is a second schematic flowchart of the image stitching method provided in this embodiment of the invention, as shown below. Figure 2 As shown, prior to step 101 above, the image stitching method further includes the following steps: Step 201: For each of the stitched images, perform feature matching on the stitched image and the corresponding reference image to obtain the second feature point and the third feature point in the stitched image that match each of the first feature points in the corresponding reference image.

[0043] For example, for each stitched image, scale-invariant feature transform (SIFT) is performed on the stitched image and the corresponding reference image to detect key points and select key points. Then, feature point matching is performed to find the imaging points of each object in the reference image in the corresponding stitched image, forming matching feature point pairs. This yields a set of feature points that match each first feature point representing the object in the reference image. The feature points in the feature point set are then refined to obtain second and third feature points that match each first feature point.

[0044] The specific process of refining feature points in a feature point set is as follows: In practical applications, the same object (such as a building) may have a large number of similar matching points, and it is necessary to avoid similar matching points interfering with the final matching result. Therefore, during the feature point matching process, a K-Dimensional tree (KD tree) can be used to match each feature point to its k1 nearest neighbors. This method can significantly improve the matching speed and accuracy. After the feature point matching is completed, a large number of feature point pairs will be obtained, and there may be a certain number of incorrect matches. Therefore, it is necessary to remove the incorrect matching feature point pairs according to preset rules, that is, to refine the feature point set matched by the first feature point.

[0045] Analysis revealed that if the stitched image contains seams, the imaging point of an object in the reference image will have two feature point pairs in the stitched image. Furthermore, clustering algorithms confirmed the existence of two sets of feature point pairs. Figure 3 This is one of the schematic diagrams of the reference image and the stitched image provided in the embodiments of the present invention. Figure 4 This is a second schematic diagram of the reference image and stitched image provided in the embodiments of the present invention, such as... Figure 3 and Figure 4 As shown, a feature point in the reference image has two feature point pairs in the stitched image, located on the right side of the left image and the left side of the right image, respectively, around the stitched seam. Therefore, during the feature point pair purification and feature matching stages, it is necessary to ensure that the first feature point in the reference image has two corresponding feature points in the stitched image, namely the second and third feature points. The second and third feature points are located on both sides of the stitched seam in the stitched image, and there should not be multiple similar interference points, otherwise it will affect the final stitching result. Therefore, the purpose of purifying the feature point set matched by the first feature point is to ensure that the final result has the second and third feature points that match the first feature point, avoiding the influence of multiple similar interference points on the final stitching result, which can greatly reduce the impact of symmetrical and repetitive features on stitching recognition.

[0046] It should be noted that Scale Invariant Feature Transform (SIFT) extracts local features of an image, remaining invariant to rotation, scaling, and brightness changes, and also maintaining a certain degree of stability against changes in field of view, radial transformation, and noise.

[0047] It should be noted that the Random Sample Consensus (RANSAC) method can be used to purify the feature point set. This method first randomly selects a set of matching points to estimate the model parameters, then evaluates the degree of agreement between all matching points and the estimated model, and finally selects the model corresponding to the points with the highest degree of agreement.

[0048] Step 202: Based on the positions of each of the second feature points and each of the third feature points, determine the direction adjustment parameters, which are used to characterize the adjustment direction of the splicing distance parameters.

[0049] For example, taking the horizontal direction as the stitching direction, for each first feature point, the vertical positions of the second feature point matching the first feature point and the third feature point matching the first feature point are compared. Assuming the second feature point is on the left side of the stitching seam and the third feature point is on the right side of the stitching seam, if the vertical position of the second feature point is greater than that of the third feature point, it indicates that the stitching distance parameter is too large and needs to be reduced. In this case, the initial direction adjustment parameter can be set to 1, where 1 represents the need to reduce the stitching distance parameter. If the vertical position of the second feature point is less than that of the third feature point, it indicates that the stitching distance parameter is too small and needs to be increased. In this case, the initial direction adjustment parameter can be set to -1, where -1 represents the need to increase the stitching distance parameter. Finally, the final direction adjustment parameter is determined based on the total number of 1s and -1s counted.

[0050] Step 203: Determine the target stitching distance adjustment parameters based on the positions of each of the second feature points and each of the third feature points.

[0051] For example, for each first feature point, the distance between the second and third feature points is determined based on the positions of the second and third feature points corresponding to the first feature point. Then, the target stitching distance adjustment parameter is determined based on the distances between the second and third feature points corresponding to each of the first feature points.

[0052] Step 204: Based on the direction adjustment parameter and the target stitching distance adjustment parameter, adjust the initial stitching distance parameter to obtain the reference stitching distance parameter.

[0053] For example, once the orientation adjustment parameter and the target stitching distance adjustment parameter are obtained, the product of the orientation adjustment parameter and the target stitching distance adjustment parameter can be calculated, and the sum of this product and the initial stitching distance parameter can be determined as the reference stitching distance parameter.

[0054] Step 205: Correct the gaps in the stitched image based on the reference stitching distance parameter to obtain the corrected stitched image.

[0055] For example, when the reference stitching distance parameter is obtained, the stitching algorithm corrects the gaps in the stitched image based on the reference stitching distance parameter to obtain the corrected stitched image. Following the same method, the corrected stitched image corresponding to each stitched image can be obtained.

[0056] Step 206: Based on the corrected stitching image, readjust the reference stitching distance parameter to obtain the target stitching distance parameter.

[0057] For example, after obtaining the corrected stitching image, it is determined whether there are still stitching gaps in the corrected stitching image. If there are still stitching gaps in the corrected stitching image, the positions of the second feature point and the third feature point that match the first feature point are re-determined based on the corrected stitching image. Then, based on the re-determined positions of the second feature point and the third feature point that match the first feature point, new orientation adjustment parameters and new target stitching distance adjustment parameters are determined. Then, based on the new orientation adjustment parameters and the new target stitching distance adjustment parameters, the reference stitching distance parameter is readjusted until there are no gaps in the corrected stitching image. Finally, the reference stitching distance parameter obtained by the adjustment is determined as the target stitching distance parameter.

[0058] It should be noted that when the panoramic camera device is started, a secondary channel (pipe) can be created in advance. The adjustment process of the reference stitching distance parameter is carried out in this secondary channel without the user's awareness, until the final target stitching distance parameter is obtained. The target stitching distance parameter is applied to the image stitching of the target scene, and the detected stitching seams are corrected based on the target stitching distance parameter. The correction process is also carried out in this secondary channel. After the correction is completed, the resulting panoramic image is displayed to improve the user experience.

[0059] In this embodiment, the gaps in the stitched image can be corrected based on a determined reference stitching distance parameter. Based on the corrected stitched image, the reference stitching distance parameter is readjusted until there are no gaps in the corrected stitched image. The finally adjusted reference stitching distance parameter is determined as the target stitching distance parameter, thereby realizing the automatic determination of the target stitching distance parameter of the target scene. Since the standard for determining the target stitching distance parameter is that there are no gaps in the corrected stitched image, the accuracy of the target stitching distance parameter determination can be improved, which in turn can improve the accuracy of image stitching.

[0060] In one embodiment, step 201 above determines the direction adjustment parameters based on the positions of each of the second feature points and each of the third feature points, which can be implemented in the following way: For each of the first feature points, based on the positions of the second feature points corresponding to the first feature points and the third feature points corresponding to the first feature points, the distance between the feature points and / or the positional difference in the vertical direction of the splicing direction are determined; if the number of positional differences greater than a preset value is greater than a first preset number, and / or the number of feature point distances greater than a preset distance is greater than a second preset number, the direction adjustment parameter is determined based on the positions of each of the second feature points and the positions of each of the third feature points.

[0061] Among them, the preset value and preset distance are actual measured empirical values, and the preset value and preset distance are different for different resolutions.

[0062] For example, for each first feature point, the coordinates of the corresponding second feature point are represented as follows: Ai ( xi , yi The coordinates of the third feature point corresponding to the first feature point are represented as follows: Bi ( xi ′, yi If the feature point distance is ′), then the feature point distance is ′). Taking the horizontal splicing direction as an example, the positional difference in the vertical direction of the splicing direction is... Then, using the same calculation method, the feature point distances for all first feature points can be obtained. and position difference .

[0063] When only the position difference is determined At that time, from all position differences The number of position differences greater than a preset value is determined, and the number of position differences greater than the preset value is greater than a first preset number. If the stitching gaps in the stitched images corresponding to the second and third feature points are considered severe and require correction, then the orientation adjustment parameters need to be determined based on the positions of each second and third feature point. The number of position differences greater than a preset value is less than or equal to a first preset number. If the seam gap is not serious or there is no seam gap, then no correction is needed.

[0064] When only the distance of feature points is determined At that time, the distance from all feature points The number of feature point distances greater than a preset distance is determined, and the number of feature point distances greater than the preset distance is greater than a second preset number. If the stitching gaps in the stitched images corresponding to the second and third feature points are considered severe and require correction, then the orientation adjustment parameters need to be determined based on the positions of each second and third feature point. The number of feature points with distances greater than a preset distance is less than or equal to a second preset number. If the seam gap is not serious or there is no seam gap, then no correction is needed.

[0065] Simultaneously determine the position difference Distance to feature points At that time, from all position differences The number of positional differences greater than a preset value is determined, and the distance from all feature points is calculated. The number of feature point distances greater than a preset distance is determined, and the number of position differences greater than a preset value is greater than a first preset number. Furthermore, the number of feature point distances greater than a preset distance is greater than a second preset number. If the stitching gaps in the stitched images corresponding to the second and third feature points are considered severe and require correction, then the orientation adjustment parameters need to be determined based on the positions of each second and third feature point. The number of position differences greater than a preset value is less than or equal to a first preset number. The number of feature points with a distance greater than a preset distance is less than or equal to a second preset number. If the seam gap is not serious or there is no seam gap, then no correction is needed.

[0066] In this embodiment, the severity of seam gaps in the stitched image can be judged based on the distance and / or position difference of feature points. For stitched images with severe seam gaps, the orientation adjustment parameters are determined based on the positions of each second feature point and each third feature point. There is no need to process stitched images with mild seam gaps, thereby saving computational resources.

[0067] In one embodiment, step 201 above determines the direction adjustment parameters based on the positions of each of the second feature points and each of the third feature points, which can be implemented in the following way: For each of the first feature points, based on the coordinates of the second feature point matched by the first feature point in the vertical direction of the splicing direction and the coordinates of the third feature point matched by the first feature point in the vertical direction of the splicing direction, the initial direction adjustment parameter corresponding to the first feature point is determined; the number of identical initial direction adjustment parameters is counted; and the initial direction adjustment parameter corresponding to the largest number is determined as the direction adjustment parameter.

[0068] For example, for the same matching point Ai ( xi , yi )and Bi ( xi ′, yi Taking the horizontal splicing direction as an example, let's assume... Ai ( xi , yi On the left side of the seam in the stitched image, Bi ( xi ′, yi ′) On the right side of the seam of the stitched image, if yi Greater than or equal to yi The '' indicates that the splicing distance parameter is too large and needs to be reduced; if yi Less than yiThe result indicates that the splicing distance parameter is too small and needs to be increased. Let the direction adjustment parameter be di, then di satisfies the following formula (3): (3) Count the total number of 1s and the total number of -1s. Compare the total number of 1s and the total number of -1s, and determine the direction adjustment parameter corresponding to the maximum total number as the final direction adjustment parameter.

[0069] In this embodiment, the initial direction adjustment parameter corresponding to the first feature point can be determined based on the coordinates of the second feature point matched by the first feature point in the vertical direction of the splicing direction and the coordinates of the third feature point matched by the first feature point in the vertical direction of the splicing direction. Then, the final direction adjustment parameter can be determined based on the initial direction adjustment parameters corresponding to all the first feature points, thereby improving the accuracy of the direction adjustment parameter determination.

[0070] In one embodiment, the target stitching distance adjustment parameters include a first stitching distance adjustment parameter and a second stitching distance adjustment parameter. Figure 5 This is the third flowchart illustrating the image stitching method provided in this embodiment of the invention, as shown below. Figure 5 As shown, step 205 above, based on the corrected seam image, readjusts the reference stitching distance parameter, which can be achieved through the following steps: Step 2051: Based on the first change in the average value of the first pixel difference in the stitching direction in the corrected stitched image relative to the average value of the second pixel difference in the stitching direction in the uncorrected stitched image, determine a new first stitching distance adjustment parameter. The new first stitching distance adjustment parameter is positively correlated with the first change. The average value of the first pixel difference is determined based on the coordinates of the second feature point in the stitching direction and the coordinates of the third feature point in the stitching direction in the corrected stitched image.

[0071] For example, given the corrected stitched image and the original stitched image, we can obtain the second and third feature points in the corrected stitched image that match the first feature point in the reference image, and we can also obtain the second and third feature points in the original stitched image that match the first feature point in the reference image. For any stitched image, the coordinates of the second feature point corresponding to the first feature point are represented as follows: Ai ( xi , yi The coordinates of the third feature point corresponding to the first feature point are represented as follows: Bi ( xi ′, yi The second and third feature points can also be called the same matching points of the first feature point. The average pixel difference of all the same matching points in the stitching direction is calculated. Taking the horizontal direction of splicing as an example, the average pixel difference Specifically, it can be expressed using the following formula (1): (1) in, This indicates the number of identical matching points.

[0072] According to the above formula (1), the average value of the first pixel difference in the stitching direction in the corrected stitched image and the average value of the second pixel difference in the stitching direction in the original stitched image can be obtained. When the first change of the average value of the first pixel difference relative to the average value of the second pixel difference is positive, it indicates that... If the value increases, the first stitching distance adjustment parameter will be used. It also increases; when the first change in the average value of the first pixel difference relative to the average value of the second pixel difference is negative, it indicates... Get smaller and Positive correlation, that is , This indicates a positive correlation, used to adjust the first stitching distance parameter. The adjusted first stitching distance adjustment parameter will be used as the new first stitching distance adjustment parameter.

[0073] Step 2052: Based on the second change in the first average distance corresponding to the corrected stitched image relative to the second average distance corresponding to the uncorrected stitched image, determine a new second stitching distance adjustment parameter. The new second stitching distance adjustment parameter is positively correlated with the second change. The first average distance is determined based on the position of the second feature point and the position of the third feature point in the corrected stitched image.

[0074] For example, it is also necessary to calculate the average distance of all identical matching points. Satisfying Euclidean distance, average distance Specifically, it can be expressed using the following formula (2): (2) According to the above formula (2), the first average distance corresponding to the corrected stitch image and the second average distance corresponding to the uncorrected stitch image can be obtained. When the second change of the first average distance relative to the second average distance is a positive value, it indicates that... If it increases, the second splicing distance adjustment parameter will be used. It also increases; when the second change in the first average distance relative to the second average distance is negative, it indicates... Get smaller and Positive correlation, that is d mean To achieve the second splicing distance adjustment parameter The adjusted second splicing distance adjustment parameter will be used as the new second splicing distance adjustment parameter.

[0075] Step 2053: Based on the new first splicing distance adjustment parameter and the new second splicing distance adjustment parameter, readjust the reference splicing distance parameter.

[0076] For example, the reference splicing distance parameter can be readjusted based on the following formula (4), which can also be a model of the relationship between the seam and the splicing distance parameter: (4) in, D cur This indicates the current stitching distance parameter. D next This indicates the adjusted splicing distance parameter. d Indicates the direction adjustment parameters. 1 indicates the new second stitching distance adjustment parameter. This indicates the new first stitching distance adjustment parameter.

[0077] In this embodiment, a new first stitching distance adjustment parameter is determined based on the first change in the average value of the first pixel difference in the stitching direction in the corrected stitched image relative to the average value of the second pixel difference in the stitching direction in the uncorrected stitched image. A new second stitching distance adjustment parameter is then determined based on the second change in the first average distance corresponding to the corrected stitched image relative to the second average distance corresponding to the uncorrected stitched image. Furthermore, based on the new first and second stitching distance adjustment parameters, the reference stitching distance parameter is continuously readjusted until the target stitching distance parameter is obtained. This achieves automatic determination of the target stitching distance parameter for the target scene. Since the criterion for determining the target stitching distance parameter is the absence of gaps in the corrected stitched image, the accuracy of the target stitching distance parameter determination is improved, further enhancing the accuracy of image stitching.

[0078] In one embodiment, prior to step 101 described above, the image stitching method further includes the following steps: For each detection block in each detection region of the reference panoramic image, the horizontal field of view center of the detail camera is controlled to rotate to the center line position of the reference panoramic image, and the vertical field of view center of the detail camera is controlled to rotate to the detection block. The image in the detection block includes the stitching image, and the detection block is a block that divides the detection region in the vertical direction. The detail camera is controlled to shoot based on the target magnification to obtain the reference image corresponding to the detection block.

[0079] For example, Figure 6 This is a real-world image of the reference panoramic image provided in the embodiments of the present invention, such as... Figure 6 As shown, there are stitching gaps in the reference panoramic image, and the left and right images are not at the same height, resulting in discontinuity in the image at the stitching gaps. Figure 7 This is a schematic diagram of a reference panoramic image provided in an embodiment of the present invention, such as... Figure 7 As shown, assuming the panoramic stitching channel is m (where 2≤m≤8), then the number of stitching seams is m-1. The size of the resulting reference panoramic image after stitching is W×H, then the x-coordinate of the centerline of each stitching seam can be set as ( , , …, If the width of the splicing gap is set to w, then the range of the horizontal coordinate of the first detection area is ( , Similarly, the horizontal coordinate range of other detection areas is obtained, and the height of each detection area is H. For each detection area, the detection area is divided into equal parts in the vertical direction. k Block, obtain the detection area corresponding to k For each detection block, the horizontal field of view center of the detail camera is rotated to the center line position of the reference panoramic image (the center line position of the entire reference panoramic image). The vertical field of view center of the detail camera is then rotated to that detection block, and the detail camera is controlled to capture an image at the target magnification, obtaining a reference image corresponding to the detection block. Feature extraction and feature matching are performed on the reference image and the seam images included in the detection block. After the first detection block is acquired and its reference image and seam images are feature-matched, the vertical field of view center of the detail camera is rotated to the second detection block, and the detail camera is controlled to capture an image at the target magnification, obtaining a reference image corresponding to the second detection block. This process is repeated until all detection blocks in all detection areas have completed reference image acquisition and feature matching, ultimately obtaining the second and third feature points matched for all first feature points.

[0080] It should be noted that when obtaining all the second and third feature points that match the first feature point, deduplication can be performed to remove the correspondences where the first, second, and third feature points are all the same, in order to reduce the amount of computation.

[0081] It should be noted that when the field of view of a detail camera and a panoramic camera are the same, the detection area does not necessarily need to be divided equally in the vertical direction. k The detection area is divided into blocks. The vertical field of view of the detail camera is rotated to the detection area for image acquisition. Feature extraction and matching are then performed on the stitching seams within the detection area and the reference image acquired by the detail camera within the detection area. When the field of view of the detail camera is smaller than that of the panoramic camera, to ensure that the detail camera can capture images of all stitching seams in the reference panoramic image, the detection area needs to be divided vertically into equal parts. k The camera controls the vertical field of view of the detailed camera to rotate to a detection block for image acquisition each time.

[0082] In this embodiment, all detection areas, including the stitching seams, are determined from the reference panoramic image, and each detection area is divided into multiple detection blocks in the vertical direction. The detail camera is controlled to rotate to each detection block to acquire the reference image, which facilitates feature matching between the reference image and the corresponding stitching seam image, thereby determining the target stitching distance parameter for image stitching.

[0083] In one embodiment, prior to step 101 described above, the image stitching method further includes the following steps: Based on the correspondence between the panoramic channel focal length of the panoramic camera and the detail channel focal length of the detail camera, the target detail channel focal length corresponding to the current panoramic channel focal length of the panoramic camera is determined; based on the correspondence between the detail channel focal length and the motor position, the target motor position corresponding to the target detail channel focal length is determined; based on the correspondence between the motor position and the magnification, the target magnification corresponding to the target motor position is determined.

[0084] For example, the integrated camera system includes a panoramic camera and a detail camera. The panoramic camera is used for panoramic observation, while the detail camera is used for tracking and capturing images. To meet the requirements of this business, the panoramic camera and the detail camera are pre-calibrated to establish a correspondence between the panoramic channel focal length of the panoramic camera and the detail channel focal length of the detail camera. After calibration, the detail camera can be controlled to rotate to any specified coordinate position in the reference panoramic image to see the same scene. Each detection area of ​​the reference panoramic image is vertically divided into... k This block allows control of the vertical movement of the pan-tilt head of the detailed camera device.k The focal length of the panoramic channel of the panoramic camera is known to be... The focal lengths of detailed camera devices, from wide-angle to telephoto, can be expressed as: Where z represents the position of the motor (e.g., the Zoom motor), This indicates the number of motor positions, and there is a corresponding mapping relationship between motor positions and magnification. After dividing the panoramic image into blocks for each detection region, the equivalent focal length of each detection block can be obtained. Using the equivalent focal length as the current panoramic channel focal length of the panoramic camera, and based on the correspondence between the panoramic channel focal length of the panoramic camera and the detail channel focal length of the detail camera, the target detail channel focal length corresponding to the current panoramic channel focal length of the panoramic camera can be determined. The correspondence between the focal length of the mid-detail channel and the motor position can determine the target motor position corresponding to the focal length of the target detail channel. Furthermore, based on the correspondence between the motor position and the magnification, the target magnification X corresponding to the target motor position can be determined.

[0085] It should be noted that there may be some errors in the calibration process of the panoramic and detail camera equipment. This could lead to a significant deviation between the image obtained by the detail camera equipment after zooming to the target magnification X and the corresponding image from the panoramic camera equipment, making subsequent detection impossible. Therefore, the equivalent focal length of each detection block can be determined using a coefficient. Magnify to obtain Then, the target magnification corresponding to the target motor position is queried using the same method described above, in order to reduce the deviation between the image obtained by the detail camera device after zooming to the target magnification X and the corresponding image of the panoramic camera device.

[0086] In this embodiment, based on the correspondence between the panoramic channel focal length of the panoramic camera and the detail channel focal length of the detail camera, the target detail channel focal length corresponding to the current panoramic channel focal length of the panoramic camera is determined. Based on the correspondence between the detail channel focal length and the motor position, the target motor position corresponding to the target detail channel focal length is determined. Based on the correspondence between the motor position and the magnification, the target magnification corresponding to the target motor position is determined. This achieves automatic determination of the target magnification, enabling the detail camera to obtain an image identical to the corresponding image from the panoramic camera after zooming to the target magnification.

[0087] Figure 8 This is a schematic diagram of the image stitching system provided in an embodiment of the present invention, as shown below. Figure 8As shown, the image stitching system includes an image acquisition module, a gap detection module, a gap correction module, and a result output module. The image acquisition module acquires stitching seam images from a reference panoramic image of the target scene using a panoramic camera, and acquires a reference image corresponding to the stitching seam image using a detail camera. The gap detection module performs feature matching between the stitching seam image and the reference image for each detection block in each detection region of the reference panoramic image, obtaining second and third feature points corresponding to each first feature point. The gap correction module determines orientation adjustment parameters and target stitching distance adjustment parameters based on the positions of the second and third feature points corresponding to each first feature point, and finally obtains the target stitching distance parameter for image stitching. The result output module stitches at least two images to be stitched based on the target stitching distance parameter to obtain and output the panoramic image.

[0088] The image stitching system provided in this invention can automatically detect seams from the image perspective and guide the adjustment of the stitching distance parameter in the stitching algorithm to achieve precise correction of the seams, thereby generating high-quality panoramic images without stitching gaps.

[0089] The image stitching device provided by the present invention will be described below. The image stitching device described below can be referred to in correspondence with the image stitching method described above.

[0090] Figure 9 This is a schematic diagram of the image stitching device provided in an embodiment of the present invention, as shown below. Figure 9 As shown, the image stitching device 900 includes a first acquisition unit 901, a second acquisition unit 902, and a stitching unit 903; wherein: The first acquisition unit 901 is used to acquire at least two images to be stitched together for the target scene; The second acquisition unit 902 is used to acquire the target stitching distance parameter corresponding to the target scene; the target stitching distance parameter is obtained by feature matching based on at least one stitching image and a reference image corresponding to each stitching image, wherein the stitching image is a stitching image in a reference panoramic image of the target scene; The stitching unit 903 is used to stitch together the at least two images to be stitched based on the target stitching distance parameter to obtain a panoramic image.

[0091] The image stitching device provided by this invention stitches at least two images of a target scene together based on a target stitching distance parameter to obtain a panoramic image. The target stitching distance parameter is obtained by feature matching between at least one stitching seam image and a corresponding reference image for each stitching seam image. The stitching seam image is a stitching seam image within a reference panoramic image of the target scene. This invention can pre-determine the target stitching distance parameter for the target scene based on feature matching between at least one stitching seam image and the corresponding reference image for each stitching seam image. Then, it automatically stitches the images of the target scene based on this target stitching distance parameter, enabling automatic correction of stitching gaps in the panoramic image and thus improving the quality of the panoramic image.

[0092] Based on any of the above embodiments, the image stitching device 900 further includes: The matching unit is used to perform feature matching on each of the stitched images and the corresponding reference image to obtain second feature points and third feature points in the stitched image that match each first feature point in the corresponding reference image; The first determining unit is used to determine the direction adjustment parameter based on the position of each of the second feature points and the position of each of the third feature points, wherein the direction adjustment parameter is used to characterize the adjustment direction of the splicing distance parameter; The second determining unit is used to determine the target stitching distance adjustment parameter based on the position of each of the second feature points and the position of each of the third feature points; The first adjustment unit is used to adjust the initial stitching distance parameter based on the direction adjustment parameter and the target stitching distance adjustment parameter to obtain the reference stitching distance parameter; The correction unit is used to correct the gaps in the stitched image based on the reference stitching distance parameter to obtain the corrected stitched image. The second adjustment unit is used to readjust the reference stitching distance parameter based on the corrected stitching image to obtain the target stitching distance parameter.

[0093] Based on any of the above embodiments, the first determining unit is further specifically used for: For each of the first feature points, the initial direction adjustment parameters corresponding to the first feature point are determined based on the coordinates of the second feature point matched by the first feature point in the vertical direction of the splicing direction and the coordinates of the third feature point matched by the first feature point in the vertical direction of the splicing direction. Count the number of adjustments made with the same initial direction; The initial direction adjustment parameter corresponding to the maximum quantity is determined as the direction adjustment parameter.

[0094] Based on any of the above embodiments, the first determining unit is specifically used for: For each of the first feature points, based on the position of the second feature point corresponding to the first feature point and the position of the third feature point corresponding to the first feature point, the feature point distance and / or the position difference in the vertical direction of the splicing direction are determined; If the number of position differences greater than a preset value is greater than a first preset number, and / or the number of feature point distances greater than a preset distance is greater than a second preset number, the direction adjustment parameter is determined based on the position of each second feature point and the position of each third feature point.

[0095] Based on any of the above embodiments, the target stitching distance adjustment parameter includes a first stitching distance adjustment parameter and a second stitching distance adjustment parameter; the second adjustment unit is specifically used for: Based on the first change in the average value of the first pixel difference in the stitching direction in the corrected stitched image relative to the average value of the second pixel difference in the stitching direction in the uncorrected stitched image, a new first stitching distance adjustment parameter is determined. The new first stitching distance adjustment parameter is positively correlated with the first change. The first average pixel difference is determined based on the coordinates of the second feature point in the stitching direction and the coordinates of the third feature point in the stitching direction in the corrected stitched image. Based on the second change in the first average distance corresponding to the corrected stitched image relative to the second average distance corresponding to the uncorrected stitched image, a new second stitching distance adjustment parameter is determined. The new second stitching distance adjustment parameter is positively correlated with the second change. The first average distance is determined based on the position of the second feature point and the position of the third feature point in the corrected stitched image. Based on the new first stitching distance adjustment parameter and the new second stitching distance adjustment parameter, the reference stitching distance parameter is readjusted.

[0096] Based on any of the above embodiments, the splicing unit 903 is specifically used for: Based on the target stitching distance parameter, the at least two images to be stitched are stitched together to obtain a panoramic image.

[0097] Figure 10 This is a schematic diagram of the physical structure of the electronic device provided in the embodiments of the present invention, such as... Figure 10As shown, the electronic device may include a processor 1010, a communications interface 1020, a memory 1030, and a communication bus 1040, wherein the processor 1010, the communications interface 1020, and the memory 1030 communicate with each other via the communication bus 1040. The processor 1010 can call logical instructions in the memory 1030 to execute an image stitching method, which includes: acquiring at least two images to be stitched for a target scene; acquiring a target stitching distance parameter corresponding to the target scene; the target stitching distance parameter is obtained by feature matching based on at least one stitching seam image and a reference image corresponding to each stitching seam image, wherein the stitching seam image is a stitching seam image in a reference panoramic image of the target scene; and stitching the at least two images to be stitched based on the target stitching distance parameter to obtain a panoramic image.

[0098] Furthermore, the logical instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the image stitching method provided by the above methods. The method includes: acquiring at least two images to be stitched for a target scene; acquiring a target stitching distance parameter corresponding to the target scene; the target stitching distance parameter is obtained by feature matching based on at least one stitching seam image and a reference image corresponding to each stitching seam image, wherein the stitching seam image is a stitching seam image in a reference panoramic image of the target scene; and stitching the at least two images to be stitched based on the target stitching distance parameter to obtain a panoramic image.

[0100] In another aspect, the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an image stitching method provided by the above methods. The method includes: acquiring at least two images to be stitched for a target scene; acquiring a target stitching distance parameter corresponding to the target scene; the target stitching distance parameter being obtained by feature matching based on at least one stitching seam image and a reference image corresponding to each stitching seam image, wherein the stitching seam image is a stitching seam image in a reference panoramic image of the target scene; and stitching the at least two images to be stitched based on the target stitching distance parameter to obtain a panoramic image.

[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image stitching method, characterized in that, include: Obtain at least two images to be stitched together for the target scene; Obtain the target stitching distance parameter corresponding to the target scene; The target stitching distance parameter is obtained by feature matching based on at least one stitching seam image and a reference image corresponding to each stitching seam image, wherein the stitching seam image is a stitching seam image in a reference panoramic image of the target scene; Based on the target stitching distance parameter, the at least two images to be stitched are stitched together to obtain a panoramic image.

2. The image stitching method according to claim 1, characterized in that, The method further includes: For each of the stitched images, feature matching is performed between the stitched image and the corresponding reference image to obtain second and third feature points in the stitched image that match each first feature point in the corresponding reference image; Based on the positions of each of the second feature points and each of the third feature points, a direction adjustment parameter is determined, which is used to characterize the adjustment direction of the splicing distance parameter; Based on the positions of each of the second feature points and each of the third feature points, the target stitching distance adjustment parameters are determined. Based on the direction adjustment parameters and the target stitching distance adjustment parameters, the initial stitching distance parameters are adjusted to obtain reference stitching distance parameters; The gaps in the stitched image are corrected based on the reference stitching distance parameter to obtain a corrected stitched image; Based on the corrected stitch image, the reference stitching distance parameter is readjusted to obtain the target stitching distance parameter.

3. The image stitching method according to claim 2, characterized in that, The determination of direction adjustment parameters based on the positions of each of the second feature points and each of the third feature points includes: For each of the first feature points, the initial direction adjustment parameters corresponding to the first feature point are determined based on the coordinates of the second feature point matched by the first feature point in the vertical direction of the splicing direction and the coordinates of the third feature point matched by the first feature point in the vertical direction of the splicing direction. Count the number of adjustments made with the same initial direction; The initial direction adjustment parameter corresponding to the maximum quantity is determined as the direction adjustment parameter.

4. The image stitching method according to claim 2, characterized in that, The determination of direction adjustment parameters based on the positions of each of the second feature points and each of the third feature points includes: For each of the first feature points, based on the position of the second feature point corresponding to the first feature point and the position of the third feature point corresponding to the first feature point, the feature point distance and / or the position difference in the vertical direction of the splicing direction are determined; If the number of position differences greater than a preset value is greater than a first preset number, and / or the number of feature point distances greater than a preset distance is greater than a second preset number, the direction adjustment parameter is determined based on the position of each second feature point and the position of each third feature point.

5. The image stitching method according to claim 2, characterized in that, The target stitching distance adjustment parameters include a first stitching distance adjustment parameter and a second stitching distance adjustment parameter; The step of readjusting the reference stitching distance parameter based on the corrected stitching image includes: Based on the first change in the average value of the first pixel difference in the stitching direction in the corrected stitched image relative to the average value of the second pixel difference in the stitching direction in the uncorrected stitched image, a new first stitching distance adjustment parameter is determined. The new first stitching distance adjustment parameter is positively correlated with the first change. The first average pixel difference is determined based on the coordinates of the second feature point in the stitching direction and the coordinates of the third feature point in the stitching direction in the corrected stitched image. Based on the second change in the first average distance corresponding to the corrected stitched image relative to the second average distance corresponding to the uncorrected stitched image, a new second stitching distance adjustment parameter is determined. The new second stitching distance adjustment parameter is positively correlated with the second change. The first average distance is determined based on the position of the second feature point and the position of the third feature point in the corrected stitched image. Based on the new first stitching distance adjustment parameter and the new second stitching distance adjustment parameter, the reference stitching distance parameter is readjusted.

6. The image stitching method according to any one of claims 1-5, characterized in that, The step of stitching together at least two images to be stitched based on the target stitching distance parameter to obtain a panoramic image includes: The panoramic image is obtained by stitching together at least two images to be stitched based on the target stitching distance parameter using a panoramic stitching processor.

7. An image stitching device, characterized in that, include: The first acquisition unit is used to acquire at least two images to be stitched together for the target scene; The second acquisition unit is used to acquire the target stitching distance parameter corresponding to the target scene; The target stitching distance parameter is obtained by feature matching based on at least one stitching seam image and a reference image corresponding to each stitching seam image, wherein the stitching seam image is a stitching seam image in a reference panoramic image of the target scene; A stitching unit is used to stitch together at least two images to be stitched based on the target stitching distance parameter to obtain a panoramic image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image stitching method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image stitching method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image stitching method as described in any one of claims 1 to 6.

Images