Image offset determination method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image offset determination method and device, electronic equipment and a storage medium. The image offset determination method comprises the following steps: acquiring a plurality of images and position relation information among the plurality of images, determining adjacent image pairs in the plurality of images according to the position relation information, and determining offset information between two images in the adjacent image pair for each adjacent image pair. Automatic and intelligent image offset determination is achieved, the processing efficiency is high, the accuracy is high, the position relation information between the images is used as a clue to assist in determining the offset information between the two images in the adjacent image pairs, unnecessary processing does not need to be carried out on non-adjacent images, the workload can be effectively reduced, the execution efficiency is improved, and the user experience is improved. And non-adjacent images are prevented from being wrongly registered, so that the accuracy can be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to an image offset determination method, an image stitching method, an image offset determination device, an image stitching device, an electronic device, and a storage medium. Background Technology

[0002] In the field of image processing, there are application scenarios where local images of a target object are acquired and stitched together to obtain a complete image of the target object. The following explanation uses an optical microscope as an example. An optical microscope is an imaging device widely used in fields such as biology, medicine, and materials science. It mainly consists of components such as an objective lens, a camera, and a stage. The objective lens is used for optical magnification of the target object, the camera is used to acquire images of the target object, and the stage is used to place the target object.

[0003] In optical microscopy imaging, the physical area containing the target object is typically divided into multiple acquisition fields. Images are acquired for each field of view, resulting in multiple images from different perspectives. The stage usually has motors in two orthogonal directions, horizontal and vertical, which move the target object to each acquisition field to acquire the corresponding image. After acquiring multiple images, they can be stitched together to provide a complete image of the target object.

[0004] However, due to the precision errors in the motor movement of the stage, the stitching operation is not a simple image placement but requires determining the offset information of adjacent images. Related technologies typically use manual methods or image feature-based algorithms to determine this offset information, resulting in slow processing speed and low accuracy of the offset information. Summary of the Invention

[0005] The present invention was proposed in view of the above-mentioned problems.

[0006] According to one aspect of the present invention, an image offset determination method is provided. The image offset determination method includes:

[0007] Acquire multiple images and the positional relationship information between the multiple images, wherein the multiple images include multiple local images of the target object taken at different shooting positions, and the multiple images are used to stitch together the overall image of the target object;

[0008] Based on the positional relationship information, adjacent image pairs are determined among multiple images, wherein each adjacent image pair includes two images whose shooting positions are horizontally or vertically adjacent among multiple images;

[0009] For each adjacent image pair, offset information between the two images in the adjacent image pair is determined, wherein the offset information is used to indicate the degree of horizontal offset and vertical offset between the two images in the adjacent image pair.

[0010] For example, determining the offset information between two images in the adjacent image pair includes: determining the peak correlation matrix of the two images in the adjacent image pair based on a phase registration algorithm, wherein the horizontal and vertical coordinates of the matrix points in the peak correlation matrix correspond to the horizontal and vertical offsets between the two images in the adjacent image pair, respectively; determining the target peak points in the peak correlation matrix; determining the candidate offset coordinates of the first image relative to the second image in the adjacent image pair based on the horizontal and vertical coordinates of all target peak points in the peak correlation matrix and the positional relationship information between the two images in the adjacent image pair; and determining the target offset coordinates from the candidate offset coordinates to obtain the offset information.

[0011] For example, determining the target peak point in the peak correlation matrix includes: determining a first allowable range of horizontal coordinates and a second allowable range of vertical coordinates of the peak point in the peak correlation matrix based on a preset overlap rate and a preset overlap error rate between images, wherein the preset overlap rate represents the overlap range of two images in each adjacent image pair; and determining the target peak point within the first and second allowable ranges of the peak correlation matrix.

[0012] For example, determining the target peak point in the peak correlation matrix includes: performing mean filtering on the peak correlation matrix to obtain a filtered peak correlation matrix; and determining the target peak point in the filtered peak correlation matrix.

[0013] For example, before performing mean filtering on the peak correlation matrix to obtain the filtered peak correlation matrix, determining the target peak point in the peak correlation matrix further includes: determining a first allowed range of horizontal coordinates and a second allowed range of vertical coordinates of the peak point in the peak correlation matrix based on a preset overlap rate and a preset overlap error rate between images, wherein the preset overlap rate represents the overlap range of two images in each adjacent image pair; setting the values ​​of all matrix points in the peak correlation matrix that are outside the first and second allowed ranges as target values, wherein the target values ​​include the minimum or average value or 0 of the values ​​of all matrix points in the peak correlation matrix.

[0014] For example, determining the target peak point in the peak correlation matrix includes: selecting a preset number of peak points in the peak correlation matrix, in descending order of their values, as the target peak points.

[0015] For example, determining a target peak point in the peak correlation matrix includes: determining a peak threshold based on the maximum value in the peak correlation matrix; and determining a target peak point in the peak correlation matrix based on the peak threshold, wherein the value of the target peak point is greater than or equal to the peak threshold.

[0016] For example, determining candidate offset coordinates of the first image relative to the second image in an adjacent image pair based on the horizontal and vertical coordinates of all target peak points in the peak correlation matrix and the positional relationship information between the two images in the adjacent image pair includes: determining multiple inferred offset coordinates corresponding to each of the target peak points based on the horizontal and vertical coordinates of all target peak points in the peak correlation matrix and the positional relationship information between the two images in the adjacent image pair; determining a first offset coordinate range for the horizontal offset of the first image relative to the second image in the adjacent image pair and a second offset coordinate range for the vertical offset based on a preset overlap rate and a preset overlap error rate between the images, wherein the preset overlap rate represents the overlap range of the two images in each adjacent image pair; and determining the inferred offset coordinates located within the first offset coordinate range and the second offset coordinate range as candidate offset coordinates among all inferred offset coordinates.

[0017] For example, there are multiple candidate offset coordinates. Determining the target offset coordinate from the candidate offset coordinates to obtain offset information includes: for each candidate offset coordinate, calculating the image cross-correlation value of the two images in the adjacent image pair at the overlapping position represented by the candidate offset coordinate to obtain the confidence value corresponding to the candidate offset coordinate; and determining the candidate offset coordinate with the largest corresponding confidence value as the target offset coordinate to obtain offset information.

[0018] According to another aspect of the present invention, an image stitching method is also provided. The image stitching method includes: the image offset determination method as described above; the image stitching method further includes: stitching multiple images based on the offset information of each adjacent image pair to obtain an overall image of the target object.

[0019] According to another aspect of the present invention, an image offset determination device is also provided. The image offset determination device includes: a first acquisition module, a first determination module, and a second determination module. The first acquisition module is configured to acquire multiple images and positional relationship information between the multiple images, wherein the multiple images include multiple partial images of a target object taken at different shooting positions, and the multiple images are used to stitch together a complete image of the target object; the first determination module is configured to determine adjacent image pairs among the multiple images based on the positional relationship information, wherein each adjacent image pair includes two images whose shooting positions are horizontally or vertically adjacent; the second determination module is configured to determine offset information between the two images in each adjacent image pair, wherein the offset information indicates the degree of horizontal and vertical offset between the two images in the adjacent image pair.

[0020] According to another aspect of the present invention, an image stitching apparatus is also provided. The image stitching apparatus includes: an image offset determining device for performing the image offset determining method as described above; the image stitching apparatus further includes a stitching module; the stitching module is used to stitch multiple images based on the offset information of each adjacent image pair to obtain an overall image of a target object.

[0021] According to another aspect of the present invention, an electronic device is also provided, comprising: a processor and a memory, wherein the memory stores computer program instructions, which, when executed by the processor, are used to perform the image offset determination method as described above or the image stitching method as described above.

[0022] According to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, which, when executed, are used to perform the image offset determination method or the image stitching method as described above.

[0023] In the above technical solution, the positional relationship information between multiple images is used to determine adjacent image pairs. For each adjacent image pair, the offset information between the two images in that pair is determined. This achieves automated and intelligent image offset determination, with high processing efficiency and accuracy. Furthermore, by using the positional relationship information between images as a clue to assist in determining the offset information between the two images in an adjacent image pair, unnecessary processing of non-adjacent images is eliminated, effectively reducing workload, improving execution efficiency, and avoiding incorrect registration of non-adjacent images, thus also improving accuracy.

[0024] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0025] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0026] Figure 1 A schematic flowchart of an image offset determination method according to an embodiment of the present invention is shown;

[0027] Figure 2 A visualization of the peak correlation matrix according to an embodiment of the present invention is shown;

[0028] Figure 3A schematic diagram of a first allowable value range and a second allowable value range in a peak correlation matrix according to an embodiment of the present invention is shown;

[0029] Figure 4 A schematic diagram showing partial inferred offset coordinates of a target peak point according to an embodiment of the present invention is provided.

[0030] Figure 5 A schematic block diagram of an image offset determination apparatus according to an embodiment of the present invention is shown;

[0031] Figure 6 A schematic block diagram of an image stitching apparatus according to an embodiment of the present invention is shown;

[0032] Figure 7 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a subset of embodiments of the present invention.

[0034] In related technologies, multiple images are typically stitched together manually, meaning the offset information between images is determined manually. This method has poor accuracy in determining offset information and is easily affected by subjective factors. Another approach utilizes the features of each image to determine the offset information between multiple images. This method requires extracting features from each image and determining the offset information between images based on the differences in features between every two images. This method is computationally very demanding, has low efficiency in determining offset information, and is prone to misclassification. For example, non-adjacent images might be identified as adjacent images due to similar features, or adjacent images might be identified as non-adjacent images due to small overlap or large feature differences, leading to incorrect image registration and affecting the accuracy of subsequent image stitching.

[0035] To at least address the aforementioned technical problems, this invention provides an image offset determination method. This method identifies adjacent image pairs within a plurality of images based on positional relationship information, and for each adjacent image pair, determines the offset information between the two images within that pair. This approach utilizes positional relationship information to identify potentially overlapping adjacent image pairs within a plurality of images, and then determines the offset information between these adjacent image pairs. Compared to manual methods, the solution provided by this invention is an intelligent and automated image offset determination scheme, requiring less user intervention and achieving high accuracy in offset information determination. Compared to schemes that utilize features to determine offset information, the solution provided by this invention effectively reduces computational load, improves the efficiency of offset information determination, and significantly enhances the accuracy in determining adjacent images, thereby contributing to improved accuracy in offset information determination and subsequent image stitching. The image offset determination method can be applied to any electronic device, i.e., executed by any electronic device. Specifically, the image offset determination method can be applied to the processor of any electronic device, i.e., executed by the processor of any electronic device.

[0036] For example, Figure 1 A schematic flowchart of an image offset determination method according to an embodiment of the present invention is shown. Figure 1 As shown, the image offset determination method includes steps S1100, S1200 and S1300.

[0037] In step S1100, multiple images and their positional relationships are acquired. These multiple images include several partial images of the target object taken from different shooting positions. These images are then stitched together to form a complete image of the target object. The multiple images may include multiple partial images of the same target object taken from different shooting positions, with overlapping areas between adjacent images. Each image captures a different region of the target object, and subsequent stitching processing forms the complete image of the target object. For example, for the same target object, by sequentially moving the shooting position horizontally and capturing partial images, a row of partial images can be captured. The shooting position can then be adjusted vertically to the next row, and this process can be repeated until the captured partial images cover the entire area of ​​the target object, thus acquiring multiple images. These multiple images of the target object can be acquired using any image acquisition device, such as an optical microscope or any similar image acquisition device. The image acquisition device may include a motor, such as the motor of the stage of an optical microscope. The motor can move the target object to different shooting positions to acquire images of the target object at each shooting position. For example, multiple partial images of a target object can be captured using an optical microscope. The stage of an optical microscope typically has motors in two orthogonal directions, horizontal and vertical, enabling movement of the target object to change the shooting position and capture partial images of the target object, thus obtaining multiple images. The positional relationship information between these multiple images can be used to indicate their spatial relationship (simply referred to as positional relationship). For example, positional relationship information can include image position information, which indicates whether each image in the multiple images is adjacent to other images. The two images in an adjacent image pair can be vertically adjacent or horizontally adjacent. For example, multiple images include image 1, image 2, image 3, and image 4. The positional relationship information between these images indicates that image 1 and image 2 are horizontally adjacent, image 3 and image 4 are horizontally adjacent, image 1 and image 3 are vertically adjacent, and image 2 and image 4 are vertically adjacent. In some embodiments, when acquiring multiple images, a preset name can be obtained for each image. The preset name can directly indicate the positional relationship between the multiple images. In other words, positional relationship information can include a preset name for the image. For example, moving the target object horizontally along the first row and acquiring three images, named I... 1-1 I 1-2 and I 1-3 Move the target object horizontally along the second row and acquire three images respectively, named I. 2-1 I 2-2 and I 2-3Six images were acquired in total. These six images were named according to a horizontal and vertical arrangement pattern. The preset names of the images indicate that... 1-1 and I 1-2 For horizontally adjacent images, I 1-2 and I 1-3 For horizontally adjacent images, I 2-1 and I 2-2 For horizontally adjacent images, I 2-2 and I 2-3 For horizontally adjacent images, I 1-1 and I 2-1 For vertically adjacent images, I 1-2 and I 2-2 For vertically adjacent images, I 1-3 and I 2-3 These are vertically adjacent images. In other embodiments, when acquiring multiple images, the shooting position coordinates of each image can be obtained, and the positional relationship between the multiple images can be determined based on the shooting position coordinates of each image. In other words, the positional relationship information can include the shooting position coordinates.

[0038] In step S1200, adjacent image pairs are determined from multiple images based on positional relationship information. Each adjacent image pair includes two images whose shooting positions are horizontally or vertically adjacent. Adjacent images can be pairs of two images whose shooting positions are spatially adjacent (left-right) or vertically (top-bottom). Adjacent images can be determined from multiple images based on positional relationship information, and thus, two adjacent images are identified as an adjacent image pair. For example, multiple images include image 1, image 2, image 3, and image 4. The positional relationship information between the multiple images indicates that image 1 and image 2 are horizontally adjacent, image 3 and image 4 are horizontally adjacent, image 1 and image 3 are vertically adjacent, and image 2 and image 4 are vertically adjacent. Therefore, based on the positional relationship information of these four images, four adjacent image pairs can be determined: adjacent image pair 1 can include image 1 and image 2, adjacent image pair 2 can include image 3 and image 4, adjacent image pair 3 can include image 1 and image 3, and adjacent image pair 4 can include image 2 and image 4.

[0039] In step S1300, for each adjacent image pair, the offset information between the two images in the adjacent image pair is determined. The offset information indicates the degree of lateral and vertical offset between the two images in the adjacent image pair. The offset information can indicate the relative displacement between the two images in the horizontal (horizontal) and vertical (vertical) directions. The offset information can be the coordinates of a preset registration point of one image in the image coordinate system of the other image, thus representing the degree of lateral and vertical offset between the two images. The preset registration point is any pixel in the image, such as the top-left corner, top-right corner, bottom-left corner, bottom-right corner, center point, etc. The two images in an adjacent image pair usually contain partially overlapping areas. Based on the offset information between the two images in the adjacent image pair, the overlapping area can be determined for stitching the two images in the adjacent image pair. Any method can be used to process the two images in an adjacent image pair to determine the offset information between them. For example, a scale-invariant feature transform (SIFT) algorithm can be used to extract feature points from two images separately, and then the extracted feature points can be matched and filtered to determine the offset information. Alternatively, a cross-correlation algorithm can be used to process the two images of an adjacent image pair, determine the region with the highest cross-correlation value, and thus determine the offset information. Exemplarily, the operation of determining the offset information for multiple adjacent image pairs can be performed in parallel or sequentially according to a preset order. For example, multiple adjacent image pairs can be processed simultaneously to determine the positional offset information between the two images in each adjacent image.

[0040] In the above technical solution, the positional relationship information between multiple images is used to determine adjacent image pairs. For each adjacent image pair, the offset information between the two images in that pair is determined. This achieves automated and intelligent image offset determination, with high processing efficiency and accuracy. Furthermore, by using the positional relationship information between images as a clue to assist in determining the offset information between the two images in an adjacent image pair, unnecessary processing of non-adjacent images is eliminated, effectively reducing workload, improving execution efficiency, and avoiding incorrect registration of non-adjacent images, thus also improving accuracy.

[0041] For example, step S1300, which determines the offset information between two images in the adjacent image pair, includes steps S1310, S1320, S1330, and S1340.

[0042] In step S1310, based on the phase registration algorithm, the peak correlation matrix of the two images in the adjacent image pair is determined. The horizontal and vertical coordinates of the matrix points in the peak correlation matrix correspond to the horizontal and vertical offsets between the two images in the adjacent image pair, respectively. The phase registration algorithm is an image registration method based on Fourier transform, which determines the offset information between the two images by calculating the phase correlation in the frequency domain. Fourier transforms can be performed on the two images in the adjacent image pair to obtain their corresponding spectra. The inverse Fourier transform of the cross-power spectra of the two images yields the peak correlation matrix of the two images in the adjacent image pair. Each matrix point in the peak correlation matrix represents a possible offset, and thus the horizontal and vertical coordinates of the matrix point in the peak correlation matrix correspond to the horizontal and vertical offsets between the two images in the adjacent image pair, respectively.

[0043] In step S1320, a target peak point is determined in the peak correlation matrix. The peak correlation matrix may contain multiple peak points. Each of these peak points could be a correctly matched position, but calculating candidate offset coordinates for each peak point and then filtering them would be computationally intensive. Therefore, the target peak point can be determined from among the multiple peak points in the peak correlation matrix. In some embodiments, peak points with values ​​greater than a peak threshold can be determined as the target peak point. In other embodiments, a predetermined number of peak points with values ​​ranked from largest to smallest can be determined as the target peak point. Figure 2 A visual schematic diagram of the peak correlation matrix according to an embodiment of the present invention is shown. Figure 2 As shown, the peak correlation matrix has multiple peak points, and the target peak point can be determined from among these multiple peak points.

[0044] In step S1330, based on the horizontal and vertical coordinates of all target peak points in the peak correlation matrix and the positional relationship information between the two images in the adjacent image pair, candidate offset coordinates of the first image relative to the second image in the adjacent image pair are determined. For each target peak point among all target peak points, multiple inferred offset coordinates can be determined based on the horizontal and vertical coordinates of the target peak point in the peak correlation matrix. It can be understood that since the phase registration algorithm is based on Fourier transform, and Fourier transform is periodic, for a peak point, there can be multiple inferred offset coordinates based on the horizontal and vertical coordinates in the peak correlation matrix. The inferred offset coordinates can be filtered based on the positional relationship information between the two images in the adjacent image pair to determine the candidate offset coordinates of the first image relative to the second image in the adjacent image pair. For example, a reasonable offset range can be determined based on the positional relationship information between the two images in the adjacent image pair, and the inferred offset coordinates within the reasonable offset range can be determined as candidate offset coordinates.

[0045] In step S1340, a target offset coordinate is determined from the candidate offset coordinates to obtain offset information. In some embodiments, for each candidate offset coordinate, the overlapping area between two images in an adjacent image pair under that candidate offset coordinate can be determined based on the candidate offset coordinate. The correlation of the overlapping area is judged, and the candidate offset coordinate with the highest correlation among multiple candidate offset coordinates can be determined as the target offset coordinate. The target offset coordinate can represent the horizontal and vertical offset degrees between two images in the adjacent image pair, thereby obtaining offset information. In other embodiments, for each candidate offset coordinate, the overlapping area between two images in an adjacent image pair under that candidate offset coordinate can be determined based on the candidate offset coordinate. A preset number of feature points are selected in the overlapping area, and the average matching degree of the preset number of feature points in the two images is calculated. The candidate offset coordinate with the highest average matching degree among multiple candidate offset coordinates can be determined as the target offset coordinate.

[0046] In the above technical solution, based on the phase registration algorithm, the peak correlation matrix of the two images in the adjacent image pair is determined, thereby identifying the target peak point. Combined with the positional relationship information between the two images in the adjacent image pair, candidate offset coordinates are determined, and the target offset coordinate is then determined from the candidate offset coordinates to obtain offset information. Therefore, based on the phase registration algorithm, frequency domain processing of the two images in the adjacent image pair can quickly determine multiple candidate offset coordinates and select a suitable target offset coordinate. The calculation is simple, the processing speed is fast, and registration only needs to be performed on the two images in each adjacent image pair, eliminating the need for unnecessary processing of non-adjacent images, effectively reducing workload and improving accuracy.

[0047] For example, step S1320, which determines the target peak point in the peak correlation matrix, includes steps S1321 and S1322.

[0048] In step S1321, based on the preset overlap rate and preset overlap error rate between images, a first allowable range of horizontal coordinates and a second allowable range of vertical coordinates for the peak points in the peak correlation matrix are determined. The preset overlap rate represents the overlap range between two images in each adjacent image pair. The preset overlap rate can be a pre-defined proportion of the overlapping area between adjacent images to the area of ​​a single image, set during image acquisition. The preset overlap rate can be expressed as a percentage. For example, a preset overlap rate of 10% indicates that approximately 10% of the area between the two images in each adjacent image pair overlaps. The preset overlap error rate represents the allowable deviation range from the preset overlap rate, considering potential errors during actual shooting. These errors may originate from limitations in the movement accuracy of the shooting equipment, operator handshake, environmental factors, etc. The preset overlap error rate can be divided into a horizontal preset overlap error rate and a vertical preset overlap error rate. The preset overlap error rate can be expressed as a percentage; the horizontal preset overlap error rate represents the horizontal offset range of the overlapping area, and the vertical preset overlap error rate represents the vertical offset range of the overlapping area. Based on preset overlap rates and preset overlap error rates between images, a first allowable range of horizontal coordinates and a second allowable range of vertical coordinates for peak points in the peak correlation matrix can be determined. For example, if the preset overlap rate is R, the preset horizontal overlap error rate is r1, the preset vertical overlap error rate is r2, the horizontal side length of the image is W, and the vertical side length is H, for horizontally adjacent image pairs, the preset overlap rate represents the horizontal overlap range between the two images in the adjacent image pair. The first allowable range of horizontal coordinates for peak points in the peak correlation matrix can be [max(0, R-r1)*W, min(1, R+r1)*W] and [W-min(1, R+r1)*W, W-max(0, R-r1)*W]. The second allowed range of values ​​for the vertical coordinate can be [-r2*H, r2*H] and [H-r2*H, H+r2*H]. For vertically adjacent image pairs, the preset overlap rate represents the vertical overlap range between the two images in the adjacent image pair. The first allowed range of values ​​for the horizontal coordinate of the peak point in the peak correlation matrix can be [-r1*W, r1*W] and [W-r1*W, W+r1*W]. The second allowed range of values ​​for the vertical coordinate can be [max(0, R-r2)*H, min(1, R+r2)*H].

[0049] And [H-min(1, R+r2)*H, H-max(0, R-r2)*H]. For example, the preset overlap rate is 10%, the preset horizontal overlap error rate is 2%, and the preset vertical overlap error rate is 2%. The horizontal side length of the image is 1000 pixels, the vertical side length of the image is 500 pixels, and the size of the peak correlation matrix is ​​the same as the image size. If it is a pair of horizontally adjacent images, the first allowed value range of the horizontal coordinate can be [80((10%-2%)*1000), 120((10%+2%)*1000)] and [880(1000-120), 920(1000-80)]. The second allowed value range of the vertical coordinate can be [-10(-2%*50)]. [0), 10(2%*500)] and [490(500-10), 510(500+10)]; If it is a vertically adjacent image pair, the first allowed value range of the horizontal coordinate can be [-100(-10%*1000), 100(10%*1000)] and [900(1000-100), 1100(1000+100)], and the second allowed value range of the vertical coordinate can be [40((10%-2%)*500), 60((10%+2%)*500)] and [440(500-60), 460(500-40)].

[0050] In step S1322, a target peak point is determined within the first and second allowed value ranges of the peak correlation matrix. In some embodiments, a selectable region for peak points can be determined in the peak correlation matrix based on the first and second allowed value ranges, and the target peak point can be determined within the selectable region. For example, peak points with values ​​greater than a threshold in the selectable region are selected as the target peak points. In other embodiments, a preset number of peak points can be filtered in the peak correlation matrix, and then the peak points within the first and second allowed value ranges from the preset number of peak points are determined as the target peak points. Figure 3 A schematic diagram illustrating a first and second allowed value range in a peak correlation matrix according to an embodiment of the present invention is shown. Figure 3 As shown, the area displaying the peak point is the area within the first and second allowed value ranges, which is the selectable area. The target peak point can be determined within the selectable area.

[0051] In the above technical solution, based on a preset overlap rate and a preset overlap error rate between images, a first allowable range of horizontal coordinates and a second allowable range of vertical coordinates for the peak points in the peak correlation matrix are determined. The target peak point is then determined within these ranges. This ensures that the target peak point is within a reasonable range, preventing peak points outside this range from being identified as the target peak point due to noise, thus improving the accuracy of determining the target peak point and consequently improving the accuracy of determining the offset information.

[0052] For example, step S1320, which determines the target peak point in the peak correlation matrix, includes steps S1323 and S1324.

[0053] In step S1323, mean filtering is performed on the peak correlation matrix to obtain a filtered peak correlation matrix. Mean filtering sets the value of each matrix point in the peak correlation matrix to the average value of matrix points within a preset range surrounding that point. The preset range can be set according to actual needs; for example, the preset range can be 3*3, meaning that the average value of nine matrix points, including the given matrix point, is determined, and the value of the matrix point is set to this average value. Mean filtering of the peak correlation matrix can smooth the data and suppress noise in the peak correlation matrix.

[0054] In step S1324, the target peak point is determined in the filtered peak correlation matrix. The filtered peak correlation matrix maintains the same dimensions as the original peak correlation matrix, but its value distribution changes. The local maxima regions become smoother and more concentrated, noise peaks are suppressed, and the signal-to-noise ratio is improved, allowing for better determination of the target peak point. The specific method for determining the target peak point is similar to that in step S1320 above, and can be referred to step S1320 above. For simplicity, it will not be repeated here.

[0055] In the above technical solution, mean filtering is applied to the peak correlation matrix to obtain a filtered peak correlation matrix. The target peak point is then determined from this filtered peak correlation matrix. Therefore, mean filtering of the peak correlation matrix can suppress noise interference, improve the signal-to-noise ratio, and increase the accuracy of determining the target peak point.

[0056] For example, before performing mean filtering on the peak correlation matrix in step S1323 to obtain the filtered peak correlation matrix, step S1320 further includes steps S1325 and S1326. It can be understood that steps S1325 and S1326 are performed before step S1323.

[0057] In step S1325, based on the preset overlap rate and preset overlap error rate between images, a first allowable range of horizontal coordinates and a second allowable range of vertical coordinates for the peak points in the peak correlation matrix are determined. The preset overlap rate represents the overlap range between two images in each adjacent image pair. Step S1325 is similar to step S1321, and the method used to determine the first allowable range of horizontal coordinates and the second allowable range of vertical coordinates for the peak points in the peak correlation matrix in step S1321 above can be referenced. For simplicity, it will not be repeated here.

[0058] In step S1326, the values ​​of all matrix points in the peak correlation matrix that are outside the first and second allowed value ranges are set as target values. The target values ​​include the minimum, average, or zero of all matrix points in the peak correlation matrix. Specifically, the target values ​​can be set to the matrix points whose horizontal coordinates are outside the first and second allowed value ranges or whose vertical coordinates are outside the second allowed value range. In other words, the matrix points within the first and second allowed value ranges remain unchanged, while the matrix points outside these ranges are set as target values. The target value can be the minimum, average, or zero of all matrix points in the peak correlation matrix. This can further suppress noise in the peak correlation matrix. When performing mean filtering on the peak correlation matrix, it can prevent matrix points outside the first and second allowed value ranges from affecting the matrix points within those ranges.

[0059] In the above technical solution, before applying mean filtering to the peak correlation matrix, the values ​​of all matrix points outside the first and second allowable value ranges in the peak correlation matrix are set as target values. This avoids the influence of matrix points outside the reasonable range on the matrix points within the reasonable range during mean filtering, further reducing the impact of noise on determining the target peak point and improving the accuracy of target peak point determination.

[0060] For example, step S1320, determining the target peak points in the peak correlation matrix, includes step S1327. In step S1327, in the peak correlation matrix, a preset number of peak points, arranged in descending order of their values, are selected as target peak points. The preset number can represent a fixed number of target peak points to be selected in the peak correlation matrix. In simple scenarios or when computational resources are limited, the preset number may be small (e.g., 2-5); in complex scenarios or when high accuracy is required, the preset number may be large (e.g., 5-10). Users can set the preset number as needed. Higher peak point values ​​generally indicate a greater likelihood of matching the correct offset information. In the peak correlation matrix, peak points (points with peaks in the matrix) can be sorted from high to low values, with priority given to selecting the preset number of peak points whose values ​​decrease sequentially from the maximum as target peak points.

[0061] In the above technical solution, a preset number of peak points are selected from the peak correlation matrix, with the values ​​decreasing sequentially from the largest to the smallest, as target peak points. This allows for the rapid selection of the preset number of target peak points with higher accuracy.

[0062] For example, step S1320, determining the target peak point in the peak correlation matrix, includes steps S1328 and S1329. In step S1328, the peak threshold is determined based on the maximum value in the peak correlation matrix. The maximum value of the peak point can be obtained from the peak correlation matrix, and the peak threshold can be determined based on the maximum value in the peak correlation matrix. For example, t times the maximum value can be determined as the peak threshold, where t can be any value from 0.5 to 0.95. For instance, 0.9 times the maximum value in the peak correlation matrix can be determined as the peak threshold.

[0063] In step S1329, target peak points are determined in the peak correlation matrix based on the peak threshold, wherein the value of the target peak point is greater than or equal to the peak threshold. Peak points in the peak correlation matrix whose values ​​are greater than or equal to the peak threshold are identified as target peak points. Multiple target peak points may be included, and each peak point may potentially match the correct offset information.

[0064] In the above technical solution, the peak threshold is determined based on the maximum value in the peak correlation matrix, and peaks with values ​​greater than or equal to the peak threshold are identified as target peak points. Therefore, by associating the peak threshold with the maximum value, the target peak point can be determined quickly and accurately.

[0065] For example, steps S1327, S1328, and S1329 can be executed sequentially. For instance, step S1327 can be executed first to determine a preset number of peak points. Then, steps S1328 and S1329 can be executed for the preset number of peak points to determine whether the preset number of peak points is greater than or equal to a peak threshold. If it is greater than or equal to the peak threshold, it is determined as a target peak point; if it is less than the peak threshold, it is not considered a target peak point. For example, if the preset number is 5 and the peak threshold is 120, in the process of using a preset number of peak points whose values ​​decrease sequentially from the maximum as the target peak points, the values ​​of the 5 peak points decreasing sequentially from the maximum are 150, 140, 130, 90, and 80. Peak points with values ​​of 90 and 80 are less than the peak threshold, so the target peak points can include peak points with values ​​of 150, 140, and 130. Steps S1327, S1328, and S1329 can be executed in parallel, that is, in the process of determining the preset number of peak points, it is determined whether the peak points are greater than or equal to the peak threshold.

[0066] For example, step S1330 determines the candidate offset coordinates of the first image relative to the second image in the adjacent image pair based on the horizontal and vertical coordinates of all target peak points in the peak correlation matrix and the positional relationship information between the two images in the adjacent image pair, including steps S1331, S1332 and S1333.

[0067] In step S1331, based on the horizontal and vertical coordinates of all target peak points in the peak correlation matrix and the positional relationship information between the two images in the adjacent image pair, multiple inferred offset coordinates corresponding to each of the target peak points are determined.

[0068] For each target peak point among all target peak points, multiple inferred offset coordinates can be determined based on its horizontal and vertical coordinates in the peak correlation matrix. It can be understood that since the phase registration algorithm is based on Fourier transform, and Fourier transform is periodic, a peak point can have multiple inferred offset coordinates based on its horizontal and vertical coordinates in the peak correlation matrix. For example, an adjacent image pair includes a first image and a second image. The horizontal side length of both the first and second images can be W, and the vertical side length can be H. An offset coordinate system can be established based on the upper left corner of the second image. The horizontal coordinate x and vertical coordinate y of a peak point in the peak correlation matrix can then determine the inferred offset coordinates in the offset coordinate system as: (x, y), (x, Hy), (Wx, y), and (Wx, Hy). If the adjacent image pair is a horizontally adjacent pair, meaning the horizontal positional relationship between the first and second images is determined, but the vertical positional relationship between the first and second images is uncertain, then a peak point... The horizontal coordinate (x) and vertical coordinate (y) in the peak correlation matrix can also determine the inferred offset coordinates in the offset coordinate system, including (x, -y), (x, H+y), (Wx, -y), and (Wx, H+y). If adjacent image pairs are vertically adjacent, meaning the vertical positional relationship between the first and second images is determined, but the horizontal positional relationship between the first and second images is uncertain, then the horizontal coordinate (x) and vertical coordinate (y) of a peak point in the peak correlation matrix can also determine the inferred offset coordinates in the offset coordinate system, including (-x, y), (-x, Hy), (W+x, y), and (W+x, Hy). In other words, for the horizontal and vertical coordinates of a peak point in the peak correlation matrix, eight inferred offset coordinates can be determined. For a pair of horizontally adjacent image points, the horizontal x-coordinate and vertical y-coordinate of a peak point in the peak correlation matrix can be determined as (x, y), (x, Hy), (Wx, y), (Wx, Hy), (x, -y), (x, H+y), (Wx, -y), and (Wx, H+y), for a total of eight inferred offset coordinates. For a pair of vertically adjacent image points, the horizontal x-coordinate and vertical y-coordinate of a peak point in the peak correlation matrix can be determined as (x, y), (x, Hy), (Wx, y), (Wx, Hy), (-x, y), (-x, Hy), (W+x, y), and (W+x, Hy), for a total of eight inferred offset coordinates. Each of all target peak points can be determined with eight inferred offset coordinates.

[0069] Figure 4 A schematic diagram illustrating a portion of the inferred offset coordinates of a target peak point according to an embodiment of the present invention is shown. Figure 4As shown, the pink rectangle represents the second image in an adjacent image pair, and the green rectangle represents the first image in the same pair. An offset coordinate system is established with the top-left corner of the second image as the reference point. The offset coordinates are then inferred to be the position of the top-left corner of the first image within that offset coordinate system. For simplicity... Figure 4 The offset coordinate system is not shown. The horizontal side length of both the first and second images can be W, and the vertical side length can be H. A peak point has horizontal coordinates x and vertical coordinates y in the peak correlation matrix. Figure 4 In the middle, from left to right, are the predicted offset coordinates (x, y), (x, Hy), (Wx, y), and (Wx, Hy) corresponding to the target peak point.

[0070] In step S1332, based on the preset overlap rate and preset overlap error rate between images, the first offset coordinate range of the horizontal offset of the first image relative to the second image in the adjacent image pair and the second offset coordinate range of the vertical offset are determined. The preset overlap rate represents the overlap range of the two images in each adjacent image pair. The preset overlap rate can be a pre-set proportion of the overlapping area between adjacent images to the area of ​​a single image during image acquisition. The preset overlap rate can be expressed as a percentage. For example, a preset overlap rate of 10% indicates that approximately 10% of the area of ​​the two images in each adjacent image pair overlaps. The preset overlap error rate can represent the allowable deviation range from the preset overlap rate, considering potential errors during actual shooting. These errors may originate from limitations in the movement accuracy of the shooting device, operator handshake, environmental factors, etc. The preset overlap error rate can be divided into a horizontal preset overlap error rate and a vertical preset overlap error rate. The preset overlap error rate can be expressed as a percentage; the horizontal preset overlap error rate represents the horizontal offset range of the overlapping area, and the vertical preset overlap error rate represents the vertical offset range of the overlapping area. Based on a preset overlap rate and a preset overlap error rate between images, a first offset coordinate range and a second offset coordinate range for the horizontal offset of the first image relative to the second image in the adjacent image pair are determined. The first and second offset coordinate ranges can correspond to the first and second allowed value ranges in step S1321; that is, the first offset coordinate range can be numerically identical to the first allowed value range, and the second offset coordinate range can be numerically identical to the second allowed value range. It can be understood that when step S1321 is executed, the values ​​of the first and second allowed value ranges can be reused to determine the first offset coordinate range for the horizontal offset of the first image relative to the second image and the second offset coordinate range for the vertical offset. If step S1321 is not executed, a similar method to step S1321 can be used to determine the first offset coordinate range for the horizontal offset of the first image relative to the second image and the second offset coordinate range for the vertical offset. The specific determination method can be referred to the above step S1321, and for simplicity, it will not be repeated here.

[0071] In step S1333, among all the predicted offset coordinates, those falling within the range of the first and second offset coordinates are determined as candidate offset coordinates. All predicted offset coordinates include eight predicted offset coordinates corresponding to each target peak point. For each predicted offset coordinate, it can be determined whether its horizontal and vertical coordinates fall within the range of the first and second offset coordinates. If they do, the predicted offset coordinate is determined as a candidate offset coordinate; if either its horizontal or vertical coordinate is outside the range, it is not a candidate offset coordinate and can be deleted.

[0072] In the above technical solution, multiple inferred offset coordinates are determined for each of the target peak points. Among all the inferred offset coordinates, those falling within the first offset coordinate range and the second offset coordinate range are identified as candidate offset coordinates. Thus, by filtering all inferred offset coordinates using the first and second offset coordinate ranges, candidate offset coordinates are determined within a reasonable range. This reduces the computational workload for determining offset information from candidate offset coordinates and improves the efficiency of offset information determination.

[0073] For example, there are multiple candidate offset coordinates. If there is only one candidate offset coordinate, it can be directly used as the target offset coordinate to obtain offset information. If there are multiple candidate offset coordinates, it is necessary to filter and determine one candidate offset coordinate as the target offset coordinate to obtain offset information. Step S1340, which determines the target offset coordinate from the candidate offset coordinates to obtain offset information, includes steps S1341 and S1342.

[0074] In step S1341, for each candidate offset coordinate, the image cross-correlation value of the two images in the adjacent image pair at the overlapping position represented by the candidate offset coordinate is calculated to obtain the confidence value corresponding to the candidate offset coordinate. For each candidate offset coordinate, the overlapping region of the two images in the adjacent image pair under that candidate offset coordinate can be determined, and the image cross-correlation value of the overlapping region of the two images can be calculated to obtain the confidence value corresponding to the candidate offset coordinate. The image cross-correlation value can be used to indicate the similarity of the overlapping regions of the two images. The larger the image cross-correlation value, the higher the similarity of the overlapping regions of the images, that is, the more accurate the candidate offset coordinate. It can be understood that inaccurate candidate offset coordinates will lead to the introduction of non-overlapping regions in the overlapping region determined under the candidate offset coordinate, and the corresponding image cross-correlation value will be relatively low. Therefore, the accuracy of the candidate offset coordinate can be evaluated by the image cross-correlation value. Any algorithm can be used to calculate the image cross-correlation value of the overlapping region of the two images. For example, the image cross-correlation value of the overlapping region of the two images can be calculated by the Normalized Cross-Correlation (NCC) algorithm to serve as the confidence value corresponding to the candidate offset coordinate.

[0075] In step S1342, the candidate offset coordinate with the highest corresponding confidence value is determined as the target offset coordinate to obtain offset information. It can be understood that the higher the confidence value, the higher the similarity of the overlapping areas of the two images at the corresponding candidate offset coordinates. Therefore, the candidate offset coordinate with the highest corresponding confidence value among multiple candidate offset coordinates can be determined as the target offset coordinate to obtain offset information.

[0076] In the above technical solution, for each candidate offset coordinate, the cross-correlation value of the two images in the adjacent image pair at the overlapping position represented by the candidate offset coordinate is calculated to obtain the confidence value corresponding to the candidate offset coordinate; the candidate offset coordinate with the largest confidence value is determined as the target offset coordinate to obtain offset information. Therefore, by filtering multiple candidate offset coordinates through the cross-correlation value at the overlapping position, it is possible to avoid filtering the entire image region, improving processing speed, while ensuring the accuracy of the determined offset information.

[0077] Exemplarily, according to another aspect of the present invention, an image stitching method is also provided. The image stitching method includes the image offset determination method described above. The image stitching method further includes: stitching multiple images based on the offset information of each adjacent image pair to obtain an overall image of a target object. The offset information of each adjacent image pair can be determined based on the image offset determination method described above. Multiple images can be stitched based on all offset information to obtain an overall image of the target object. Optionally, each offset information may also have a corresponding confidence level; suitable offset information can be determined from all offset information based on the confidence level to stitch multiple images together.

[0078] In the above technical solution, multiple images are stitched together based on the offset information of each adjacent image pair determined by the image offset determination method. This allows for fast and accurate stitching of multiple images.

[0079] By way of example, according to another aspect of the present invention, an image offset determination device is also provided. Figure 5 A schematic block diagram of an image offset determination apparatus 500 according to an embodiment of the present invention is shown. Figure 5 As shown, the image offset determination device 500 includes a first acquisition module 510, a first determination module 520, and a second determination module 530.

[0080] The first acquisition module 510 is used to acquire multiple images and positional relationship information between the multiple images. The multiple images include multiple partial images of the target object taken from different shooting positions, and the multiple images are stitched together to form a complete image of the target object. The first determination module 520 is used to determine adjacent image pairs among the multiple images based on the positional relationship information. Each adjacent image pair includes two images whose shooting positions are horizontally or vertically adjacent. The second determination module 530 is used to determine the offset information between the two images in each adjacent image pair. The offset information indicates the degree of horizontal and vertical offset between the two images in the adjacent image pair.

[0081] For example, the second determining module 530 includes a first determining submodule, a second determining submodule, a third determining submodule, and a fourth determining submodule. The first determining submodule is used to determine the peak correlation matrix of the two images in the adjacent image pair based on a phase registration algorithm, wherein the horizontal and vertical coordinates of the matrix points in the peak correlation matrix correspond to the horizontal and vertical offsets between the two images in the adjacent image pair, respectively. The second determining submodule is used to determine the target peak point in the peak correlation matrix. The third determining submodule is used to determine the candidate offset coordinates of the first image relative to the second image in the adjacent image pair based on the horizontal and vertical coordinates of all target peak points in the peak correlation matrix and the positional relationship information between the two images in the adjacent image pair. The fourth determining submodule is used to determine the target offset coordinates from the candidate offset coordinates to obtain offset information.

[0082] For example, the second determining submodule includes a first determining unit and a second determining unit. The first determining unit is used to determine a first allowed range of horizontal coordinates and a second allowed range of vertical coordinates of the peak point in the peak correlation matrix based on a preset overlap rate and a preset overlap error rate between images, wherein the preset overlap rate represents the overlap range of the two images in each adjacent image pair. The second determining unit is used to determine the target peak point within the first and second allowed ranges of the peak correlation matrix.

[0083] For example, the second determining submodule includes a filtering unit and a third determining unit. The filtering unit is used to perform mean filtering on the peak correlation matrix to obtain a filtered peak correlation matrix. The third determining unit is used to determine the target peak point in the filtered peak correlation matrix.

[0084] For example, the second determining submodule includes a fourth determining unit and a fifth determining unit. The fourth determining unit is used to determine a first allowed range of horizontal coordinates and a second allowed range of vertical coordinates for peak points in the peak correlation matrix based on a preset overlap rate and a preset overlap error rate between images, wherein the preset overlap rate represents the overlap range between two images in each adjacent image pair. The fifth determining unit is used to set the values ​​of all matrix points in the peak correlation matrix that are outside the first and second allowed ranges as target values, wherein the target values ​​include the minimum or average value or 0 of all matrix points in the peak correlation matrix.

[0085] For example, the second determining submodule includes a sixth determining unit. The sixth determining unit is used to select a preset number of peak points in the peak correlation matrix as target peak points, in descending order of their values.

[0086] For example, the second determining submodule includes a seventh determining unit and an eighth determining unit. The seventh determining unit is used to determine a peak threshold based on the maximum value in the peak correlation matrix. The eighth determining unit is used to determine a target peak point in the peak correlation matrix based on the peak threshold, wherein the value of the target peak point is greater than or equal to the peak threshold.

[0087] For example, the third determining submodule includes a ninth determining unit, a tenth determining unit, and an eleventh determining unit. The ninth determining unit is used to determine multiple inferred offset coordinates corresponding to each of the target peak points based on the horizontal and vertical coordinates of all target peak points in the peak correlation matrix and the positional relationship information between the two images in the adjacent image pair. The tenth determining unit is used to determine a first offset coordinate range for the horizontal offset of the first image relative to the second image in the adjacent image pair and a second offset coordinate range for the vertical offset based on a preset overlap rate and a preset overlap error rate between the images, wherein the preset overlap rate represents the overlap range between the two images in each adjacent image pair. The eleventh determining unit is used to determine the inferred offset coordinates located within the first offset coordinate range and the second offset coordinate range from all inferred offset coordinates as candidate offset coordinates.

[0088] For example, there are multiple candidate offset coordinates, and the fourth determination submodule includes a calculation unit and a twelfth determination unit. The calculation unit calculates the cross-correlation value of the two images in the adjacent image pair at the overlapping position represented by the candidate offset coordinate for each candidate offset coordinate, to obtain the confidence value corresponding to the candidate offset coordinate. The twelfth determination unit determines the candidate offset coordinate with the largest corresponding confidence value as the target offset coordinate, to obtain offset information.

[0089] By way of example, according to another aspect of the present invention, an image stitching device is also provided. Figure 6 A schematic block diagram of an image stitching apparatus 600 according to an embodiment of the present invention is shown. Figure 6 As shown, the image stitching device 600 includes an image offset determination device 500 and a stitching module 610. The image offset determination device 500 is used to perform the image offset determination method described above; the stitching module 610 is used to stitch multiple images based on the offset information of each adjacent image pair to obtain a complete image of the target object.

[0090] By way of example, according to another aspect of the present invention, an electronic device is also provided. Figure 7A schematic block diagram of an electronic device 700 according to an embodiment of the present invention is shown. The electronic device 700 includes a processor 710 and a memory 720. The memory 720 stores computer program instructions, which, when executed by the processor 710, are used to perform the image offset determination method or the image stitching method described above.

[0091] By way of example, according to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, which, when executed, are used to perform the image offset determination method or the image stitching method described above. The storage medium may, for example, include an erasable programmable read-only memory (EPROM), a portable read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The storage medium may be any combination of one or more computer-readable storage media.

[0092] By way of example, according to another aspect of the present invention, a computer program product is also provided, including computer program instructions, which, when run, are used to perform the image offset determination method or the image stitching method as described above.

[0093] Those skilled in the art can understand the specific implementation schemes and beneficial effects of the above-mentioned image offset determination method and image stitching method by reading the relevant descriptions. For the sake of brevity, they will not be elaborated further here.

[0094] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of this application. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of this application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.

[0095] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0097] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0098] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more of the various inventive aspects, features of this application are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of this application. However, this approach should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, the inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0099] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0100] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0101] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the image offset determination apparatus or image stitching apparatus according to the embodiments of this application. This application can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0102] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0103] The above description is merely a specific embodiment or illustration of the embodiments of this application. The scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. The scope of protection of this application shall be determined by the scope of the claims.

Claims

1. A method for determining image offset, characterized in that, include: Acquire multiple images and positional relationship information between the multiple images, wherein the multiple images include multiple local images of the target object taken at different shooting positions, and the multiple images are used to stitch together a whole image of the target object; Based on the positional relationship information, adjacent image pairs are determined among the plurality of images, wherein each adjacent image pair includes two images whose shooting positions are horizontally or vertically adjacent among the plurality of images; For each of the adjacent image pairs, offset information between the two images in the adjacent image pair is determined, wherein the offset information is used to indicate the degree of horizontal offset and the degree of vertical offset between the two images in the adjacent image pair.

2. The image offset determination method according to claim 1, characterized in that, Determining the offset information between two images in the adjacent image pair includes: Based on the phase registration algorithm, the peak correlation matrix of the two images in the adjacent image pair is determined, wherein the horizontal and vertical coordinates of the matrix points in the peak correlation matrix correspond to the horizontal and vertical offsets between the two images in the adjacent image pair, respectively. In the peak correlation matrix, the target peak point is determined; Based on the horizontal and vertical coordinates of all the target peak points in the peak correlation matrix and the positional relationship information between the two images in the adjacent image pair, the candidate offset coordinates of the first image relative to the second image in the adjacent image pair are determined. The target offset coordinates are determined from the candidate offset coordinates to obtain the offset information.

3. The image offset determination method according to claim 2, characterized in that, Determining the target peak point in the peak correlation matrix includes: Based on the preset overlap rate and preset overlap error rate between images, a first allowable range of horizontal coordinates and a second allowable range of vertical coordinates of the peak points in the peak correlation matrix are determined, wherein the preset overlap rate represents the overlap range of the two images in each adjacent image pair; The target peak point is determined within the first and second allowed value ranges of the peak correlation matrix.

4. The image offset determination method according to claim 2, characterized in that, Determining the target peak point in the peak correlation matrix includes: The peak correlation matrix is ​​subjected to mean filtering to obtain the filtered peak correlation matrix; The target peak point is determined from the filtered peak correlation matrix.

5. The image offset determination method according to claim 4, characterized in that, Before performing mean filtering on the peak correlation matrix to obtain the filtered peak correlation matrix, the step of determining the target peak point in the peak correlation matrix further includes: Based on the preset overlap rate and preset overlap error rate between images, a first allowable range of horizontal coordinates and a second allowable range of vertical coordinates of the peak points in the peak correlation matrix are determined, wherein the preset overlap rate represents the overlap range of the two images in each adjacent image pair; The values ​​of all matrix points in the peak correlation matrix that are outside the first and second allowed value ranges are set as target values, wherein the target values ​​include the minimum or average value or 0 of all matrix points in the peak correlation matrix.

6. The image offset determination method according to claim 2, characterized in that, Determining the target peak point in the peak correlation matrix includes: In the peak correlation matrix, a preset number of peak points, whose values ​​decrease sequentially from the maximum, are selected as the target peak points.

7. The image offset determination method according to claim 2, characterized in that, Determining the target peak point in the peak correlation matrix includes: The peak threshold is determined based on the maximum value in the peak correlation matrix. Based on the peak threshold, the target peak point is determined in the peak correlation matrix, wherein the value of the target peak point is greater than or equal to the peak threshold.

8. The image offset determination method according to claim 2, characterized in that, The step of determining the candidate offset coordinates of the first image relative to the second image in the adjacent image pair based on the horizontal and vertical coordinates of all the target peak points in the peak correlation matrix and the positional relationship information between the two images in the adjacent image pair includes: Based on the horizontal and vertical coordinates of all the target peak points in the peak correlation matrix and the positional relationship information between the two images in the adjacent image pair, multiple inferred offset coordinates corresponding to each of the target peak points are determined. Based on a preset overlap rate and a preset overlap error rate between images, a first offset coordinate range for the horizontal offset of the first image relative to the second image in the adjacent image pair and a second offset coordinate range for the vertical offset are determined, wherein the preset overlap rate represents the overlap range of the two images in each adjacent image pair; Of all the predicted offset coordinates, the predicted offset coordinates located within the range of the first offset coordinate and the range of the second offset coordinate are determined as the candidate offset coordinates.

9. The image offset determination method according to any one of claims 2 to 8, characterized in that, The number of candidate offset coordinates is multiple, and the step of determining the target offset coordinate from the candidate offset coordinates to obtain the offset information includes: For each candidate offset coordinate, the image cross-correlation value of the two images in the adjacent image pair at the overlapping position represented by the candidate offset coordinate is calculated to obtain the confidence value corresponding to the candidate offset coordinate; The candidate offset coordinate with the highest corresponding confidence value is determined as the target offset coordinate to obtain the offset information.

10. An image stitching method, characterized in that, include: The image offset determination method as described in any one of claims 1 to 9; The image stitching method further includes: Based on the offset information of each of the adjacent image pairs, the multiple images are stitched together to obtain the overall image of the target object.

11. An image offset determination device, characterized in that, include: The first acquisition module is used to acquire multiple images and positional relationship information between the multiple images, wherein the multiple images include multiple local images of the target object taken at different shooting positions, and the multiple images are used to stitch together to form an overall image of the target object; The first determining module is used to determine adjacent image pairs among the plurality of images based on the positional relationship information, wherein each adjacent image pair includes two images among the plurality of images whose shooting positions are horizontally or vertically adjacent; The second determining module is used to determine the offset information between the two images in each of the adjacent image pairs, wherein the offset information is used to indicate the degree of horizontal offset and the degree of vertical offset between the two images in the adjacent image pair.

12. An image stitching device, characterized in that, include: An image offset determination device, configured to perform the image offset determination method as described in any one of claims 1 to 9; The image stitching device also includes: The stitching module is used to stitch together the multiple images based on the offset information of each of the adjacent image pairs to obtain the overall image of the target object.

13. An electronic device, comprising: Processor and memory, characterized in that, The memory stores computer program instructions, which, when executed by the processor, are used to perform the image offset determination method as described in any one of claims 1 to 9 or the image stitching method as described in claim 10.

14. A storage medium on which program instructions are stored, characterized in that, The program instructions, when executed, are used to perform the image offset determination method as described in any one of claims 1 to 9 or the image stitching method as described in claim 10.

Images