Image alignment method and network device

Abstract

The present disclosure provides an image alignment method and a network device. The method comprises: downsampling a plurality of original images; performing block processing on a first downsampled image and a second downsampled image obtained by the downsampling; determining a region type of each second downsampled block image; and mapping the second downsampled block image to a first original image and a second original image according to the region type. Different alignment feature parameters are used for different region types to align the first original image and the second original image. Thus, the calculation amount of the alignment scheme on the original image can be reduced, the time required for alignment can be shortened, and the overall image alignment efficiency and accuracy can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image alignment method and a network device. Background Technology

[0002] Multi-frame image alignment can broaden the color range, smooth out image noise, and enhance image detail, thus enriching the image content. However, multi-frame image alignment requires ensuring that the captured frames are strictly aligned to avoid ghosting and other issues that negatively impact the fusion quality due to large pixel discrepancies.

[0003] Currently, image content is often used to calculate the image features of the frame to be aligned and the reference frame separately, and then further corrected images to be matched are obtained. However, when relying on image features for image alignment, relatively complex feature descriptions are required to achieve good accuracy, which greatly increases the computational load. At the same time, because this method is highly correlated with image content, the final correction effect is not good. Summary of the Invention

[0004] This disclosure provides an image alignment method and network device that improves the efficiency and accuracy of image alignment by downsampling and segmenting the original image.

[0005] In one aspect, this embodiment provides an image alignment method, the method comprising: dividing a first downsampled image into multiple first downsampled block images and dividing a second downsampled image into multiple second downsampled block images, wherein the first downsampled image is an image obtained by downsampling a first original image and the second downsampled image is an image obtained by downsampling a second original image; determining the region type of each second downsampled block image, wherein different region types correspond to different alignment feature parameters; and determining alignment feature parameters of the first original image and the second original image based on the region type of each second downsampled block image, wherein the alignment feature parameters are used at least to align the first original image and the second original image.

[0006] In embodiments of this disclosure, the method further includes: acquiring a first original image and a second original image; reducing the first original image by a first preset reduction factor to obtain a first downsampled image; and reducing the second original image by a second preset reduction factor based on the size of the first downsampled image to obtain a second downsampled image.

[0007] In embodiments of this disclosure, the number of first downsampled block images is the same as the number of second downsampled block images, and the position of the first downsampled block image in the first downsampled image is the same as the position of the second downsampled block image in the second downsampled image.

[0008] In embodiments of this disclosure, the method further includes: determining a texture richness value for each second downsampled block image; and determining an alignment offset value for the second downsampled block image based on a first downsampled block image corresponding to the second downsampled block image.

[0009] In embodiments of this disclosure, determining the region type of each second downsampled block image includes: determining the region type of the second downsampled block image as a motion region when the alignment offset value of the second downsampled block image is greater than a first motion threshold; or determining the region type of the second downsampled block image as a background-rich region when the alignment offset value of the second downsampled block image is less than or equal to the first motion threshold and the texture richness value of the second downsampled block image is greater than the first texture threshold; or determining the region type of the second downsampled block image as a background-flat region when the alignment offset value of the second downsampled block image is less than or equal to the first motion threshold and the texture richness value of the second downsampled block image is less than the first texture threshold.

[0010] In embodiments of this disclosure, determining alignment feature parameters of the first original image and the second original image based on the region type of each second downsampled block image includes: dividing the first original image into multiple first original block images based on multiple first downsampled block images, wherein the number of first original block images is the same as the number of first downsampled block images, and the position of the first original block images in the first original image is the same as the position of the first downsampled block images in the first downsampled image; dividing the second original image into multiple second original block images based on multiple second downsampled block images, wherein the number of second original block images is the same as the number of second downsampled block images, and the position of the second original block images in the second original image is the same as the position of the second downsampled block images in the second downsampled image; and determining the region type of each first original block image and the second original block image based on the region type of each second downsampled block image to determine the alignment feature parameters of the first original image and the second original image.

[0011] In the embodiments of this disclosure, when the region type is a motion region, no alignment feature points are extracted; when the region type is a flat background region, the number of alignment feature points extracted is reduced; and when the region type is a rich background region, the number of alignment feature points extracted is increased.

[0012] In embodiments of this disclosure, the method further includes: extracting alignment feature points for each first original block image and each second original block image based on the number of alignment feature points extracted; determining an alignment matrix based on the alignment feature points for each first original block image and each second original block image; and aligning the first original image and the second original image based on the alignment matrix.

[0013] In embodiments of this disclosure, the method further includes: storing and outputting the aligned first original image and the second original image.

[0014] On the other hand, embodiments of this disclosure provide a network device applied to an image alignment method. The device includes: a partitioning module, configured to partition a first downsampled image into multiple first downsampled block images and a second downsampled image into multiple second downsampled block images, wherein the first downsampled image is an image obtained by downsampling a first original image, and the second downsampled image is an image obtained by downsampling a second original image; a first determining module, configured to determine the region type of each second downsampled block image, wherein different region types correspond to different alignment feature parameters; and a second determining module, configured to determine the alignment feature parameters of the first original image and the second original image based on the region type of each second downsampled block image, wherein the alignment feature parameters are used at least to align the first original image and the second original image.

[0015] In another aspect, embodiments of this disclosure provide a non-transitory computer-readable storage medium for storing computer-readable instructions that, when executed by a processor, cause the processor to perform the aforementioned image alignment method.

[0016] In another aspect, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the above-described image alignment method. Attached Figure Description

[0017] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 The schematic diagram illustrates an environmental application according to an embodiment of the present disclosure.

[0019] Figure 2 A flowchart illustrating an image alignment method according to an embodiment of the present disclosure is shown.

[0020] Figure 3 A schematic diagram illustrating the determination of alignment offset values ​​according to an embodiment of the present disclosure is provided.

[0021] Figure 4 A flowchart illustrating another image alignment method according to an embodiment of the present disclosure is shown.

[0022] Figure 5 A block diagram of a network device according to an embodiment of the present disclosure is shown schematically.

[0023] Figure 6 A block diagram illustrating a non-transitory computer-readable storage medium according to an embodiment of the present disclosure is shown.

[0024] Figure 7 A block diagram illustrating a computer program product according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this disclosure more apparent, exemplary embodiments according to this disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments of this disclosure. It should be understood that this disclosure is not limited to the exemplary embodiments described herein.

[0026] A single frame captured by a single camera typically reflects only a limited amount of content within the same scene, at the same exposure level, and from the same viewpoint. To broaden the color gamut, smooth out image noise, enhance image detail, and enrich image content, multi-frame image fusion techniques have been developed, including image enhancement, image filtering, image super-resolution, and image stitching. Since a single camera cannot capture multiple frames simultaneously, image alignment techniques are necessary to ensure strict alignment of the captured frames and prevent ghosting and other issues that could affect the fusion quality due to large pixel discrepancies. Therefore, the quality of multi-frame image alignment directly impacts the quality of the fusion process.

[0027] Currently, image alignment often involves calculating image statistics or features between the frame to be aligned and the reference frame based on image content. The deviation in image statistics is then calculated, or image features are matched to obtain the corrected image to be aligned. However, relying on image features for image alignment requires complex feature descriptions to achieve good accuracy, significantly increasing computational complexity. Furthermore, because this method is highly correlated with image content, it often suffers from poor correction results, particularly between frames with local motion and large parallax, due to significant deviations in image statistics estimation and poor image feature matching. This limits its applicability.

[0028] Based on this, embodiments of this disclosure provide an image alignment method that effectively reduces the computational load of image feature alignment methods by downsampling a first original image and a second original image. Furthermore, the first and second downsampled images are divided into blocks, and the region type of each block is calculated. Different alignment feature parameters are used to align the first and second original images using different region types. This improves the accuracy and efficiency of alignment.

[0029] The following section will combine... Figures 1-4 A detailed description of an image alignment method according to an embodiment of this disclosure is provided.

[0030] Figure 1 The schematic diagram illustrates an environmental application according to an embodiment of the present disclosure.

[0031] like Figure 1 As shown, computing device 101 can align multiple frames of images that need to be aligned. Computing device 101 can be, for example, a mobile terminal such as a smartphone or tablet, or a network device such as a server. For instance, if computing device 101 is a smartphone, the smartphone can obtain multiple images taken by a single camera from its local storage unit and display the aligned images to the user through its display device. As another example, if computing device 101 is a server, the server can obtain multiple images to be aligned from other network devices or local storage units and transmit the aligned images to the user's mobile terminal.

[0032] Figure 2 A flowchart illustrating an image alignment method according to an embodiment of the present disclosure is shown.

[0033] like Figure 2 As shown, in step S201, the first downsampled image is divided into multiple first downsampled block images and the second downsampled image is divided into multiple second downsampled block images. The first downsampled image is the image obtained after downsampling the first original image, and the second downsampled image is the image obtained after downsampling the second original image.

[0034] In embodiments of this disclosure, the first original image can be any frame among multiple frames that need to be aligned, and the second original image can be a frame following the first original image among the multiple frames. The first downsampled image can be a scaled-down version of the first original image, with a size and resolution smaller than the first original image. The second downsampled image can be a scaled-down version of the second original image, with a size and resolution smaller than the second original image.

[0035] The computing device can divide the first downsampled image and the second downsampled image into multiple non-overlapping first downsampled block images and second downsampled block images according to a certain size. Dividing the first downsampled image and the second downsampled image into blocks is beneficial for subsequent block extraction of alignment features, thereby improving alignment accuracy.

[0036] The size of each first downsampled block image can be the same or different, and the size of each second downsampled block image can also be the same or different. Preferably, the size of each first downsampled block image and the size of each second downsampled block image are approximately the same. It is understood that the size of each first downsampled block image is approximately the same because in some cases, it is impossible to divide a first downsampled image into several identical first downsampled block images, and similarly, it is impossible to divide a second downsampled block image into several identical second downsampled block images.

[0037] S202. Determine the region type of each second downsampled block image. Different region types correspond to different alignment feature parameters.

[0038] In embodiments of this disclosure, the computing device can determine the region type of each second downsampled block image. The region type can be, for example, background, motion, face, animal, etc. The region type can represent the feature attributes of the second downsampled block image and can be used to provide differentiation guidance for subsequent determination of alignment feature parameters.

[0039] Different region types can employ different alignment feature parameters, which can include extraction conditions such as the number of alignment feature points extracted, the extraction threshold for alignment feature points, and the extraction criteria for alignment feature points. For example, for a second downsampled block image of a face, a larger number of alignment feature points can be extracted, along with a higher extraction criteria, thereby achieving accurate face alignment. Conversely, for a second downsampled block image of a background, a relatively smaller number of alignment feature points can be extracted to achieve background alignment.

[0040] Different alignment feature parameters can be used for different region types, allowing for precise matching of region characteristics and improved alignment accuracy. Simultaneously, while maintaining alignment accuracy, computational efficiency can be improved, reducing unnecessary computation.

[0041] S203. Based on the region type of each second downsampled block image, determine the alignment feature parameters of the first original image and the second original image. The alignment feature parameters are used at least to align the first original image and the second original image.

[0042] In embodiments of this disclosure, when the region type of each second downsampled block image is determined, the computing device can divide the first original image and the second original image into a plurality of corresponding original block images based on the position and size of each second downsampled block image. Further, the region types of the plurality of corresponding original block images in the first original image and the second original image are determined based on the region type of each second downsampled block image.

[0043] For example, there are four second downsampled block images, all of the same size. Based on the position and size of each second downsampled block image, the first original image is divided into four first original block images, and the second original image is also divided into four second original block images. All four first original block images are of the same size, and their arrangement is the same as that of the four second downsampled block images. Furthermore, based on the region type of each second downsampled block image, the region type of each of the four first original block images is determined. For example, the region type of the first original block image located at position A is the same as the region type of the second downsampled block image located at position A.

[0044] In the embodiments of this disclosure, when the region type is determined, the alignment feature parameters corresponding to the region type can be determined, thereby enabling different alignment strategies to be adopted for different block images, so as to improve alignment efficiency while ensuring alignment effect.

[0045] In embodiments of this disclosure, such as Figure 2 The method further includes: acquiring a first original image and a second original image; reducing the first original image by a first preset reduction factor to obtain a first downsampled image; and reducing the second original image by a second preset reduction factor based on the size of the first downsampled image to obtain a second downsampled image.

[0046] The computing device can acquire multiple original images that need to be aligned, such as a first original image and a second original image. Further, the computing device can reduce the size of the first original image according to a first preset reduction ratio, and obtain a first downsampled image after reducing the size of the first original image. For example, a 1080p image can be downsampled by 1 / 4 or 1 / 16. The first preset reduction ratio can be determined based on the size of the first original image. For example, if the first original image is a 1080p image, it can be downsampled by 1 / 4; if the first original image is a 4k image, it can be downsampled by 1 / 16, to reduce the first original image to a size suitable for processing.

[0047] Furthermore, the second original image can be reduced in size according to the size of the first downsampled image and the second preset reduction factor, so that the size of the second downsampled image is equal to or similar to the size of the first downsampled image.

[0048] In some embodiments, if the size of the first original image is the same as the size of the second original image, then the first preset reduction ratio is the same as the second preset reduction ratio, so that the size of the first downsampled image is equal to the size of the second downsampled image. In some embodiments, if the size of the first original image is similar to the size of the second original image, then the second original image can be reduced to a size similar to the first downsampled image based on the size of the first downsampled image.

[0049] In embodiments of this disclosure, after acquiring the first original image and the second original image, the computing device can further preprocess the first original image and the second original image. For example, it can adjust the underlying noise and brightness levels of the first original image and the second original image to make their underlying noise and brightness levels consistent, thereby improving alignment accuracy.

[0050] According to the embodiments of this disclosure, the first original image and the second original image are subjected to processing, which can ensure that the subsequent processes such as determining the region type of the block image and extracting aligned feature points are less time-consuming and more stable.

[0051] In embodiments of this disclosure, the number of first downsampled block images is the same as the number of second downsampled block images, and the position of the first downsampled block image in the first downsampled image is the same as the position of the second downsampled block image in the second downsampled image.

[0052] In step S201 above, when dividing the first downsampled image into multiple first downsampled block images, it can be divided into multiple non-overlapping small blocks according to a first fixed size, for example, each block being 32×32 pixels in size. When dividing the second downsampled image into multiple second downsampled block images, it can be divided into multiple non-overlapping small blocks according to a second fixed size, which is similar to the first fixed size, thereby dividing the second downsampled image into multiple second downsampled block images, and the size of each second downsampled block image is similar to the size of each first downsampled block image.

[0053] In some embodiments, when dividing multiple first downsampled block images and multiple first downsampled block images, the same segmentation rules need to be used. The same segmentation rules can mean that the pixel size of each small block is the same or similar, and the total number of rows and columns is exactly the same. For example, if both the first downsampled image and the second downsampled image are 960×540 images, then they are divided into 32×30 pixel blocks, totaling 30 columns × 18 rows. After segmentation, the size of both the first downsampled block image and the second downsampled block image is 32×30 pixels, and the first downsampled block image in row m and column n in the multiple first downsampled block images corresponds to the second downsampled block image in row m and column n in the multiple second downsampled block images.

[0054] It should be noted that for first and second downsampled images of different sizes, the pixel size of each small block is similar when using the same segmentation rule. This is because the first and second original images are not exactly the same. After downsampling, the size of the first and second downsampled images remains within a certain range. Therefore, while ensuring that the total number of rows and columns is exactly the same, the pixel size of each small block is not exactly equal, but rather quite similar.

[0055] According to embodiments of this disclosure, the first downsampled image and the second downsampled image are sampled using the exact same block rules, ensuring that each first downsampled block image has a corresponding second downsampled block image, thus providing a basis for aligning the first original image and the second original image.

[0056] In embodiments of this disclosure, such as Figure 2 The method also includes: determining the texture richness value of each second downsampled block image; and determining the alignment offset value of the second downsampled block image based on the first downsampled block image corresponding to the second downsampled block image.

[0057] Texture richness values ​​can be used to measure the texture complexity of a second downsampled block image; a higher texture richness value indicates richer texture, while a lower value indicates flatter texture. Computing devices can estimate the texture richness of image blocks using methods such as image block variance.

[0058] The alignment offset value measures the inter-frame motion offset between the second downsampled block image and the first and second downsampled images, and is a pixel-level offset distance. When determining the alignment offset value of the second downsampled block image, the image block matching difference value can be calculated within a preset search area around the first downsampled block image, centered on the first downsampled block image. The image block matching difference value calculation includes, but is not limited to, using the pixel difference SAD statistical value to evaluate block similarity. The smaller the SAD value, the smaller the pixel difference between the candidate offset position image and the image to be matched, and the higher the similarity. For example, if the preset search area around the first downsampled block image includes multiple candidate offset positions, the SAD value of the second downsampled block image and each of the multiple candidate offset positions can be calculated sequentially. Then, the candidate offset position with the smallest SAD value is selected as the target offset position of the second downsampled block image, and this target offset position is used as the alignment offset value of the second downsampled block image.

[0059] Figure 3 This is a schematic diagram illustrating the determination of alignment offset values ​​according to an embodiment of this disclosure. Figure 3 In the image (a), the first downsampled image 301 is divided into multiple first downsampled block images 3011. Figure 3 In (b), the second downsampled image 302 is divided into multiple second downsampled block images 3021, and each first downsampled block image 3011 has a corresponding second downsampled block image 3021. The number of first downsampled block images 3011 is the same as the number of second downsampled block images 3021.

[0060] like Figure 3 As shown in (b), when calculating the alignment offset value of the second downsampled block image 3021 located in the 3rd row and 4th column, matching is performed within a search area 3012 surrounding the first downsampled block image 3011 located in the 3rd row and 4th column. The search area 3012 surrounding the first downsampled block image 3011 may include multiple candidate offset positions 3013, and the SAD value between each candidate offset position 3013 and the second downsampled block image 3021 is calculated. The candidate offset position 3013 with the smallest SAD value is selected as the target offset position of the second downsampled block image 3021, and this target offset position is used as the alignment offset value of the second downsampled block image 3021.

[0061] In embodiments of this disclosure, S202 includes: determining the region type of the second downsampled block image as a motion region when the alignment offset value of the second downsampled block image is greater than a first motion threshold; or determining the region type of the second downsampled block image as a background-rich region when the alignment offset value of the second downsampled block image is less than or equal to the first motion threshold and the texture richness value of the second downsampled block image is greater than the first texture threshold; or determining the region type of the second downsampled block image as a background-flat region when the alignment offset value of the second downsampled block image is less than or equal to the first motion threshold and the texture richness value of the second downsampled block image is less than the first texture threshold.

[0062] The computing device can set a first motion threshold and a first texture threshold. The first motion threshold can be used to measure the magnitude of the alignment offset value. When the alignment offset value is greater than the first motion threshold, it is determined that the block image belongs to a moving region, and the moving region does not need to be forcibly aligned. When the alignment offset value is less than or equal to the first motion threshold, it is determined that the block image does not belong to a moving region and alignment is required.

[0063] When the alignment offset value is less than or equal to the first motion threshold, the block image needs to be re-evaluated using the first texture threshold. The first texture threshold measures the complexity of the texture within the block image. When the texture richness value is greater than the first texture threshold, the block image is determined to belong to a region with a rich background, requiring the extraction of more feature points for alignment. When the texture richness value is less than or equal to the first texture threshold, the block image is determined to belong to a region with a flat background, allowing for alignment by extracting a smaller number of feature points.

[0064] In some embodiments, after determining the region type of each second downsampled block image, the region type can be mapped to a numerical value. For example, a numerical mask can be determined for each region type, and then the corresponding numerical mask can be filled into each second downsampled block image. For instance, the numerical mask for a motion region is 255. If the second downsampled block image in the i-th row and j-th column belongs to a motion region, then the numerical mask corresponding to that second downsampled block image in the i-th row and j-th column is 255. After determining the numerical mask for each second downsampled block image, all second downsampled block images can be stitched together in the row and column order of the blocks to obtain a numerical mask image with the same size as the second downsampled images and the same block division.

[0065] The numerical mask obtained using the above method has the same size as the second downsampled image. Further, this numerical mask can be upsampled, that is, enlarged to the size of the second original image. For example, bilinear interpolation can be used to enlarge the numerical mask to the size of the second original image, aligning it with the size of the second original image for easier subsequent operations.

[0066] Figure 4A flowchart illustrating another image alignment method according to an embodiment of the present disclosure is shown.

[0067] like Figure 4 As shown, S203 above includes:

[0068] S401. Based on multiple first downsampled block images, the first original image is divided into multiple first original block images. The number of first original block images is the same as the number of first downsampled block images, and the position of the first original block image in the first original image is the same as the position of the first downsampled block image in the first downsampled image.

[0069] In embodiments of this disclosure, a computing device can divide a first original image into multiple first original block images based on the positional relationship of multiple first downsampled block images and the same segmentation rules. For example, if the number of first downsampled block images is 30 rows × 10 columns, then the number of first original block images is also 30 rows × 10 columns, and the first original block images correspond one-to-one with the first downsampled block images.

[0070] S402. Based on multiple second downsampled block images, the second original image is divided into multiple second original block images. The number of second original block images is the same as the number of second downsampled block images, and the position of the second original block image in the second original image is the same as the position of the second downsampled block image in the second downsampled image.

[0071] Similar to the first original block image, the computing device can divide the second original image into multiple second original block images based on the same block division rules, according to the positional relationship of multiple second downsampled block images. The second original block images correspond one-to-one with the second downsampled block images.

[0072] S403. Based on the region type of each second downsampled block image, determine the region type of each first original block image and second original block image to determine the alignment feature parameters of the first original image and the second original image.

[0073] In the embodiments of this disclosure, since the first original block image adopts the same segmentation rule as the first downsampled block image, and the second original block image adopts the same segmentation rule as the second downsampled block image, and since the segmentation rules of the first downsampled block image and the second downsampled block image are the same, there is a one-to-one correspondence between the first original block image and the second original block image, and each also has a one-to-one correspondence with the second downsampled block image. In this case, the region type of each first original block image and the region type of each second original block image can be determined using the region type of each second downsampled block image.

[0074] In some embodiments, the region type of each first original block image and the region type of each second original block image can also be determined using the upsampled numerical mask image. It is understood that the upsampled numerical mask image also has multiple block images, and each block image has a one-to-one correspondence with both the first and second original block images.

[0075] Furthermore, after determining the region type of each first original block image and each second original block image, different alignment feature parameters are configured for each first original block image and each second original block image according to the region type.

[0076] According to embodiments of this disclosure, by dividing the original image into multiple original block images and determining the region type of each original block image based on the region type of the second downsampled block image, the alignment feature parameters of different regions of the original image can be determined more precisely, thereby improving alignment accuracy.

[0077] In the embodiments of this disclosure, when the region type is a motion region, no alignment feature points are extracted; when the region type is a flat background region, the number of alignment feature points extracted is reduced; and when the region type is a rich background region, the number of alignment feature points extracted is increased.

[0078] For moving regions, image features may not need to be extracted. For regions with flat backgrounds, feature extraction restrictions can be relaxed to reduce the number of aligned feature points extracted. Conversely, for regions with rich backgrounds, feature extraction restrictions can be tightened to increase the number of aligned feature points extracted. Extraction restrictions may include, for example, an extraction threshold. For tightened restrictions, the extraction threshold can be increased, ensuring that only aligned feature points with extremely high response values ​​and strong eigenvalues ​​are extracted.

[0079] In embodiments of this disclosure, such as Figure 4 The method further includes: extracting alignment feature points for each first original block image and each second original block image based on the number of alignment feature points extracted; determining an alignment matrix based on the alignment feature points for each first original block image and each second original block image; and aligning the first original image and the second original image based on the alignment matrix.

[0080] Based on the alignment feature parameters of each first original block image and each second original block image, alignment features, such as ORB features, are extracted for each first original block image and each second original block image to obtain the corresponding alignment feature points. Further, the alignment feature points of the first original block image are matched with the alignment feature points of the second original block image. For example, the similarity between the descriptor of an alignment feature point in the first original block image and the descriptors of all alignment feature points in the second original block image is calculated, and the pair with the highest similarity is the matching pair. Then, the computing device can also convert the coordinates of local alignment feature points in the first original block image to global coordinates of the first original image, and the coordinates of local alignment feature points in the second original block image to global coordinates of the second original image. Based on the global coordinates and the alignment feature point matching pairs, global alignment feature point matching pairs are obtained.

[0081] After matching the alignment feature points, an alignment matrix can be calculated using the matching pairs, preferably global alignment feature point matching pairs, to establish a mapping relationship between the alignment feature point coordinates of the second original image and the alignment feature point coordinates of the first original image. Then, the calculated alignment matrix is ​​applied to every pixel of the second original image to perform a global geometric transformation, ensuring that the transformed second original image and the first original image are perfectly matched in spatial position, angle, scale, and perspective, thus achieving precise overlap between the two images.

[0082] In embodiments of this disclosure, such as Figure 2 The method also includes storing and outputting the aligned first and second original images. The computing device can store the aligned first and second original images and send them to a terminal device, such as a mobile terminal used by a user.

[0083] Figure 5 A block diagram of a network device according to an embodiment of the present disclosure is schematically illustrated; like Figure 5 As shown, the network device 500 applied to the image alignment method includes a segmentation module 501, a first determination module 502, and a second determination module 503.

[0084] The segmentation module 501 is used to divide the first downsampled image into multiple first downsampled block images and the second downsampled image into multiple second downsampled block images. The first downsampled image is the image obtained after downsampling the first original image, and the second downsampled image is the image obtained after downsampling the second original image. The first determination module 502 is used to determine the region type of each second downsampled block image. Different region types correspond to different alignment feature parameters. The second determination module 503 is used to determine the alignment feature parameters of the first original image and the second original image based on the region type of each second downsampled block image. The alignment feature parameters are used to align the first original image and the second original image at least.

[0085] Figure 6 A block diagram illustrating a non-transitory computer-readable storage medium according to an embodiment of the present disclosure is shown schematically. like Figure 6 As shown, a non-transitory computer-readable storage medium 600 of this disclosure embodiment is used to store computer-readable instructions 601, which, when executed by a processor, cause the processor to perform the image alignment method as described above.

[0086] Figure 7 A block diagram illustrating a computer program product according to an embodiment of the present disclosure is shown schematically. like Figure 7 As shown, a computer program product 700 according to an embodiment of the present disclosure includes a computer program 701, which, when executed by a processor, implements the image alignment method as described above.

[0087] The above description, with reference to the accompanying drawings, illustrates an image alignment method and network device according to embodiments of the present disclosure. The method involves downsampling multiple frames of original images to reduce them to the same or similar size. Then, the first and second downsampled images obtained from the downsampling process are divided into blocks using the same block division rules, resulting in first downsampled block images and second downsampled block images with the same number of blocks and approximately the same block size. Further, the region type of each second downsampled block image is determined by determining its texture richness value and alignment offset value. Then, the region type of each second downsampled block image is mapped to the first and second original block images, and different alignment feature parameters are used for different region types to extract alignment feature points from the first and second original block images. This reduces the computational load of subsequent image feature-based alignment schemes on the original images, shortens the alignment time, and improves overall image alignment efficiency and accuracy by configuring different feature extraction parameters for different regions.

[0088] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0089] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0090] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0091] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0092] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described above can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0093] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0094] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. An image alignment method, characterized in that, The method includes: The first downsampled image is divided into multiple first downsampled block images and the second downsampled image is divided into multiple second downsampled block images. The first downsampled image is the image obtained by downsampling the first original image, and the second downsampled image is the image obtained by downsampling the second original image. Determine the region type of each second downsampled block image, with different region types corresponding to different alignment feature parameters; and Based on the region type of each second downsampled block image, the alignment feature parameters of the first original image and the second original image are determined, and the alignment feature parameters are used at least to align the first original image and the second original image.

2. The image alignment method according to claim 1, characterized in that, The method further includes: Obtain the first original image and the second original image; The first original image is reduced by a first preset reduction factor to obtain the first downsampled image; and Based on the size of the first downsampled image, the second original image is reduced by a second preset reduction factor to obtain the second downsampled image.

3. The image alignment method according to claim 1, characterized in that, The number of the first downsampled block images is the same as the number of the second downsampled block images, and the position of the first downsampled block image in the first downsampled image is the same as the position of the second downsampled block image in the second downsampled image.

4. The image alignment method according to claim 3, characterized in that, The method further includes: Determine the texture richness value for each of the second downsampled block images; and Based on the first downsampled block image corresponding to the second downsampled block image, the alignment offset value of the second downsampled block image is determined.

5. The image alignment method according to claim 4, characterized in that, Determining the region type of each second downsampled block image includes: When the alignment offset value of the second downsampled block image is greater than the first motion threshold, the region type of the second downsampled block image is determined to be a motion region; or When the alignment offset value of the second downsampled block image is less than or equal to the first motion threshold, and the texture richness value of the second downsampled block image is greater than the first texture threshold, the region type of the second downsampled block image is determined to be a background rich region; or When the alignment offset value of the second downsampled block image is less than or equal to the first motion threshold, and the texture richness value of the second downsampled block image is less than the first texture threshold, the region type of the second downsampled block image is determined to be a flat background region.

6. The image alignment method according to claim 5, characterized in that, The step of determining the alignment feature parameters of the first original image and the second original image based on the region type of each second downsampled block image includes: Based on multiple first downsampled block images, the first original image is divided into multiple first original block images. The number of first original block images is the same as the number of first downsampled block images, and the position of the first original block image in the first original image is the same as the position of the first downsampled block image in the first downsampled image. Based on multiple second downsampled block images, the second original image is divided into multiple second original block images, the number of second original block images being the same as the number of second downsampled block images, and the positions of the second original block images in the second original image being the same as the positions of the second downsampled block images in the second downsampled image; and Based on the region type of each second downsampled block image, the region type of each first original block image and the second original block image is determined to determine the alignment feature parameters of the first original image and the second original image.

7. The image alignment method according to claim 6, characterized in that, When the region type is the motion region, no alignment feature points are extracted; when the region type is the flat background region, the number of alignment feature points extracted is reduced; when the region type is the rich background region, the number of alignment feature points extracted is increased.

8. The image alignment method according to claim 7, characterized in that, The method further includes: Based on the number of alignment feature points extracted, alignment feature points are extracted for each of the first original block images and each of the second original block images; Based on the alignment feature points of each first original block image and each second original block image, an alignment matrix is ​​determined; and Based on the alignment matrix, the first original image and the second original image are aligned.

9. The image alignment method according to claim 1, characterized in that, The method further includes: storing and outputting the aligned first original image and the second original image.

10. A network device applied to an image alignment method, characterized in that, The device includes: The partitioning module is used to partition a first downsampled image into multiple first downsampled block images and a second downsampled image into multiple second downsampled block images. The first downsampled image is the image obtained by downsampling a first original image, and the second downsampled image is the image obtained by downsampling a second original image. A first determining module is configured to determine the region type of each second downsampled block image, wherein different region types correspond to different alignment feature parameters; and The second determining module is used to determine the alignment feature parameters of the first original image and the second original image based on the region type of each second downsampled block image, wherein the alignment feature parameters are used to align the first original image and the second original image at least.

Images