Image stitching method and device, computer device and storage medium
By using a feature point stitching algorithm, a pyramid is constructed by sliding matching and Gaussian convolution between the template image and the input image to detect feature points. This solves the problems of high computational cost and poor registration effect of existing stitching algorithms, and achieves efficient and robust image stitching.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 王宇鸿
- Filing Date
- 2020-06-29
- Publication Date
- 2026-07-31
AI Technical Summary
Existing splicing algorithms involve large amounts of computation, high time costs, and are prone to program interruption due to division operations, and the registration effect is poor.
A feature point stitching algorithm is adopted. The similarity is calculated by sliding matching between the template image and the input image. The minimum value is selected to determine the overlapping area. The image is enlarged and Gaussian convolution is performed to construct a pyramid. The position, scale and orientation of feature points are detected and the image is stitched together.
It improves the accuracy and stability of image stitching, reduces computational load, and enhances robustness to changes in lighting and viewing angle shifts, meeting the application needs of the medical field.
Smart Images
Figure CN113935896B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image stitching, and more particularly to an image stitching method, apparatus, computer equipment, and storage medium. Background Technology
[0002] Existing stitching algorithms are mainly based on area and ratio. However, area-based algorithms suffer from excessive computation, leading to high time costs. While ratio-based algorithms reduce time costs compared to area-based algorithms, they still have significant time costs and often cause program interruptions due to the numerous divisions required. Since the computer calculates the red, green, and blue (RGB) values of the corresponding image pixels during division, and RGB values can be zero, special handling is required when the denominator is zero; otherwise, program execution will be interrupted. Therefore, stitching algorithms are not ideal in terms of both computational complexity and program implementation. Summary of the Invention
[0003] The purpose of this application is to provide an image stitching method, apparatus, computer equipment, and storage medium to solve the technical problems of excessive computational load and poor registration effect in image stitching.
[0004] To address the aforementioned technical problems, this application provides an image stitching method, employing the following technical solution:
[0005] Obtain the first input image and the second input image;
[0006] The template image is used for sliding matching with the first input image, and multiple similarities are calculated during the matching process. The template image is part or all of the second input image.
[0007] The minimum value of the multiple similarities is selected to obtain the target similarity, and the target overlapping region is obtained based on the target similarity;
[0008] The target features are obtained by determining the features of the first input image and the features of the second input image through the target overlapping region;
[0009] Based on the target features, the first input image and the second input image are stitched together.
[0010] Furthermore, the step of determining the features of the first input image and the second input image through the target overlapping region to obtain the target features specifically includes:
[0011] Enlarge the first input image and the second input image to obtain the corresponding transformed image;
[0012] The transformed image is obtained by multiple Gaussian convolutions to obtain corresponding multi-layer images;
[0013] A Gaussian difference pyramid is constructed using the multi-layered images;
[0014] The Gaussian difference pyramid used for feature detection yields the location, scale, and orientation of the feature points.
[0015] The position, scale, and orientation of the feature points are used as features of the first input image and the second input image to obtain the target features.
[0016] Furthermore, the step of transforming the image through multiple Gaussian convolutions to obtain the corresponding multi-layer image specifically includes:
[0017] pass The image is transformed by Gaussian convolution, where x, y are pixel coordinates, x0, y0 are constants, and σ is a smoothing coefficient.
[0018] Furthermore, the step of constructing a Gaussian pyramid from the multi-layered images specifically includes:
[0019] The transformed image is used as the first layer of the Gaussian pyramid, and the corresponding i-th layer image is used as the i-th layer of the Gaussian pyramid. An n-layer Gaussian pyramid is constructed, where i is any integer between 2 and n.
[0020] The difference between the next layer of the Gaussian pyramid in the same group and the current layer of the Gaussian pyramid is calculated to obtain the Gaussian difference pyramid of layer n-1.
[0021] Furthermore, the step of stitching the first input image and the second input image together based on the target features specifically includes:
[0022] The first input image and the second input image are spatially superimposed to obtain the superimposed image;
[0023] Obtain the target features of the overlapping region, wherein the target features include at least the width parameter of the overlapping region;
[0024] The superimposed image is then blended according to the width parameter of the overlapping region.
[0025] Furthermore, before the step of spatially superimposing the first input image and the template image to obtain the superimposed image, the method further includes:
[0026] Obtain the first feature point and the second feature point in the overlapping region, where the second feature point is the smallest neighborhood point of the first feature point;
[0027] Calculate the Euclidean distance between the first feature point and the second feature point to obtain the first Euclidean distance;
[0028] Obtain the third feature point, which is the second smallest neighborhood point of the first feature point;
[0029] Calculate the Euclidean distance between the first feature point and the third feature point to obtain the second Euclidean distance;
[0030] If the Euclidean distance of the first Euclidean distance and the Euclidean distance of the second Euclidean distance are greater than the threshold, then the first feature point is deleted.
[0031] Furthermore, after the steps of acquiring the first input image and the second input image, the method further includes:
[0032] The first input image is converted into grayscale values and mapped to the corresponding grayscale values using a monotonic grayscale mapping curve to obtain a grayscale image.
[0033] The grayscale image is denoised using a Gaussian blur to obtain a filtered image.
[0034] The filtered image is smoothed using convolution operations to obtain the complete image;
[0035] The complete image is Fourier transformed to obtain a frequency domain image;
[0036] The frequency domain image is processed by differentiation.
[0037] To address the aforementioned technical problems, this application also provides an image stitching device, which employs the following technical solution:
[0038] The acquisition module is used to acquire the first input image and the second input image;
[0039] The matching module is used to perform sliding matching between the template image and the first input image, and calculate multiple similarities during the matching process. The template image is part or all of the second input image.
[0040] The overlapping region calculation module is used to select the minimum value of the multiple similarities to obtain the target similarity, and to obtain the target overlapping region based on the target similarity;
[0041] The target feature acquisition module is used to determine the features of the first input image and the features of the second input image through the target overlapping region to obtain the target features;
[0042] The stitching module stitches the first input image and the second input image together based on the target features.
[0043] Furthermore, the matching module is also used for:
[0044] Enlarge the first input image and the second input image to obtain the corresponding transformed image;
[0045] The transformed image is obtained by multiple Gaussian convolutions to obtain corresponding multi-layer images;
[0046] A Gaussian difference pyramid is constructed using the multi-layered images;
[0047] The Gaussian difference pyramid used for feature detection yields the location, scale, and orientation of the feature points.
[0048] The position, scale, and orientation of the feature points are used as features of the first input image and the second input image to obtain the target features.
[0049] Furthermore, the matching module is also used for:
[0050] pass The image is transformed by Gaussian convolution, where x, y are pixel coordinates, x0, y0 are constants, and σ is a smoothing coefficient.
[0051] Furthermore, the matching module is also used for:
[0052] The transformed image is used as the first layer of the Gaussian pyramid, and the corresponding i-th layer image is used as the i-th layer of the Gaussian pyramid. An n-layer Gaussian pyramid is constructed, where i is any integer between 2 and n.
[0053] The difference between the next layer of the Gaussian pyramid in the same group and the current layer of the Gaussian pyramid is calculated to obtain the Gaussian difference pyramid of layer n-1.
[0054] Furthermore, the splicing module is also used for:
[0055] The first input image and the second input image are spatially superimposed to obtain the superimposed image;
[0056] Obtain the target features of the overlapping region, wherein the target features include at least the width parameter of the overlapping region;
[0057] The superimposed image is then blended according to the width parameter of the overlapping region.
[0058] Furthermore, the image stitching device also includes a threshold calculation module, which is further used for:
[0059] Obtain the first feature point and the second feature point in the overlapping region, where the second feature point is the smallest neighborhood point of the first feature point;
[0060] Calculate the Euclidean distance between the first feature point and the second feature point to obtain the first Euclidean distance;
[0061] Obtain the third feature point, which is the second smallest neighborhood point of the first feature point;
[0062] Calculate the Euclidean distance between the first feature point and the third feature point to obtain the second Euclidean distance;
[0063] If the Euclidean distance of the first Euclidean distance and the Euclidean distance of the second Euclidean distance are greater than the threshold, then the first feature point is deleted.
[0064] Furthermore, the image stitching device also includes a frequency domain analysis module, which is further used for:
[0065] The first input image is converted into grayscale values and mapped to the corresponding grayscale values using a monotonic grayscale mapping curve to obtain a grayscale image.
[0066] The grayscale image is denoised using a Gaussian blur to obtain a filtered image.
[0067] The filtered image is smoothed using convolution operations to obtain the complete image;
[0068] The complete image is Fourier transformed to obtain a frequency domain image;
[0069] The frequency domain image is processed by differentiation.
[0070] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:
[0071] A computer device includes at least one connected processor, memory, and input / output unit, wherein the memory is used to store program code, and the processor is used to call the program code in the memory to execute the steps of the image stitching method described above.
[0072] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:
[0073] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the image stitching method described above.
[0074] Compared with the prior art, the embodiments of this application have the following main advantages:
[0075] This application proposes a feature point stitching algorithm for stitching morphological images. It eliminates seams in the stitched images and exhibits good robustness to changes in illumination and viewpoint shifts. It meets image standards for practical medical applications and improves the accuracy and stability of image registration compared to previous methods. It also increases the matching speed and the probability of correct matching. Attached Figure Description
[0076] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0077] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;
[0078] Figure 2 A flowchart of an embodiment of the image stitching method according to this application;
[0079] Figure 3 This is a schematic diagram of the structure of an embodiment of the image stitching device according to this application;
[0080] Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation
[0081] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.
[0082] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0083] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
[0084] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.
[0085] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.
[0086] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.
[0087] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.
[0088] It should be noted that the image stitching method provided in the embodiments of this application is generally executed by a server / terminal device, and correspondingly, the image stitching device is generally set in the server / terminal device.
[0089] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.
[0090] Continue to refer to Figure 2 A flowchart illustrating an embodiment of the image stitching method according to this application is shown. The image stitching method includes the following steps:
[0091] Step 201: Obtain the first input image and the second input image.
[0092] In this embodiment, the first input image and the second input image are images with the same pixel resolution, and the image stitching method is run on an electronic device (e.g., Figure 1 The server / terminal device shown can receive user requests for calibration via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultrawideband) connections, and other currently known or future wireless connection methods.
[0093] In this embodiment, since image overlap often occurs in edge regions, a template factor is determined locally at the edge of the first image (the size of the template factor varies depending on the image size; theoretically, a larger template results in higher matching accuracy, but also increases computational complexity). The value of K is generally not zero. The template grayscale value information is recorded, where K represents the similarity of the match.
[0094] Step 202: Perform sliding matching between the template image and the first input image, and calculate multiple similarities during the matching process. The template image is part or all of the second input image.
[0095] In this embodiment, a template image is slid across the first input image, and a K value is calculated for each location it passes. The k value is similar to variance; the smaller the k value, the greater the similarity between the template and the target region. The algorithm employs... The least squares difference is used to measure the similarity of the matches. Here, B(x,y) represents the grayscale information of each region in the image, while A(x,y) represents the grayscale information of the selected template image.
[0096] Step 203: Select the minimum value of the multiple similarities to obtain the target similarity, and obtain the target overlapping region based on the target similarity.
[0097] In this embodiment, automatic matching is performed on the first input image to obtain multiple calculated K values. After traversing the image, the region with the smallest K value is determined as the pre-processed overlapping region.
[0098] Step 204: Determine the features of the first input image and the features of the second input image through the target overlapping region to obtain the target features.
[0099] In this embodiment, the target feature is the sum of the features of the first input image and the features of the second input image. An improved Scale-Invariant Feature Transform (SIFT) algorithm is used to obtain the target feature from the feature set. First, feature detection is performed in scale space to determine the position and scale of the feature points. Then, the principal direction of the gradient in the neighborhood of the feature point is used as the directional feature of that feature point, achieving scale and orientation independence for the operator. The SIFT algorithm is invariant to image scale and rotation, and exhibits good robustness to image deviations caused by changes in illumination and viewpoint. This improves the matching speed and the probability of correct matching.
[0100] Step 205: Based on the target features, perform image stitching on the first input image and the second input image.
[0101] In this embodiment, the image stitching adopts a weighted average image fusion method.
[0102] In this embodiment, this application proposes a feature point stitching algorithm for stitching morphological images. It eliminates seams in the stitched images and exhibits good robustness to changes in illumination and viewpoint shifts. It meets image standards for practical medical applications and improves the accuracy and stability of image registration compared to previous image stitching methods.
[0103] In some optional implementations, the step of determining the features of the first input image and the second input image through the target overlapping region to obtain the target features specifically includes:
[0104] Enlarge the first input image and the second input image to obtain the corresponding transformed image;
[0105] The transformed image is obtained by multiple Gaussian convolutions to obtain corresponding multi-layer images;
[0106] A Gaussian difference pyramid (DOG) is constructed using the multi-layered images;
[0107] The Gaussian difference pyramid used for feature detection yields the location, scale, and orientation of the feature points.
[0108] The position, scale, and orientation of the feature points are used as features of the first input image and the second input image to obtain the target features.
[0109] In the above implementation, a Gaussian pyramid is constructed. First, the first input image is enlarged by a factor of two to form the first layer of the first group of the Gaussian pyramid. This first layer image is then subjected to Gaussian convolution (essentially Gaussian smoothing or filtering) to form the second layer of the first group of the pyramid. The second layer image is then smoothed using a smoothing coefficient, and the resulting image forms the third layer. This process is repeated until L layers of images are obtained. Within the same group, each layer has the same size, only the smoothing coefficients differ. This yields the first group of Gaussian difference pyramids. Similarly, the second input image is used to generate each difference image layer by layer, and all difference images are combined to form a difference pyramid, resulting in the second group of Gaussian difference pyramids. In the Gaussian difference pyramid, each sampling point is compared with all its neighboring points to extract the image's extreme points as feature points. The feature points extracted by the scale-invariant feature transform algorithm are used to perform extreme value detection on the Gaussian difference pyramid to obtain the position and scale of the feature points. The orientation of the feature points is determined according to the pixel distribution of the feature points. The position, scale, and orientation of the feature points are combined to generate a feature vector.
[0110] In some optional implementations, the step of transforming the image through multiple Gaussian convolutions to obtain the corresponding multi-layer image specifically includes:
[0111] pass The image is transformed by Gaussian convolution, where x, y are pixel coordinates, x0, y0 are constants, and σ is a smoothing coefficient.
[0112] In the above implementation, the parameter σ is taken as a fixed value of 1.6 in the SIFT operator. This leverages the invariance of SIFT features to rotation and scale, as well as their robustness to noise, viewpoint changes, and illumination variations.
[0113] In some optional implementations, the step of constructing a Gaussian difference pyramid from the multi-layered images specifically includes:
[0114] The transformed image is used as the first layer of the Gaussian pyramid, and the corresponding i-th layer image is used as the i-th layer of the Gaussian pyramid. An n-layer Gaussian pyramid is constructed, where i is any integer between 2 and n.
[0115] The difference between the next layer of the Gaussian pyramid in the same group and the current layer of the Gaussian pyramid is calculated to obtain the Gaussian difference pyramid of layer n-1.
[0116] In the above implementation, within the same group, the dimensions of each layer of the image are the same, only the smoothing coefficients are different. σ is a fixed value that needs to be manually adjusted to achieve the desired effect; the corresponding smoothing coefficients are 0, σ, ..., k, respectively. n-2σ. Next, the Gaussian difference pyramid is constructed. The first layer of the first group of the Gaussian difference pyramid is obtained by subtracting the first layer of the first group from the second layer of the first group of the Gaussian pyramid of the first input image. The second group is constructed similarly from the second input image. This process continues, generating each difference image group by group and layer by layer, with all difference images forming the difference pyramid. In the Gaussian difference pyramid, each sampling point is compared with all its neighboring points to extract the extreme points of the image as feature points.
[0117] In some optional implementations, the step of stitching the first input image and the second input image together based on the target features specifically includes:
[0118] The first input image and the second input image are spatially superimposed to obtain the superimposed image;
[0119] Obtain the target features of the overlapping region, wherein the target features include at least the width parameter of the overlapping region;
[0120] The superimposed image is then blended according to the width parameter of the overlapping region.
[0121] In the above implementation, n matching points are randomly selected from all m matching feature points as sampling samples. The parameter x of the image transformation matrix M is estimated using the n matching point set. The number of samples in the subset of all m matching point sets that satisfy the transformation parameter x is calculated. Specifically, the estimation process involves randomly selecting some matching points to generate a fitting model, then estimating M using the fitting model, and finally selecting samples that satisfy the fitting model to estimate the number of samples. If the number of samples is large enough, the process stops; otherwise, the above operation is repeated. This iterative sliding matching process is used to solve for the optimal geometric transformation model. The first input image and the template image are transformed to the reference image coordinate system, and image fusion is then performed. Image fusion uses a weighted average image fusion method. The first input image f1 and the second input image f2 are spatially superimposed, and the superimposed image can be represented as:
[0122]
[0123] Where d1 and d2 represent weight values, which are related to the width of the overlapping region, and d2 + d1 = 1 (0 <= d1, d2 <= 1). In the overlapping region, d1 gradually changes from 1 to 0, and d2 gradually changes from 0 to 1, thereby achieving the transition fusion of the overlapping region's image f1 to f2. The calculation formulas for d1 and d2 are as follows:
[0124] Where x i The x-coordinate represents the horizontal coordinate of the current pixel. l With x rThese represent the x-coordinates of the left and right boundaries of the overlapping region, respectively.
[0125] In some optional implementations, the step of spatially superimposing the first input image and the template image to obtain the superimposed image further includes:
[0126] Obtain the first feature point and the second feature point in the overlapping region, where the second feature point is the smallest neighborhood point of the first feature point;
[0127] Calculate the Euclidean distance between the first feature point and the second feature point to obtain the first Euclidean distance;
[0128] Obtain the third feature point, which is the second smallest neighborhood point of the first feature point;
[0129] Calculate the Euclidean distance between the first feature point and the third feature point to obtain the second Euclidean distance;
[0130] If the Euclidean distance of the first Euclidean distance and the Euclidean distance of the second Euclidean distance are greater than the threshold, then the first feature point is deleted.
[0131] In the above embodiments, the minimum neighborhood point refers to the feature point with the smallest Euclidean distance to the first feature point. A screening process is performed on all feature points. A threshold is set to remove feature points that do not have a good matching relationship. An effective measurement method is to compare the distance between the nearest neighbor (NN) and the second nearest neighbor (SNN). A threshold T is set for NN / SNN (generally between 0.6 and 0.8). If the value of NN / SNN is less than the threshold, the point is retained; otherwise, it is removed. If the value is less than or equal to the threshold, it is considered a valid feature point and is retained.
[0132] In some optional implementations, the steps of obtaining the first input image and the second input image are followed by:
[0133] The first input image is converted into grayscale values and mapped to the corresponding grayscale values using a monotonic grayscale mapping curve to obtain a grayscale image.
[0134] The grayscale image is denoised using a Gaussian blur to obtain a filtered image.
[0135] The filtered image is smoothed using convolution operations to obtain the complete image;
[0136] The complete image is Fourier transformed to obtain a frequency domain image;
[0137] The frequency domain image is processed by differentiation.
[0138] In the above embodiments, Gaussian blur is used to denoise the image. Convolution operations smooth the original image, filtering out isolated noise points and small structural elements to reduce image noise. First- or second-order differential operations highlight changes in image grayscale and enhance image details. Based on the differential properties of the Fourier transform, in the frequency domain, differential operations can actually strengthen high-frequency components.
[0139] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).
[0140] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0141] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of an image stitching device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.
[0142] like Figure 3 As shown, the image stitching device 300 described in this embodiment includes: an acquisition module 301, a matching module 302, an overlapping region calculation module 303, a target feature acquisition module 304, and a stitching module 305.
[0143] in:
[0144] The acquisition module 301 is used to acquire the first input image and the second input image;
[0145] The matching module 302 is used to perform sliding matching between the template image and the first input image, and calculate multiple similarities during the matching process. The template image is part or all of the second input image.
[0146] The overlapping region calculation module 303 is used to select the minimum value of the multiple similarities to obtain the target similarity, and to obtain the target overlapping region based on the target similarity;
[0147] The target feature acquisition module 304 is used to determine the features of the first input image and the features of the second input image through the target overlapping region to obtain the target features;
[0148] The stitching module 305 stitches the first input image and the second input image together according to the target features.
[0149] In some optional implementations of this embodiment, the matching module is further used for:
[0150] Enlarge the first input image and the second input image to obtain the corresponding transformed image;
[0151] The transformed image is obtained by multiple Gaussian convolutions to obtain corresponding multi-layer images;
[0152] A Gaussian difference pyramid is constructed using the multi-layered images;
[0153] The Gaussian difference pyramid used for feature detection yields the location, scale, and orientation of the feature points.
[0154] The position, scale, and orientation of the feature points are used as features of the first input image and the second input image to obtain the target features.
[0155] In some optional implementations of this embodiment, the matching module is further used for:
[0156] pass The image is transformed by Gaussian convolution, where x, y are pixel coordinates, x0, y0 are constants, and σ is a smoothing coefficient.
[0157] The transformed image is used as the first layer of the Gaussian pyramid, and the corresponding i-th layer image is used as the i-th layer of the Gaussian pyramid. An n-layer Gaussian pyramid is constructed, where i is any integer between 2 and n.
[0158] The difference between the next layer of the Gaussian pyramid in the same group and the current layer of the Gaussian pyramid is calculated to obtain the Gaussian difference pyramid of layer n-1.
[0159] In some optional implementations of this embodiment, the splicing module is further used for:
[0160] The first input image and the second input image are spatially superimposed to obtain the superimposed image;
[0161] Obtain the target features of the overlapping region, wherein the target features include at least the width parameter of the overlapping region;
[0162] The superimposed image is then blended according to the width parameter of the overlapping region.
[0163] In some optional implementations of this embodiment, the apparatus 300 further includes a threshold calculation module: the threshold calculation module is used for:
[0164] Obtain the first feature point and the second feature point in the overlapping region, where the second feature point is the smallest neighborhood point of the first feature point;
[0165] Calculate the Euclidean distance between the first feature point and the second feature point to obtain the first Euclidean distance;
[0166] Obtain the third feature point, which is the second smallest neighborhood point of the first feature point;
[0167] Calculate the Euclidean distance between the first feature point and the third feature point to obtain the second Euclidean distance;
[0168] If the Euclidean distance of the first Euclidean distance and the Euclidean distance of the second Euclidean distance are greater than the threshold, then the first feature point is deleted.
[0169] In some optional implementations of this embodiment, the apparatus 300 further includes a frequency domain analysis module: the frequency domain analysis module is used for:
[0170] The first input image is converted into grayscale values and mapped to the corresponding grayscale values using a monotonic grayscale mapping curve to obtain a grayscale image.
[0171] The grayscale image is denoised using a Gaussian blur to obtain a filtered image.
[0172] The filtered image is smoothed using convolution operations to obtain the complete image;
[0173] The complete image is Fourier transformed to obtain a frequency domain image;
[0174] The frequency domain image is processed by differentiation.
[0175] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.
[0176] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.
[0177] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.
[0178] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 6. Of course, the memory 41 may include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as program code for image stitching methods. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.
[0179] In some embodiments, the processor 42 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run program code stored in the memory 41 or process data, for example, to run the program code for the image stitching method.
[0180] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.
[0181] This application also provides another embodiment, namely, a computer-readable storage medium storing an image stitching program, which can be executed by at least one processor to cause the at least one processor to perform the steps of the image stitching method as described above.
[0182] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0183] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.
Claims
1. An image stitching method, characterized by, Includes the following steps: Obtain the first input image and the second input image; The template image is matched with the first input image, and multiple similarities are calculated during the matching process. The template image is part or all of the second input image. The minimum value of the multiple similarities is selected to obtain the target similarity, and the target overlapping region is obtained based on the target similarity; The target features are obtained by determining the features of the first input image and the features of the second input image through the target overlapping region; Based on the target features, the first input image and the second input image are stitched together. The step of determining the features of the first input image and the second input image through the target overlapping region to obtain the target features specifically includes: Enlarge the first input image and the second input image to obtain the corresponding transformed image; The transformed image is obtained by multiple Gaussian convolutions to obtain corresponding multi-layer images; A Gaussian difference pyramid is constructed using the multi-layered images; The Gaussian difference pyramid used for feature detection yields the location, scale, and orientation of the feature points. The position, scale, and orientation of the feature points are used as features of the first input image and the second input image to obtain the target features.
2. The image stitching method of claim 1, wherein, The specific steps of transforming the image through multiple Gaussian convolutions to obtain the corresponding multi-layer image include: pass The image is transformed by Gaussian convolution, where x, y are pixel coordinates. , It is a constant. This is the smoothing coefficient.
3. The image stitching method of claim 2, wherein, The step of constructing a Gaussian difference pyramid from the multi-layered images specifically includes: The transformed image is used as the first layer of Gaussian pyramid, and the corresponding i-th layer image is used as the i-th layer of Gaussian pyramid to construct an n-layer Gaussian pyramid, where i is any positive integer between 2 and n. The difference between the next layer of the Gaussian pyramid in the same group and the current layer of the Gaussian pyramid is calculated to obtain the Gaussian difference pyramid of layer n-1.
4. The image stitching method of claim 1, wherein, The step of stitching the first input image and the second input image together based on the target features specifically includes: The first input image and the second input image are spatially superimposed to obtain the superimposed image; Obtain the target features of the overlapping region, wherein the target features include at least the width parameter of the overlapping region; The superimposed image is then blended according to the width parameter of the overlapping region.
5. The image stitching method according to claim 4, characterized in that, Before the step of spatially superimposing the first input image and the second input image to obtain the superimposed image, the method further includes: Obtain the first feature point and the second feature point in the overlapping region, where the second feature point is the smallest neighbor point of the first feature point; Calculate the Euclidean distance between the first feature point and the second feature point to obtain the first Euclidean distance; Obtain the third feature point, which is the second smallest neighborhood point of the first feature point; Calculate the Euclidean distance between the first feature point and the third feature point to obtain the second Euclidean distance; If the ratio of the first Euclidean distance to the second Euclidean distance is greater than a threshold, then the first feature point is deleted.
6. The image stitching method of claim 1, wherein, After the steps of acquiring the first input image and the second input image, the following is also included: By using a monotonic grayscale mapping curve, the first input image and the second input image are converted into grayscale values and mapped to the corresponding grayscale values to obtain a grayscale image; The grayscale image is denoised using a Gaussian blur to obtain a filtered image. The filtered image is smoothed using convolution operations to obtain the complete image; The complete image is Fourier transformed to obtain a frequency domain image; The frequency domain image is processed by differentiation.
7. An image stitching apparatus characterized by comprising: include: The acquisition module is used to acquire the first input image and the second input image; The matching module is used to perform sliding matching between the template image and the first input image, and calculate multiple similarities during the matching process. The template image is part or all of the second input image. The overlapping region calculation module is used to select the minimum value of the multiple similarities to obtain the target similarity, and to obtain the target overlapping region based on the target similarity; The target feature acquisition module is used to determine the features of the first input image and the features of the second input image through the target overlapping region to obtain the target features; The stitching module stitches the first input image and the second input image according to the target features; Specifically, the target feature acquisition module is used for: Enlarge the first input image and the second input image to obtain the corresponding transformed image; The transformed image is obtained by multiple Gaussian convolutions to obtain corresponding multi-layer images; A Gaussian difference pyramid is constructed using the multi-layered images; The Gaussian difference pyramid used for feature detection yields the location, scale, and orientation of the feature points. The position, scale, and orientation of the feature points are used as features of the first input image and the second input image to obtain the target features.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the image stitching method as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image stitching method as described in any one of claims 1 to 6.