An initial value search method for SIFT feature point matching based on GPU acceleration

Abstract

A GPU-accelerated initial value search method for SIFT feature point matching, belonging to the field of computer vision and image processing technology, is disclosed. The algorithm includes: allocating and aligning memory for the image; constructing a multi-scale space of Gaussian pyramids across adjacent frames using the GPU to obtain a complete Gaussian pyramid and a difference-of-Gaussian pyramid, and accelerating the computation of Gaussian-blurred data on the GPU; calculating feature point descriptors; performing SIFT feature point matching; and constructing control point matching equations based on the SIFT feature point calculations. This invention performs GPU-based parallel optimization in the strain measurement process of the core synchronization mechanism of a synchronous automatic clutch, optimizing the scale space construction and descriptor calculation in the SIFT feature point detection algorithm, thereby improving the measurement speed of strain measurement in the core synchronization mechanism of the synchronous automatic clutch; and obtaining initial matching values ​​through the least squares method, thereby improving the accuracy of the initial value search.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, and particularly relates to an initial value search method for SIFT feature point matching based on GPU acceleration. Background Technology

[0002] Synchronous automatic clutches are core components of combined power transmission systems, and the stability and safety of their core synchronizing mechanism directly affect their operating efficiency and lifespan. As a critical component, the core synchronizing mechanism is subjected to high-frequency alternating loads over long periods, making it prone to deformation and fracture, which poses a challenge to the safety and reliability of the synchronous automatic clutch. To gain a deeper understanding of the deformation of the core synchronizing mechanism during operation, the three-dimensional digital image correlation method within binocular vision technology can acquire accurate strain data of the core synchronizing mechanism during operation. This data helps optimize the structure and materials of the core synchronizing mechanism, improving its load-bearing capacity and fatigue resistance, thereby extending its service life. Furthermore, the strain response and fracture process of the core synchronizing mechanism under stable resonant excitation can be studied to predict potential damage locations and develop effective maintenance strategies, further improving the reliability and safety of the synchronous automatic clutch.

[0003] While digital image correlation methods can obtain full-field strain, their large data volume, cumbersome matching calculations, and excessive time consumption prevent real-time full-field strain measurement in the synchronous automatic clutch mechanism. The SIFT corner search and matching algorithm used in the initial value search is particularly time-consuming. To address the limitations of traditional methods in measurement speed and efficiency, this invention employs a GPU-accelerated SIFT feature point matching method. By optimizing the initial value search process through parallel computing, measurement efficiency is significantly improved. Specifically, GPUs excel in accelerating 3D digital image correlation algorithms, especially the SIFT corner search algorithm, injecting new vitality into high-frequency fatigue testing technology innovation and providing more reliable data support for its design, manufacturing, and operation. In summary, this not only improves the measurement speed of digital image correlation methods but also provides new paths and possibilities for enhancing the performance and safety of synchronous automatic clutches. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the problems of the above-mentioned or existing GPU-accelerated SIFT feature point matching initial value search methods, this invention is proposed.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] This invention provides an initial value search method for SIFT feature point matching based on GPU acceleration, comprising: allocating and aligning memory for the image; constructing a multi-scale space of Gaussian pyramids for adjacent frames based on the GPU to obtain a complete Gaussian pyramid and a difference-of-Gaussian pyramid, and accelerating the calculation of Gaussian blur on the GPU; calculating feature point descriptors; matching SIFT feature points; and constructing a control point matching equation based on the calculation of SIFT feature points.

[0008] As a preferred embodiment of the initial value search method for SIFT feature point matching based on GPU acceleration described in this invention, the memory allocation and alignment of the image includes: allocating memory of width w × height h for the image, with a memory size of pitch × h × size, where pitch is the smallest multiple of 128 greater than w, and size is the number of data bytes per pixel, ensuring that the starting address of each row of data conforms to the memory alignment requirements of the GPU.

[0009] As a preferred embodiment of the GPU-accelerated SIFT feature point matching initial value search method of the present invention, wherein: the GPU-accelerated Gaussian pyramid construction includes:

[0010] The Gaussian kernel function is stored as an array in constant memory of the GPU. Gaussian blur is applied to each group of images. A two-dimensional Gaussian kernel function is then convolved with the lowest layer image. The convolution kernel size is (2R+1)×(2R+1), where R is the radius of the Gaussian blur convolution kernel window. The radius of the Gaussian blur convolution kernel window is set to 9. The two-dimensional Gaussian blur convolution kernel is shown below:

[0011]

[0012] Based on the separability of the two-dimensional Gaussian function, the two-dimensional Gaussian blur convolution kernel can be represented as the product of two one-dimensional Gaussian functions, as shown in the following equation:

[0013]

[0014] Where x and y are the position coordinates in the horizontal and vertical directions, respectively, σ is the standard deviation in the Gaussian distribution, and S is the normalization constant of the two-dimensional Gaussian function. and Let S' be the one-dimensional Gaussian function along the horizontal and vertical directions, respectively, and S' be the normalization constant of the one-dimensional Gaussian function.

[0015] After reducing the dimensionality of the two-dimensional Gaussian blur convolution kernel, the time complexity of the algorithm for an image with a resolution of w×h is reduced from O(R×R×w×h) to O(R×R×w)+O(R×R×h), thus reducing the computational load.

[0016] As a preferred embodiment of the initial value search method for SIFT feature point matching based on GPU acceleration described in this invention, the GPU acceleration calculation for Gaussian blur includes: allocating GPU threads, wherein in the GPU architecture, a thread bundle is a group of 32 threads executed in parallel; each thread block is set to have 32×32 threads, and each thread block is responsible for calculating the final convolution value of a 32×24 region.

[0017] As a preferred embodiment of the initial value search method for SIFT feature point matching based on GPU acceleration described in this invention, the GPU acceleration calculation for Gaussian blur further includes: performing convolution calculations on GPU threads. Each thread first calculates a row convolution, and for a thread block, the row convolution calculation result of a 32×32 region is obtained. The calculation result is stored in the shared memory of the thread block. Then, each thread located in the middle region of the 32×24 thread block calculates a column convolution on the row convolution calculation result in the shared memory to obtain the final convolution value in the 32×24 region. After all threads are completed, the complete image after Gaussian blur can be obtained.

[0018] As a preferred embodiment of the initial value search method for SIFT feature point matching based on GPU acceleration described in this invention, the construction of the Gaussian difference pyramid includes: performing Gaussian blurring on the original image seven times in sequence to obtain a set of eight images; taking the third-to-last image at intervals as the 0th layer image of the next set; then performing Gaussian blurring on this image seven times in sequence to obtain the next set of images, and so on. Each time at intervals, the length and width of the image are reduced to half of their original size, until the length or width of the image is only one pixel. Finally, the difference between two adjacent images in each set is taken to generate the Gaussian difference pyramid.

[0019] As a preferred embodiment of the initial value search method for SIFT feature point matching based on GPU acceleration described in this invention, the construction of the Gaussian difference pyramid further includes: after constructing the Gaussian difference pyramid, performing local extremum detection on the middle 5 layers of the 7-layer difference pyramid in each group. Each thread block contains 32 threads to complete the extremum detection of 30 pixels. When detecting each point, the maximum and minimum values ​​of the 8 points on its upper, lower, left, right and diagonal sides, as well as the 9 points at the corresponding positions in each of the upper and lower layers are calculated. It is determined whether the point is between the minimum and maximum values. If not, the point is a local extremum point and is stored as a feature point.

[0020] As a preferred embodiment of the initial value search method for SIFT feature point matching based on GPU acceleration described in this invention, the calculation of the feature point descriptor includes: classifying the feature points according to the Gaussian pyramid layer, and then calculating the principal direction and descriptor in parallel for feature points in the same layer; the calculation method of the feature descriptor is as follows: the thread block is set as a two-dimensional thread set containing 16×16, one thread block is responsible for completing the calculation of all pixels in a circular domain with radius R of a feature point and obtaining the corresponding 128-dimensional feature vector, each thread is responsible for completing the calculation of one pixel in a circular domain with radius R of the feature point in each layer of the image, obtaining the contribution of its corresponding seed point in 8 directions, accumulating it on the descriptor, placing the neighborhood information of each feature point in global memory, and returning it to the CPU to complete the normalization step of the generated SIFT descriptor.

[0021] As a preferred embodiment of the initial value search method for SIFT feature point matching based on GPU acceleration described in this invention, the SIFT feature point matching includes: in the SIFT feature point matching of two images, one thread block calculates the correlation scores of 32 feature points in image one with all feature points in image two. One thread block is divided into 32×8 threads, and the 8 threads jointly calculate the correlation scores of the same feature point with all feature points in image two. Each thread calculates the correlation scores of the same feature point with the 16-dimensional feature point descriptors of all feature points in image two. Finally, the total score is obtained through atomic addition operations, and the point with the largest correlation score is the matching point of that point.

[0022] As a preferred embodiment of the GPU-accelerated SIFT feature point matching initial value search method of the present invention, wherein: the calculation of SIFT feature points to construct the control point matching equation includes:

[0023] After obtaining the matched feature point pairs, the calculation steps for the initial value of the detection point matching are as follows: In the reference image, select the n points closest to the detection point as control points (n≥3) for initial value solving. In this algorithm, n=4 is used. After obtaining the coordinates of the control point pairs, establish the following system of equations for each pair of control points using the first-order displacement mode:

[0024]

[0025] in, The coordinates of the detection point under the reference image. The coordinates of the control points in the target image. The relative coordinates of the control point and the reference point in the reference image. Let be the first-order shape function corresponding to the detection point to be determined; u and v are the displacements of the detection point in the x and y directions of the image, respectively. and Let be the rates of change of displacement u in the x and y directions, respectively. and Let v be the rate of change of displacement v in the x and y directions, respectively. The equations for the four control points are combined and solved using the least squares method to obtain the first-order shape function, which is then used to calculate the initial matching value. .

[0026] The beneficial effects of this invention are as follows: This invention performs GPU-based parallel optimization on the initial value search algorithm based on SIFT feature point matching in the strain measurement process of the synchronous automatic clutch synchronization mechanism. The main optimizations are to the scale space construction and descriptor calculation in the SIFT feature point detection algorithm, thereby improving the measurement speed of the synchronous automatic clutch synchronization mechanism strain measurement. To adapt to the initial value search under complex working conditions of the core synchronization mechanism of the synchronous automatic clutch, a first-order displacement model equation system is established by finding the four feature point pairs with the closest matching degree to the detection point. This allows for translation, rotation, uniform shearing, and tensile deformation of the target subset, and the initial matching value is obtained through the least squares method, thereby improving the accuracy of the initial value search. It satisfies the requirement for real-time measurement of the full-field strain data of the core synchronization mechanism of the synchronous automatic clutch, which helps to optimize the structure and materials of the core synchronization mechanism, improve its load-bearing capacity and fatigue resistance, and thus extend its service life. Furthermore, it can also study the strain response and fracture process of the core synchronization mechanism under stable resonant excitation to predict possible damage locations and formulate effective maintenance strategies, further improving the reliability and safety of the synchronous automatic clutch. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of an initial value search method for SIFT feature point matching based on GPU acceleration, provided in an embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram of a GPU-accelerated Gaussian blur algorithm provided in an embodiment of the present invention. Detailed Implementation

[0030] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0031] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0032] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0033] Example 1

[0034] Reference Figures 1-2 This is one embodiment of the present invention, which provides an initial value search method for SIFT feature point matching based on GPU acceleration, including:

[0035] S1: Allocate and align memory for the image.

[0036] Preferably, memory of width w × height h is allocated to the image, and the memory size is pitch × h × size, where pitch is a minimum multiple of 128 greater than w, and size is the number of data bytes per pixel, ensuring that the starting address of each row of data conforms to the GPU's memory alignment requirements.

[0037] Furthermore, assume the image width w = 1024 pixels, the image height h = 768 pixels, and the data size of each pixel size = 4 bytes;

[0038] Calculate the number of bytes required per row when unaligned: bytes_per_row = w × size = 1024 × 4 = 4096 bytes;

[0039] In this example, 4096 is already a multiple of 128, therefore pitch = 4096;

[0040] Allocate GPU memory of size pitch×h×size: total_memory=pitch×h=4096×768=3145728 bytes. Use the cudaMallocPitch function to allocate memory for the image to automatically ensure the correct alignment of the pitch.

[0041] S2: Obtain the complete Gaussian pyramid.

[0042] Preferably, the Gaussian kernel function is stored as an array in the constant memory of the GPU. Gaussian blur is applied to each group of images, and a two-dimensional Gaussian kernel function is used to convolve with the lowest layer image. The convolution kernel size is (2R+1)×(2R+1), where R is the radius of the Gaussian blur convolution kernel window. The radius of the Gaussian blur convolution kernel window is set to 9. The two-dimensional Gaussian blur convolution kernel is shown below:

[0043]

[0044] Based on the separability of the two-dimensional Gaussian function, the two-dimensional Gaussian blur convolution kernel can be represented as the product of two one-dimensional Gaussian functions, as shown in the following equation:

[0045]

[0046] Where x and y are the position coordinates in the horizontal and vertical directions, respectively, σ is the standard deviation in the Gaussian distribution, and S is the normalization constant of the two-dimensional Gaussian function. and Let S' be the one-dimensional Gaussian function along the horizontal and vertical directions, respectively, and S' be the normalization constant of the one-dimensional Gaussian function.

[0047] After reducing the dimensionality of the two-dimensional Gaussian blur convolution kernel, the time complexity of the algorithm for an image with a resolution of w×h is reduced from O(R×R×w×h) to O(R×R×w)+O(R×R×h), thus reducing the computational load.

[0048] Furthermore, assume that the Gaussian blur convolution kernel window radius R=9 (meaning the kernel size is (2R+1)×(2R+1)=19×19), and the standard deviation σ=R / 3;

[0049] Calculate the normalization constant S′:

[0050]

[0051] Construct one-dimensional Gaussian kernels Gx and Gy of length (2R+1)(2R+1)(2R+1), where x,y∈[-R,R];

[0052] Perform a horizontal convolution on each row of the image, and calculate the following for each pixel:

[0053]

[0054] After blurring in the horizontal direction, perform convolution in the vertical direction:

[0055]

[0056] Starting from the lowest layer, the next pyramid image is obtained by applying a Gaussian blur. This blurring operation is repeated, and the images are scaled (e.g., each layer's resolution is reduced to half that of the previous layer) until the complete pyramid is constructed.

[0057] S3: Accelerates GPU computation for Gaussian blur.

[0058] Preferably, GPU threads are allocated. In the GPU architecture, a thread bundle is a group of 32 threads that execute in parallel. If the number of threads in each thread block is a multiple of 32, then all thread bundles are fully filled and there are no unused threads. Each thread block is set to have 32×32 threads, and each thread block is responsible for calculating the final convolution value of a 32×24 region.

[0059] Preferably, convolution calculations are performed on GPU threads, with each thread first calculating a row convolution. For a single thread block, the row convolution calculation result for a 32×32 region is obtained.

[0060] The calculation result is stored in the shared memory of the thread block. Then, each thread in the middle area of ​​the 32×24 thread block calculates the column convolution again on the row convolution calculation result in the shared memory to obtain the final convolution value in the 32×24 area. After all threads are finished, the complete image after Gaussian blur can be obtained.

[0061] Furthermore, the value of the one-dimensional Gaussian kernel function (with a kernel radius of 9) is calculated, stored in an array, and loaded into constant memory of the GPU for fast access during convolution.

[0062] The entire image is divided into several 32×24 small regions, each of which is processed by a thread block. This ensures that the divided blocks can completely cover the image. If the image size is not an integer multiple of 32×24, some extra pixels can be padded at the image edges to make it the correct size.

[0063] For each small block, all threads first perform a Gaussian convolution in the row direction on the small region they are responsible for, and store the result in shared memory so that subsequent operations can access it quickly.

[0064] After the row convolution is completed, the thread in the middle 32×24 region of the thread block reads the row convolution result in the shared memory again and performs Gaussian convolution in the column direction. Since the shared memory read speed is very fast, the time required for the convolution operation can be greatly reduced.

[0065] Merge the small blocks to generate the final blurred image. Merge the blur results of the 32×24 small regions calculated by each thread block to generate a complete blurred image.

[0066] S4: Obtain the complete Gaussian difference pyramid.

[0067] Preferably, the original image is subjected to seven Gaussian blurs in sequence to obtain a set of eight images. The third-to-last image is then sampled at intervals to become the 0th layer image of the next set. This image is then subjected to seven Gaussian blurs in sequence to obtain the next set of images. This process is repeated, with each interval sampling reducing the length and width of the image to half of its original size, until the length or width of the image is reduced to only one pixel. Finally, the difference between any two adjacent images in each set is taken to generate the Gaussian difference pyramid.

[0068] Preferably, after constructing the Gaussian difference pyramid, local extremum detection is performed on the middle 5 layers of the 7-layer difference pyramid in each group. Each thread block contains 32 threads to complete the extremum detection of 30 pixels. When detecting each point, the maximum and minimum values ​​of the 8 points on its top, bottom, left, right and diagonal, as well as the 9 points at the corresponding positions in each of the two layers above and below are calculated. It is determined whether the point is between the minimum and maximum values. If not, the point is a local extremum point and is stored as a feature point.

[0069] Furthermore, starting from an original image, the image is sequentially subjected to seven Gaussian blurs to obtain eight layers of images (including the original image).

[0070] Each Gaussian blur uses a different standard deviation σ, and the standard deviation increases sequentially with each blur to ensure that the smoothness of the image varies at different scales. This also ensures that the Gaussian kernel function is stored in constant memory on the GPU to improve access efficiency.

[0071] Take the third-to-last blurred image and use it as the starting layer for the next set of images. Downsample this image to reduce its resolution to half of its original value (halving both the length and width). Repeat this process seven times to obtain the next set of images. Each downsampling process gradually reduces the size of the image until the image is only one pixel long or wide.

[0072] For each set of images in the Gaussian pyramid, the image difference between two adjacent layers is calculated to generate 7 difference images. The difference operation is achieved by subtracting pixels from adjacent Gaussian blurred images to obtain the Gaussian difference pyramid. The GPU thread executes this operation in parallel to reduce processing time.

[0073] After constructing the Difference-of-Gaussian (DOB) pyramid, local extrema detection is performed on the middle five layers of the pyramid for feature point extraction. In the DOB pyramid, each pixel to be detected is compared with its eight adjacent pixels along the top, bottom, left, right, and right diagonals of the image, as well as nine pixels at the same position in the layers above and below. If the pixel value of this point is greater than or less than the values ​​of all points in the detection area, it is determined to be a local extrema and retained as a feature point.

[0074] Each GPU thread block contains 32 threads. Through proper allocation, each thread block is responsible for processing a region of 30 pixels, ensuring maximum utilization of the GPU's parallel computing capabilities. This block-based approach allows threads within each block to process pixels in different regions in parallel. Each thread reads the value of its assigned pixel, retrieves the values ​​of its eight adjacent pixels, and compares them with the values ​​of the nine pixels above and below it.

[0075] S5: Calculate the feature point descriptor.

[0076] Preferably, feature points are first classified according to the level of the Gaussian pyramid they are in, and then the principal direction and descriptor are calculated in parallel for feature points in the same level.

[0077] The calculation method for the feature descriptor is as follows: the thread block is set as a two-dimensional thread set containing 16×16. One thread block is responsible for calculating all pixels in a circular domain with radius R of a feature point and obtaining the corresponding 128-dimensional feature vector. Each thread is responsible for calculating one pixel in a circular domain with radius R of the feature point in each layer of the image, obtaining the contribution of its corresponding seed point in 8 directions, and accumulating it into the descriptor. The neighborhood information of each feature point is placed in global memory and returned to the CPU to complete the normalization step of the generated SIFT descriptor.

[0078] Furthermore, assuming the input data image resolution is 512×512 pixels, the number of feature points is 64, the radius R of the circular region of each feature point is 8 pixels, the size of each thread block is 16×16 threads, and the dimension of the descriptor is 128, assuming that the position of one of the feature points is (256,256), the area around it will be divided into a circular region with a radius of 8 for calculation.

[0079] Calculate the principal direction of the feature point. Input feature point coordinates: (256, 256). Circular region pixels: contains all pixels within an 8-pixel radius, approximately 225 pixels.

[0080] Calculate the gradient for (256, 254). Assuming the horizontal gradient is 5 and the vertical gradient is 3, we get the gradient magnitude as 52 + 32 = 34 ≈ 5.83 and the direction angle as atan(3 / 5) ≈ 30°.

[0081] The direction of each pixel is accumulated to the corresponding position of the 36-directional histogram. Assuming the direction angle (256, 254) is 30°, it will be accumulated to the histogram range of 0° to 10°. Assuming the maximum value of the histogram in the 90° direction is 40, then this direction is the main direction, and the main direction is determined to be 90°.

[0082] Gradient direction contribution calculation: Each thread calculates the gradient information of the assigned pixel. Assuming a pixel has a gradient of 7 and a direction of 110°, it is classified as "0°-45°" in the direction histogram, and its weighted contribution value is 7.

[0083] All threads within the thread block accumulate the results in shared memory, reducing global memory access. The information in each direction of the 4×4 small region is accumulated to obtain a 128-dimensional feature descriptor.

[0084] S6: Match SIFT feature points.

[0085] Preferably, in SIFT feature point matching between two images, one thread block calculates the correlation scores between 32 feature points in image one and all feature points in image two. One thread block is divided into 32×8 threads, and the 8 threads jointly calculate the correlation scores between the same feature point and all feature points in image two. Each thread calculates the correlation scores between the same feature point and the 16-dimensional feature point descriptors of all feature points in image two. Finally, the total score is obtained through atomic addition operations, and the point with the highest correlation score is the matching point of that point.

[0086] Furthermore, for example, there are 4 thread blocks, each processing 128 feature points in parallel:

[0087] Thread block 1: Process feature points 1-32 in image 1;

[0088] Thread Block 2: Process feature points 33-64 in image 1;

[0089] Thread block 3: Process feature points 65-96 in image 1;

[0090] Thread Block 4: Process feature points 97-128 in Image 1;

[0091] After each thread block completes its calculation, the best matching feature point result for each feature point in Image 1 is saved to shared memory. Finally, the result is sent back to the CPU, which aggregates all matching pairs of feature points to form complete image matching data.

[0092] By using a 32×8 thread configuration, the calculation process for each feature point is fully parallelized, significantly improving performance. The use of shared memory reduces the number of global memory accesses, further increasing data processing speed.

[0093] Assuming that single-threaded feature point matching takes 0.005 seconds, GPU parallel computing can reduce the time to less than 0.1 seconds.

[0094] S7: Construct control point matching equations based on SIFT feature point calculations.

[0095] Preferably, after obtaining the matched feature point pairs, the calculation steps for the initial value of the detection point matching are as follows: In the reference image, select the n points closest to the detection point as control points (n≥3) for initial value solving. In this algorithm, n=4 is taken. After obtaining the coordinates of the control point pairs, establish a system of equations for each pair of control points using the first-order displacement mode as follows:

[0096]

[0097] in, The coordinates of the detection point under the reference image. The coordinates of the control points in the target image. The relative coordinates of the control point and the reference point in the reference image. Let be the first-order shape function corresponding to the detection point to be determined; u and v are the displacements of the detection point in the x and y directions of the image, respectively. and Let be the rates of change of displacement u in the x and y directions, respectively. and These are the rates of change of displacement v in the x and y directions, respectively;

[0098] The equations for the four control points are combined and solved using the least squares method to obtain the first-order shape function, and then the initial matching value is calculated. .

[0099] Furthermore, assuming image A and target image B, find the matching position in image B that corresponds to the detection point in image A.

[0100] Suppose there is a feature point P in image A with coordinates (x, y, y). p ,y p Using the SIFT matching algorithm, assuming 100 pairs of feature points are obtained, 4 of them are selected as control points.

[0101] Control point coordinates (reference image A):

[0102] C1=(x c1 ,y c1 ), C2=(x c2 ,y c2 ), C3=(x c3 ,y c3 C4=(x) c4 ,y c4);

[0103] Suppose that the matching points of these four control points in the target image B are as follows:

[0104] C 1′ =(x c1′ ,y c1′ ),C 2′ =(x c2′ ,y c2′ ), C 3′ =(x c3′ ,y c3′ ), C 4′ =(x c4′ ,y c4′ );

[0105] Assume the matching point corresponding to the detection point P is P′=(x p′ ,y p′ The displacement mode can be estimated using control points. To obtain the initial value of P′, it is necessary to construct the relative displacement equation between the control points.

[0106] Representation of the displacement equation:

[0107] x p′ =a1·x p +a2·y p +b1, y p′ =a3·x p +a4·y p +b2;

[0108] A system of equations is constructed using four control points. These equations are then rearranged into matrix form AX=B. The X matrix is ​​solved using the least squares method, i.e.:

[0109]

[0110] The parameters a1, a2, a3, a4, b1, b2 of the first-order shape function are obtained. Assume that after solving, a1=0.98, a2=0.03, b1=5.1, a3=-0.04, a4=1.02, b2=3.5.

[0111] Calculate the initial matching value: P=(50,30)→P′=(x p′ ,y p′ = (54.7, 35.4).

[0112] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A GPU-accelerated SIFT feature point matching initial value search method, characterized in that, include: Allocate and align memory for the image; A multi-scale space for constructing adjacent multi-frame Gaussian pyramids based on GPU is obtained, resulting in a complete Gaussian pyramid and a Gaussian difference pyramid. The GPU is then used to accelerate the computation of Gaussian blur. Calculate the feature point descriptor; Match SIFT feature points; Constructing control point matching equations based on SIFT feature point calculations; The accelerated GPU calculation for Gaussian blur also includes: Convolution calculations are performed on GPU threads. Each thread first calculates the row convolution once. For a thread block, the row convolution calculation result of a 32×32 region is obtained. The calculation result is stored in the shared memory of the thread block. Then, each thread in the middle area of ​​the 32×24 thread block calculates the column convolution once on the row convolution calculation result in the shared memory to obtain the final convolution value in the 32×24 area. After all threads are finished, the complete image after Gaussian blur can be obtained. The calculation of the feature point descriptor includes: The feature points are first classified according to the level of the Gaussian pyramid they are in, and then the principal direction and descriptor are calculated in parallel for the feature points in the same level. The calculation method for the feature descriptor is as follows: the thread block is set as a two-dimensional thread set containing 16×16. One thread block is responsible for calculating all pixels in a circular domain with radius R of a feature point and obtaining the corresponding 128-dimensional feature vector. Each thread is responsible for calculating one pixel in a circular domain with radius R of the feature point in each layer of the image, obtaining the contribution of its corresponding seed point in 8 directions, and accumulating it into the descriptor. The neighborhood information of each feature point is placed in global memory and returned to the CPU to complete the normalization step of the generated SIFT descriptor. The matching of SIFT feature points includes: In SIFT feature point matching between two images, one thread block calculates the correlation scores between 32 feature points in image 1 and all feature points in image 2. One thread block is divided into 32×8 threads, and the 8 threads jointly calculate the correlation scores between the same feature point and all feature points in image 2. Each thread calculates the correlation scores between the same feature point and the 16-dimensional feature point descriptors of all feature points in image 2. Finally, the total score is obtained through atomic addition operations, and the point with the highest correlation score is the matching point. The calculation of control point matching equations based on SIFT feature points includes: After obtaining the matched feature point pairs, the calculation steps for the initial value of the detection point matching are as follows: In the reference image, select the n points closest to the detection point as control points (n≥3) for initial value solving. In this algorithm, n=4 is used. After obtaining the coordinates of the control point pairs, establish the following system of equations for each pair of control points using the first-order displacement mode: in, The coordinates of the detection point under the reference image. The coordinates of the control points in the target image. The relative coordinates of the control point and the reference point in the reference image. Let be the first-order shape function corresponding to the detection point to be determined; u and v are the displacements of the detection point in the x and y directions of the image, respectively. and Let be the rates of change of displacement u in the x and y directions, respectively. and These are the rates of change of displacement v in the x and y directions, respectively; The equations for the four control points are combined and solved using the least squares method to obtain the first-order shape function, and then the initial matching value is calculated. .

2. The initial value search method for GPU-accelerated SIFT feature point matching as described in claim 1, characterized in that, The memory allocation and alignment of the image includes: Allocate memory for the image with width w × height h, and the memory size is pitch × h × size, where pitch is the smallest multiple of 128 greater than w, and size is the number of data bytes per pixel. Ensure that the starting address of each row of data conforms to the GPU's memory alignment requirements.

3. The initial value search method for GPU-accelerated SIFT feature point matching as described in claim 1, characterized in that, The complete Gaussian pyramid is obtained by including: The Gaussian kernel function is stored as an array in constant memory of the GPU. Gaussian blur is applied to each group of images. A two-dimensional Gaussian kernel function is then convolved with the lowest layer image. The convolution kernel size is (2R+1)×(2R+1), where R is the radius of the Gaussian blur convolution kernel window. The radius of the Gaussian blur convolution kernel window is set to 9. The two-dimensional Gaussian blur convolution kernel is shown below: Based on the separability of the two-dimensional Gaussian function, the two-dimensional Gaussian blur convolution kernel can be represented as the product of two one-dimensional Gaussian functions, as shown in the following equation: Where x and y are the position coordinates in the horizontal and vertical directions, respectively, σ is the standard deviation in the Gaussian distribution, and S is the normalization constant of the two-dimensional Gaussian function. and Let S' be the one-dimensional Gaussian function along the horizontal and vertical directions, respectively, and S' be the normalization constant of the one-dimensional Gaussian function. After reducing the dimensionality of the two-dimensional Gaussian blur convolution kernel, the time complexity of the algorithm for an image with a resolution of w×h is reduced from O(R×R×w×h) to O(R×R×w)+O(R×R×h), thus reducing the computational load.

4. The initial value search method for GPU-accelerated SIFT feature point matching as described in claim 1, characterized in that, The GPU acceleration calculation for Gaussian blur includes: GPU threads are allocated; in the GPU architecture, a thread bundle is a group of 32 threads that execute in parallel. Each thread block is configured with 32×32 threads, and each thread block is responsible for calculating the final convolution value of a 32×24 region.

5. The initial value search method for SIFT feature point matching based on GPU acceleration as described in claim 1, characterized in that, The process of obtaining the complete Gaussian difference pyramid includes: The original image is subjected to seven Gaussian blurs in sequence to obtain a set of eight images. The third-to-last image is then sampled at intervals to become the 0th layer image of the next set. This image is then subjected to seven Gaussian blurs in sequence to obtain the next set of images. This process is repeated, with each interval sampling reducing the length and width of the image to half of its original size, until the length or width of the image is reduced to only one pixel. Finally, the difference between any two adjacent images in each set is taken to generate the Gaussian difference pyramid.

6. The initial value search method for GPU-accelerated SIFT feature point matching as described in claim 1 or 5, characterized in that, The process of obtaining a complete Gaussian difference pyramid also includes: After constructing the Gaussian difference pyramid, local extremum detection is performed on the middle 5 layers of the 7-layer difference pyramid in each group. Each thread block contains 32 threads to complete the extremum detection of 30 pixels. When detecting each point, the maximum and minimum values ​​of the 8 points on its top, bottom, left, right and diagonal, as well as the 9 points at the corresponding positions in each of the two layers above and below are calculated. It is then determined whether the point is between the minimum and maximum values. If not, the point is a local extremum point and is stored as a feature point.

Images