Method and system for accelerating image feature processing
By constructing image acquisition and feature processing models, multiple displacement transformation matrices are generated and feature space matrices are constructed, which solves the problems of low image feature processing efficiency and insufficient accuracy, and achieves efficient image registration.
Patent Information
- Application Number
- CN202411144159.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-20
AI Technical Summary
The existing image feature processing algorithms have low computational efficiency when processing large-scale data, resulting in a long feature processing cycle and insufficient comprehensive and accurate feature point extraction, which affects the registration accuracy.
By constructing an image acquisition model, simulation model, feature capture model and feature generation model, multiple image transformation matrices are generated after displacement transformation, and a new feature space matrix is constructed using feature elements to perform bit multiplication and sum calculation to accelerate image feature processing .
It improves the accuracy and robustness of image registration, reduces the consumption of computing resources and time, shortens the feature processing cycle, meets the requirements of registration speed, and makes image registration more efficient.
Smart Images

Figure CN118967763A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method and system for accelerating image feature processing, and belongs to the technical field of image processing. Background Art
[0002] Rigid registration is a widely used method in computer vision and image processing, mainly used to align two or more point clouds, images or other geometric objects. Rigid registration only allows rotation and translation transformations, without involving scaling or other deformations. In other words, in rigid registration, the shape and size of the object remain unchanged. It is used to align images obtained at different time points or with different imaging technologies. However, the existing rigid registration methods have the following main problems when processing image features:
[0003] 1. Existing image feature processing algorithms may encounter computational efficiency issues when processing large-scale data, especially in applications that require real-time processing. They require a lot of computing resources and time, resulting in a relatively long feature processing cycle, which often fails to meet speed requirements and is not efficient enough.
[0004] 2. The accuracy of existing image feature processing algorithms mainly depends on the feature points in the image, but these feature points are difficult to extract comprehensively and accurately, which affects the accuracy of registration.
[0005] The information disclosed in this Background Art is only for understanding the background of the inventive concept and therefore it may include information that does not constitute the prior art. Summary of the invention
[0006] In response to the above problem or one of the above problems, an object of the present invention is to provide a method for accelerating image feature processing. By constructing an image acquisition model, an image simulation model, a feature capture model, a feature calculation model, and a feature generation model, image features can be captured more comprehensively and accurately, thereby improving the accuracy and robustness of registration. At the same time, by using feature elements to construct a new feature space matrix, the matrix order of the original image data can be effectively reduced, thereby effectively accelerating image feature processing, saving computing resources and time, shortening the feature processing cycle, meeting the requirements of registration speed, and making image registration more efficient.
[0007] In response to the above problem or one of the above problems, the second purpose of the present invention is to provide a method and system for accelerating image feature processing. When performing rigid image registration, a plurality of image transformation matrices after displacement transformation are generated by moving the image correlation matrix, and the characteristic elements of the picture are obtained based on the image transformation matrix, so that the image features can be captured more comprehensively and accurately, and the accuracy and robustness of the registration can be improved; at the same time, a new feature space matrix is constructed using the characteristic elements, which can effectively reduce the matrix order of the original picture data, thereby effectively accelerating image feature processing, saving computing resources and time, shortening the feature processing cycle, meeting the requirements of registration speed, and making image registration more efficient.
[0008] To achieve one of the above purposes, the first technical solution of the present invention is:
[0009] A method for accelerating image feature processing comprises the following steps:
[0010] Step 1, obtaining at least one image to be registered through a pre-built image acquisition model;
[0011] Step 2, using a pre-built image simulation model, generating a numerical matrix based on the image to represent the image information;
[0012] Step 3, using a pre-built feature capture model, the numerical matrix is moved to generate a plurality of image transformation matrices after displacement transformation;
[0013] Step 4, using the pre-built feature calculation model, perform bitwise multiplication and sum calculation on the generated image transformation matrices in pairs to obtain a number of feature elements;
[0014] Step 5: Based on the pre-built feature generation model, several feature elements are assembled to obtain a new feature space matrix for representing the registration feature information of the image; and based on the new feature space matrix, accelerated image feature processing is achieved.
[0015] After continuous exploration and experiments, the present invention can capture image features more comprehensively and accurately, and improve the accuracy and robustness of registration by constructing image acquisition models, image simulation models, feature capture models, feature calculation models, and feature generation models; at the same time, by using feature elements to construct a new feature space matrix, the matrix order of the original image data can be effectively reduced, thereby effectively accelerating image feature processing, saving computing resources and time, shortening the feature processing cycle, meeting the requirements of registration speed, and making image registration more efficient.
[0016] As the preferred technical measures:
[0017] Step 3, using the pre-built feature capture model, the method of moving the numerical matrix to generate multiple displacement transformed image transformation matrices is as follows:
[0018] Step 31, respectively translate the numerical matrix in the first direction by 1*n, 2*n, and 3*n rows, and translate downward by 1*m at the same time, to obtain a first numerical matrix, a second numerical matrix, and a third numerical matrix;
[0019] Step 32, respectively translate the numerical matrix by 1*n, 2*n, and 3*n rows in the first direction, and move it by 2*m in the second direction to obtain a fourth numerical matrix, a fifth numerical matrix, and a sixth numerical matrix;
[0020] Step 33, respectively translate the numerical matrix by 1*n, 2*n, and 3*n rows in the first direction, and move it by 3*m in the second direction to obtain a seventh numerical matrix, an eighth numerical matrix, and a ninth numerical matrix;
[0021] Step 34, filling or extending the blank rows in the first numerical matrix, the second numerical matrix, the third numerical matrix, the fourth numerical matrix, the fifth numerical matrix, the sixth numerical matrix, the seventh numerical matrix, the eighth numerical matrix and the ninth numerical matrix to obtain the corresponding first response matrix, the second response matrix, the third response matrix, the fourth response matrix, the fifth response matrix, the sixth response matrix, the seventh response matrix, the eighth response matrix and the ninth response matrix;
[0022] Step 35, summing up the first response matrix, the second response matrix, the third response matrix, the fourth response matrix, the fifth response matrix, the sixth response matrix, the seventh response matrix, the eighth response matrix and the ninth response matrix to obtain nine image transformation matrices.
[0023] As the preferred technical measures:
[0024] Both n and m are positive integers, and their values are 1 or 2 or 3 or 4; or / and, the first direction is right, and the second direction is downward.
[0025] As the preferred technical measures:
[0026] Step 4: Use the pre-built feature calculation model to perform bitwise multiplication and sum calculation on the generated image transformation matrices in pairs to obtain several feature elements as follows:
[0027] Step 41, digitally processing a plurality of image transformation matrices to obtain a plurality of image storage data;
[0028] Step 42, determining the bit width of each image storage data, and based on the determination result, grouping the image storage data to obtain a first image storage data group that can be used for accelerated calculation and a second image storage data group that is not used for accelerated calculation;
[0029] Step 43, using a single instruction multiple data algorithm and a prefetch instruction, performing two-bit multiplication and sum calculation on the image storage data in the first image storage data group to obtain a first group of characteristic elements;
[0030] Step 44, using an SSE instruction to perform two-bit multiplication and sum calculation on the image storage data in the second image storage data group to obtain a second group of characteristic elements;
[0031] Step 45, summarizing the first group of characteristic elements and the second group of characteristic elements to obtain a plurality of characteristic elements.
[0032] As the preferred technical measures:
[0033] Step 42, the bit width of each image storage data is judged, and based on the judgment result, the image storage data is grouped to obtain a first image storage data group that can be used for accelerated calculation and a second image storage data group that is not used for accelerated calculation. The method is as follows:
[0034] Step 421, constructing a first image storage data group and a second image storage data group, and setting the bit width of each element to 32 bits;
[0035] Step 422, determining whether the bit width of each image storage data is divisible by 256 bit width;
[0036] When it is divisible, storing the image storage data into the first image storage data group;
[0037] When it cannot be divided evenly, the image storage data is stored in the second image storage data group.
[0038] As the preferred technical measures:
[0039] The method for performing two-bit multiplication and summation calculations on image storage data is as follows:
[0040] First, load the image storage data in the loop, and get the row start address of the image storage data through the matrix pointer;
[0041] At the same time, the prefetch instruction is used to load the image storage data that needs to be loaded next time in advance to obtain two matrix variables;
[0042] Then use the vector multiplication function to multiply the two matrix variables to obtain the characteristic elements.
[0043] As the preferred technical measures:
[0044] The number of characteristic elements is eighty-one;
[0045] The feature space matrix is a 9*9 matrix.
[0046] To achieve one of the above purposes, the second technical solution of the present invention is:
[0047] A method for accelerating image feature processing comprises the following steps:
[0048] Step 1, obtaining at least one image to be registered through a pre-built image acquisition model;
[0049] Step 2, using a pre-built image simulation model, generating a numerical matrix based on the image to represent the image information;
[0050] Step 3, using a pre-built feature capture model, the numerical matrix is moved to generate a plurality of image transformation matrices after displacement transformation;
[0051] Step 4, using a pre-built feature storage model to digitally process multiple image transformation matrices to obtain multiple sets of image transformation data;
[0052] Step 5, using a pre-built feature calculation model, a single instruction multiple data stream algorithm and a pre-fetch instruction, accelerating the processing of multiple sets of image transformation data to obtain a number of feature elements;
[0053] Step 6: Based on the pre-built feature generation model, several feature elements are assembled to obtain a low-order feature space matrix to accelerate image feature processing.
[0054] After continuous exploration and experiments, the present invention generates multiple image transformation matrices after displacement transformation by moving the image correlation matrix during rigid image registration, and obtains the characteristic elements of the image based on the image transformation matrix, so as to capture the image features more comprehensively and accurately and improve the accuracy and robustness of registration; at the same time, a new feature space matrix is constructed by using the characteristic elements, which can effectively reduce the matrix order of the original image data, thereby effectively accelerating the image feature processing, saving computing resources and time, shortening the feature processing cycle, meeting the requirements of registration speed, and making image registration more efficient.
[0055] To achieve one of the above purposes, the third technical solution of the present invention is:
[0056] A method for accelerating image feature processing, comprising the following contents:
[0057] acquiring at least one image to be registered;
[0058] According to the image, a numerical matrix is generated to represent the image information;
[0059] Performing a shift operation on the numerical matrix to generate a plurality of image transformation matrices after displacement transformation;
[0060] Perform bitwise multiplication and sum calculation on the generated image transformation matrices in pairs to obtain several feature elements;
[0061] Assemble several feature elements to obtain a simplified feature space matrix to accelerate image feature processing
[0062] The present invention can capture image features more comprehensively and accurately, and improve the accuracy and robustness of registration; at the same time, by using the feature space matrix for registration, the matrix order of the original image data can be effectively reduced, thereby effectively accelerating image feature processing, saving computing resources and time, shortening the feature processing cycle, meeting the requirements of registration speed, and making image registration more efficient.
[0063] To achieve one of the above purposes, the fourth technical solution of the present invention is:
[0064] An accelerated image feature processing system, comprising:
[0065] one or more processors;
[0066] A storage device for storing one or more programs;
[0067] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method for accelerating image feature processing.
[0068] Compared with the prior art solutions, the present invention has the following beneficial effects:
[0069] After continuous exploration and experiments, the present invention can capture image features more comprehensively and accurately, and improve the accuracy and robustness of registration by constructing image acquisition models, image simulation models, feature capture models, feature calculation models, and feature generation models; at the same time, by using feature elements to construct a new feature space matrix, the matrix order of the original image data can be effectively reduced, thereby effectively accelerating image feature processing, saving computing resources and time, shortening the feature processing cycle, meeting the requirements of registration speed, and making image registration more efficient.
[0070] Furthermore, after continuous exploration and experimentation, the present invention, when performing rigid image registration, generates multiple image transformation matrices after displacement transformation by moving the image correlation matrix, and obtains the characteristic elements of the image based on the image transformation matrix, so as to capture the image features more comprehensively and accurately, and improve the accuracy and robustness of the registration; at the same time, the characteristic elements are used to construct a new feature space matrix, which can effectively reduce the matrix order of the original image data, thereby effectively accelerating the image feature processing, saving computing resources and time, shortening the feature processing cycle, meeting the requirements of the registration speed, and making the image registration more efficient. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] Figure 1 A flow chart of a method for accelerating image feature processing according to the present invention;
[0072] Figure 2 A flow chart of the single instruction multiple data algorithm of the present invention;
[0073] Figure 3 A schematic diagram of the cache structure of the present invention;
[0074] Figure 4 Another flow chart of a method for accelerating image feature processing according to the present invention. DETAILED DESCRIPTION
[0075] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0076] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.
[0077] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "or / and" as used herein includes any and all combinations of one or more of the associated listed items.
[0078] like Figure 1 As shown, a first specific embodiment of a method for accelerating image feature processing of the present invention is as follows:
[0079] A method for accelerating image feature processing comprises the following steps:
[0080] Step 1, obtaining at least one image to be registered through a pre-built image acquisition model;
[0081] Step 2, using a pre-built image simulation model, generating a numerical matrix based on the image to represent the image information;
[0082] Step 3, using a pre-built feature capture model, the numerical matrix is moved to generate a plurality of image transformation matrices after displacement transformation;
[0083] Step 4, using the pre-built feature calculation model, perform bitwise multiplication and sum calculation on the generated image transformation matrices in pairs to obtain a number of feature elements;
[0084] Step 5: Based on the pre-built feature generation model, several feature elements are assembled to obtain a new feature space matrix, which is used to characterize the registration feature information of the image and accelerate the image feature processing.
[0085] In this embodiment, in step 3, a method of using a pre-built feature capture model to perform a movement operation on a numerical matrix to generate a plurality of image transformation matrices after displacement transformation is as follows:
[0086] Step 31, respectively translate the numerical matrix in the first direction by 1*n, 2*n, and 3*n rows, and translate downward by 1*m at the same time, to obtain a first numerical matrix, a second numerical matrix, and a third numerical matrix;
[0087] Step 32, respectively translate the numerical matrix by 1*n, 2*n, and 3*n rows in the first direction, and move it by 2*m in the second direction to obtain a fourth numerical matrix, a fifth numerical matrix, and a sixth numerical matrix;
[0088] Step 33, respectively translate the numerical matrix by 1*n, 2*n, and 3*n rows in the first direction, and move it by 3*m in the second direction to obtain a seventh numerical matrix, an eighth numerical matrix, and a ninth numerical matrix;
[0089] Step 34, filling or extending the blank rows in the first numerical matrix, the second numerical matrix, the third numerical matrix, the fourth numerical matrix, the fifth numerical matrix, the sixth numerical matrix, the seventh numerical matrix, the eighth numerical matrix and the ninth numerical matrix to obtain the corresponding first response matrix, the second response matrix, the third response matrix, the fourth response matrix, the fifth response matrix, the sixth response matrix, the seventh response matrix, the eighth response matrix and the ninth response matrix;
[0090] Step 35, summing up the first response matrix, the second response matrix, the third response matrix, the fourth response matrix, the fifth response matrix, the sixth response matrix, the seventh response matrix, the eighth response matrix and the ninth response matrix to obtain nine image transformation matrices.
[0091] In this embodiment, n and m are 1; the first direction is right, and the second direction is downward.
[0092] In this embodiment, in step 4, using the pre-built feature calculation model, the method of performing bitwise multiplication and sum calculation on the generated image transformation matrices in pairs to obtain a number of feature elements is as follows:
[0093] Step 41, digitally processing a plurality of image transformation matrices to obtain a plurality of image storage data;
[0094] Step 42, determining the bit width of each image storage data, and based on the determination result, grouping the image storage data to obtain a first image storage data group that can be used for accelerated calculation and a second image storage data group that is not used for accelerated calculation;
[0095] Step 43, using a single instruction multiple data algorithm and a prefetch instruction, performing two-bit multiplication and sum calculation on the image storage data in the first image storage data group to obtain a first group of characteristic elements;
[0096] Step 44, using an SSE instruction to perform two-bit multiplication and sum calculation on the image storage data in the second image storage data group to obtain a second group of characteristic elements;
[0097] Step 45, summarizing the first group of characteristic elements and the second group of characteristic elements to obtain a plurality of characteristic elements.
[0098] In this embodiment, in step 42, the bit width of each image storage data is judged, and based on the judgment result, the image storage data is grouped to obtain a first image storage data group that can be used for accelerated calculation and a second image storage data group that is not used for accelerated calculation. The method is as follows:
[0099] Step 421, constructing a first image storage data group and a second image storage data group, and setting the bit width of each element to 32 bits;
[0100] Step 422, determining whether the bit width of each image storage data is divisible by 256 bit width;
[0101] When it is divisible, storing the image storage data into the first image storage data group;
[0102] When it cannot be divided evenly, the image storage data is stored in the second image storage data group.
[0103] In this embodiment, the method for performing two-by-two multiplication and sum calculation on the image storage data is as follows:
[0104] First, load the image storage data in the loop, and get the row start address of the image storage data through the matrix pointer;
[0105] At the same time, the prefetch instruction is used to load the image storage data that needs to be loaded next time in advance to obtain two matrix variables;
[0106] Then use the vector multiplication function to multiply the two matrix variables to obtain the characteristic elements.
[0107] In this embodiment, the number of characteristic elements is eighty-one;
[0108] The feature space matrix is a 9*9 matrix.
[0109] A second specific embodiment of a method for accelerating image feature processing of the present invention:
[0110] A method for accelerating image feature processing comprises the following steps:
[0111] Step 1, obtaining at least one image to be registered through a pre-built image acquisition model;
[0112] Step 2, using a pre-built image simulation model, generating a numerical matrix based on the image to represent the image information;
[0113] Step 3, using a pre-built feature capture model, the numerical matrix is moved to generate a plurality of image transformation matrices after displacement transformation;
[0114] Step 4, using a pre-built feature storage model to digitally process multiple image transformation matrices to obtain multiple sets of image transformation data;
[0115] Step 5, using a pre-built feature calculation model, a single instruction multiple data stream algorithm and a pre-fetch instruction, accelerating the processing of multiple sets of image transformation data to obtain a number of feature elements;
[0116] Step 6: Based on the pre-built feature generation model, several feature elements are assembled to obtain a low-order feature space matrix to accelerate image feature processing.
[0117] like Figure 2 , Figure 3 , Figure 4 As shown, a third specific embodiment of a method for accelerating image feature processing of the present invention is as follows:
[0118] An accelerated image feature processing method is an optimization method for calculating rigid registration, which optimizes some operations in rigid registration. It specifically includes the following steps:
[0119] S1. Initialize and set relevant variables, including the following:
[0120] intwidth_mul=sizeof(_m256) / sizeof(float)
[0121] intveclen_mul=(A.cols / width_mul)*width_mul
[0122] The width multiple variable width_mul is used to calculate the number of float data that can be stored in a 256-bit length, which is used as the interval for loading data in the subsequent loop. The length of data that can be stored using _m256 is calculated using the vector length multiple variable veclen_mul. For the excess data that cannot be loaded using _m256, that is, the data that exceeds the vector length multiple variable, the ordinary instruction set of SSE needs to be used for calculation, while the data that is less than the vector length multiple variable can be accelerated using SIMD.
[0123] S2. Load data in a loop, which includes the following:
[0124] const float*mat1_row_ptr=A.ptr <float>(i);
[0125] constfloat*mat2_row_ptr=B.ptr <float>(i);
[0126] const float*mat3_row_ptr=C.ptr <float>(i);
[0127] In the loop, the data in the matrix needs to be loaded for subsequent use. The first row pointer array mat1_row_ptr takes the row start address of matrix A, the second row pointer array mat2_row_ptr takes the row start address of matrix B, and the third row pointer array mat3_row_ptr takes the row start address of matrix C. It is used for subsequent calculations. Next, use the prefetch instruction to load the data that needs to be loaded next time in advance. This can reduce the subsequent loading time.
[0128] First, determine whether the data for the next calculation exists. If it does, the prefetch memory data function _mm_prefetch will be executed. The prefetch memory data function _mm_prefetch will be translated into the PREFETCHh instruction and executed in the system. It aims to load the next data at the right time. Its expression is as follows:
[0129] mat1_row_ptr+j+width_mul, mat2_row_ptr+j+width_mul, mat3_row_ptr+j+width_mul
[0130] That is, the address of the data required for the next calculation. The specific code instructions are as follows:
[0131] if(j+step_size <veclen_mul)
[0132] {
[0133] _mm_prefetch(mat1_row_ptr+j+width_mul,_MM_HINT_NTA);
[0134] _mm_prefetch(mat2_row_ptr+j+width_mul,_MM_HINT_NTA);
[0135] _mm_prefetch(mat3_row_ptr+j+width_mul,_MM_HINT_NTA);
[0136] }
[0137] This part is also a part of the optimization in the invention, which loads the data needed for the next calculation in advance to reduce the time consumption.
[0138] Then, use three matrix loading variables _m256 to get the data in matrices A, B, and C for later calculations. Using the matrix loading instruction _mm256_loadu_ps, you can load 256-bit wide data at a time, which is larger than SSE loading data. The specific code instructions are as follows:
[0139] __m256load_mat1_a1=_mm256_loadu_ps(mat1_row_ptr+j);
[0140] __m256load_mat2_a1=_mm256_loadu_ps(mat2_row_ptr+j);
[0141] __m256load_mat3_a1=_mm256_loadu_ps(mat3_row_ptr+j);
[0142] S3. Calculation is performed in a loop, which includes the following:
[0143] Use the vector multiplication instruction _mm256_mul_ps to multiply the _mm256 variables. The first statement multiplies the matrix A data obtained in the previous step by the matrix B data obtained in the previous step. The second statement multiplies the loaded matrix A data by the corresponding matrix C data. The specific code instructions are as follows:
[0144] mul_resultd1=_mm256_mul_ps(load_mat1_a1,load_mat2_a1);
[0145] mul_resultd2=_mm256_mul_ps(load_mat1_a1,load_mat3_a1);
[0146] Use the vector multiplication instruction _mm256_mul_ps to perform multiplication calculations. 256-bit data can be multiplied at a time. One instruction can calculate more bits, which speeds up the calculation speed.
[0147] Here, the _mm256_add_ps instruction is used to add the _mm256 variables. The first statement adds the loaded partial data of matrix A to the corresponding data of matrix B to obtain the first summary variable sum_result1. The second statement adds the loaded partial data of matrix A to the corresponding data of matrix C to obtain the second summary variable sum_result2. The specific code instructions are as follows:
[0148] sum_result1=_mm256_add_pd(sum_result1,mul_resultd1);
[0149] sum_result2=_mm256_add_pd(sum_result2,mul_resultd2);
[0150] The vector addition function _mm256_add_pd is used to perform addition calculation, and 256 bits of data can be added at a time. This part is also a part of the optimization in the invention, so that when calculating the data of the A matrix, the data will not be loaded repeatedly, and the operation will be completed with one load. The traditional use of the opencv library function to execute will cause the A matrix to be loaded twice. This increases the cache hit rate of the cache memory of the present invention, thereby speeding up the calculation speed and reducing the time consumption.
[0151] After this calculation, the subsequent data is loaded in a loop until all the data are processed.
[0152] S4. General calculations, which include the following:
[0153] Previously, the AVX256 instruction set was used to operate the data that meets the 256-bit width. In order to support the situation where the data may not be divisible by 256, the processing of the data that does not meet the 256-bit width is added. The remaining data is executed using ordinary SSE instructions. The specific code instructions are as follows:
[0154] sum1+=double(mat1_row_ptr[j]*mat2_row_ptr[j]);
[0155] sum2+=double(mat1_row_ptr[j]*mat3_row_ptr[j]);
[0156] The first statement multiplies part of the data in matrix A by the corresponding data in matrix B and then adds it to the third summary variable sum1.
[0157] The second statement multiplies part of the data of matrix A by the corresponding data of matrix C and then adds it to the fourth summary variable sum2.
[0158] S5. Summarize the results, which include the following:
[0159] The result of the previous calculation is divided into two parts, one part is saved in the first summary variable sum_result1 and the second summary variable sum_result2 calculated according to the instruction set AVX256, and the other part is in the third summary variable sum1 and the fourth summary variable sum2. The two parts need to be summarized. The result of the first part is of type _mm256, which needs to be changed to float type. Therefore, it is necessary to loop width_sum times to add the float data in _mm256 to the third summary variable sum1 and the fourth summary variable sum2 respectively, so that the two parts are summarized together. The specific code instructions are as follows:
[0160] for(int i = 0; i <width_sum;++i)
[0161] {
[0162] sum1 += tmp_sum1[i];
[0163] sum2+=tmp_sum2[i];
[0164] }
[0165] Finally, sum1 and sum2 are the calculation results.
[0166] In this embodiment, SIMD technology is used (see Figure 2 ) and pre-fetch instructions to achieve accelerated rigid registration algorithm. SIMD stands for Single Instruction Multiple Data, which is a computing performance optimization technology based on a specific CPU instruction set. As the name suggests, it means that during the execution of a CPU instruction, multiple data can be calculated at the same time. This can achieve several times or even dozens of times performance improvement in data-intensive computing scenarios such as scientific computing and multimedia applications. The cache structure of the computer, see Figure 3 , data can be obtained faster in cache1, thus making data in the closer cache faster to read data. Prefetching instructions means fetching instructions or data from slower memory to faster local cache before they are actually needed, thereby improving execution performance, hence the name "prefetching".
[0167] A specific embodiment of applying the present invention to perform fast rigid registration of images:
[0168] A method for rapid rigid registration of images comprises the following steps:
[0169] The first step is to obtain two or more images to be registered through a pre-built image acquisition model;
[0170] The second step is to apply one of the above-mentioned accelerated image feature processing methods to process two or more images to obtain a corresponding number of feature space matrices;
[0171] In the third step, two or more images are quickly rigidly registered according to the corresponding feature space matrix.
[0172] In this embodiment, when performing rigid image registration, multiple image matrices after displacement transformation are generated by moving the image correlation matrix, which helps to fully and accurately capture image features and improve the accuracy and robustness of registration. In the translation process, in order to deal with empty rows or columns that appear, a filling method, edge extension or circular filling method can be used to process these empty rows to ensure the integrity of the image matrix.
[0173] In this embodiment, the translation step size can be selected as 1, 2, or 3 rows. These step sizes achieve a good balance between computational complexity and feature capture capability. The value of the translation step size is an adjustable parameter that can be optimized according to actual needs. More translation step sizes can capture richer image features, but will also increase computational complexity. Therefore, a trade-off needs to be made between accuracy and efficiency.
[0174] In this embodiment, the feature matrix is constructed by multiplying and summing the matrices in pairs, which actually calculates the dot product between different translation positions. The dot product operation measures the degree of alignment of two vectors (the image matrix after translation) and captures the correlation characteristics of the image at different translation positions. This method helps to improve the registration accuracy and forms a new feature space with a lower dimension than the original data, but retains important similarity information. The resulting feature matrix can be regarded as the feature response matrix of the image at different translation positions, which contains the registration feature information of the image. An accelerated image feature processing method of the present invention is also used at this stage to optimize the calculation speed of matrix multiplication in pairs. Inputting 1w sets of images using the original solution and the improved rigid registration both indicate that the present invention can increase the speed by about 50%.
[0175] Furthermore, the rigid registration scheme improved by the present invention is compared with the rigid registration scheme in the prior art. The time for executing the rigid registration is tested under the matrix size of 2048*2048, and 1w sets of data are tested. The final result is that the scheme in the prior art takes nearly 10 seconds to execute once, while the scheme of the present invention only takes 5 seconds to execute once. The present invention is twice as fast as the scheme in the prior art, so the present invention improves the efficiency of executing the rigid registration algorithm.
[0176] A system embodiment of the method for accelerating image feature processing using the present invention:
[0177] An accelerated image feature processing system, comprising:
[0178] one or more processors;
[0179] A storage device for storing one or more programs;
[0180] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method for accelerating image feature processing.
[0181] A computer medium embodiment using the method of the present invention:
[0182] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned accelerated image feature processing method.
[0183] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, and computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0184] The present application is described by flowcharts or / and block diagrams of the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each process or / and block in the flowchart or / and block diagram and the combination of the processes or / and blocks in the flowchart or / and block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart or / and block diagram. Figure 1 Process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0185] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 Process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0186] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 Process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0187] The model in this application is an object that objectively describes the morphological structure with the help of physical or virtual representation. The object is not equal to the object and is not limited to physical and virtual. It can be a data processing function, software program, processing mode, usage method, operation method, workflow, application process, electronic hardware, circuit module, processing system, system imitation or simulation object.
[0188] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field can still modify or replace the specific implementation methods of the present invention with equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be included in the scope of protection of the claims of the present invention.< / float> < / float> < / float>
Claims
1. A method for accelerating image feature processing, characterized in that: The following steps are involved: Step 1: Acquire at least one image to be registered using a pre-built image acquisition model; Step 2: Using a pre-built image simulation model, a numerical matrix is generated based on the image to represent the image information; Step 3: Using a pre-built feature capture model, the numerical matrix is moved to generate multiple image transformation matrices after displacement transformation; Step 4: Using the pre-built feature calculation model, perform bitwise multiplication and summation on the generated image transformation matrices to obtain several feature elements; In step 5, based on the pre-built feature generation model, several feature elements are assembled to obtain a new feature space matrix for representing the registration feature information of the image; and based on the new feature space matrix, accelerated image feature processing is achieved.
2. The method for accelerating image feature processing according to claim 1, wherein: Step 3: Using the pre-built feature capture model, the numerical matrix is moved to generate multiple image transformation matrices after displacement transformation as follows: Step 31, respectively shifting the numerical matrix in the first direction by 1*n, 2*n, and 3*n rows, and at the same time shifting it downward by 1*m, to obtain a first numerical matrix, a second numerical matrix, and a third numerical matrix; Step 32, respectively shifting the numerical matrix by 1*n, 2*n, and 3*n rows in the first direction and by 2*m in the second direction to obtain a fourth numerical matrix, a fifth numerical matrix, and a sixth numerical matrix; Step 33, respectively shifting the numerical matrix by 1*n, 2*n, and 3*n rows in the first direction and by 3*m in the second direction to obtain a seventh numerical matrix, an eighth numerical matrix, and a ninth numerical matrix; Step 34, filling or expanding the blank rows in the first numerical matrix, the second numerical matrix, the third numerical matrix, the fourth numerical matrix, the fifth numerical matrix, the sixth numerical matrix, the seventh numerical matrix, the eighth numerical matrix, and the ninth numerical matrix to obtain the corresponding first response matrix, the second response matrix, the third response matrix, the fourth response matrix, the fifth response matrix, the sixth response matrix, the seventh response matrix, the eighth response matrix, and the ninth response matrix; Step 35 , summing up the first response matrix, the second response matrix, the third response matrix, the fourth response matrix, the fifth response matrix, the sixth response matrix, the seventh response matrix, the eighth response matrix and the ninth response matrix to obtain nine image transformation matrices.
3. The method for accelerating image feature processing according to claim 2, wherein: Both n and m are positive integers, and their values are 1, 2, 3, or 4; or / and, the first direction is right, and the second direction is downward.
4. The method for accelerating image feature processing according to claim 1, wherein: Step 4: Use the pre-built feature calculation model to perform bitwise multiplication and summation on the generated image transformation matrix to obtain several feature elements as follows: Step 41, digitally processing multiple image transformation matrices to obtain multiple image storage data; Step 42: Determine the bit width of each image storage data, and group the image storage data based on the determination result to obtain a first image storage data group that can be used for accelerated calculation and a second image storage data group that is not used for accelerated calculation; Step 43, using a single instruction multiple data (SIMD) algorithm and a prefetch instruction, performing two-bit multiplication and summation calculations on the image storage data in the first image storage data group to obtain a first group of characteristic elements; Step 44, using an SSE instruction to perform two-bit multiplication and summation calculations on the image storage data in the second image storage data group to obtain a second set of characteristic elements; Step 45: Aggregate the first group of characteristic elements and the second group of characteristic elements to obtain a plurality of characteristic elements.
5. The method for accelerating image feature processing according to claim 4, wherein: Step 42, determining the bit width of each image storage data, and grouping the image storage data based on the determination result, to obtain a first image storage data group that can be used for accelerated calculation and a second image storage data group that is not used for accelerated calculation. The method is as follows: Step 421: construct a first image storage data group and a second image storage data group, and set the bit width of each element to 32 bits; Step 422, determining whether the bit width of each image storage data is divisible by 256 bits; When the number is divisible, the image storage data is stored in the first image storage data group; When the number cannot be divided evenly, the image storage data is stored in the second image storage data group.
6. The method for accelerating image feature processing according to claim 5, wherein: The method for performing two-bit multiplication and summation calculations on image storage data is as follows: First, load the image storage data in the loop and get the row start address of the image storage data through the matrix pointer; At the same time, the prefetch instruction is used to load the image storage data that needs to be loaded next time in advance to obtain two matrix variables; Then use the vector multiplication function to multiply the two matrix variables to obtain the characteristic elements.
7. The method for accelerating image feature processing according to claim 1, wherein: The number of characteristic elements is eighty-one; The feature space matrix is a 9*9 matrix.
8. A method for accelerating image feature processing, characterized in that: The following steps are involved: Step 1: Acquire at least one image to be registered using a pre-built image acquisition model; Step 2: Using a pre-built image simulation model, a numerical matrix is generated based on the image to represent the image information; Step 3: Using a pre-built feature capture model, the numerical matrix is moved to generate multiple image transformation matrices after displacement transformation; Step 4: Using a pre-built feature storage model, digitize multiple image transformation matrices to obtain multiple sets of image transformation data; Step 5: Using a pre-built feature calculation model, a single instruction multiple data algorithm and prefetch instructions, the multiple sets of image transformation data are accelerated to obtain a number of feature elements; In step 6, based on the pre-built feature generation model, several feature elements are assembled to obtain a low-order feature space matrix to accelerate image feature processing.
9. A method for accelerating image feature processing, characterized in that: It includes the following: acquiring at least one image to be registered; According to the image, a numerical matrix is generated to represent the image information; Performing a shift operation on the numerical matrix to generate a plurality of image transformation matrices after displacement transformation; Perform bitwise multiplication and sum calculation on the generated image transformation matrix in pairs to obtain several feature elements; Several feature elements are assembled to obtain a simplified feature space matrix to accelerate image feature processing.
10. An accelerated image feature processing system, characterized by: It includes: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for accelerating image feature processing according to any one of claims 1 to 9.
Citation Information
Patent Citations
Method and device for performing image registration based on slices
CN107194959A
Image processing method and device, and augmented reality equipment
CN107230199A
Image registration improved algorithm based on SIFT features
CN112132877A
Rapid and universal image registration method
CN113516693A
Lightweight image registration method, system and device based on global homography estimation
CN114565511A
