Sub-pixel image registration method, device and terminal equipment

Abstract

The present application is applicable to the technical field of sub-pixel image registration, and provides a sub-pixel image registration method, apparatus and terminal equipment. The sub-pixel image registration method includes: obtaining a reference image of a target object and a target object to be registered; performing Fourier transform on the target object to be registered and the reference image; performing band-limiting processing on the target object to be registered after the Fourier transform, filtering out invalid information in the target object to be registered; and registering the target object to be registered after the band-limiting processing based on the reference image. The present application solves the problem that the existing phase correlation method fails at high magnification.

Description

Technical Field

[0001] The present application belongs to the technical field of sub-pixel image registration, and in particular, relates to a sub-pixel image registration method, apparatus and terminal device. Background Art

[0002] Phase correlation method is a commonly used image registration algorithm. Hassan Foroosh et al. extended the correlation phase method to sub-pixel image registration. The algorithm has good registration accuracy and noise resistance for the case where only translation transformation exists, and the amount of calculation is small, which can be used for real-time position drift compensation. Its characteristics are very suitable for position drift correction in photothermal reflection microscopy.

[0003] The above method was introduced into the photothermal reflection microscopy thermal imaging system, and temperature tests were carried out on several devices including GaN HEMT at a relatively low magnification, with satisfactory results. However, when the magnification was further increased to observe the details of channel heating, the algorithm sometimes failed and the displacement could not be detected. Summary of the invention

[0004] In order to overcome the technical problem of algorithm failure existing in the related art, the embodiments of the present application provide a sub-pixel image registration method, apparatus and terminal device.

[0005] This application is implemented through the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a sub-pixel image registration method, comprising:

[0007] Acquire a reference image of the target object and an acquired image to be registered;

[0008] Performing Fourier transform on the acquired image to be registered and the reference image;

[0009] The Fourier-transformed acquired image to be registered is subjected to band-limiting processing to filter out invalid information in the acquired image to be registered;

[0010] The acquired images to be registered that have undergone band-limited processing are registered based on the reference image.

[0011] In combination with the first aspect, in some possible implementations, obtaining a reference image of the target object includes: obtaining a temporary reference image, which is any frame image of the target object, or is obtained by averaging multiple frames of unregistered images; collecting multiple frames of images of the target object, and aligning the multiple frames based on the temporary reference image; and generating a reference image based on averaging the registered multiple frames.

[0012] In combination with the first aspect, in some possible implementations, Fourier transform is performed on the image to be registered, including: calculating the Fourier transform G(u,v) of the image to be registered g(x,y) and the Fourier transform R(u,v) of the reference image r(x,y); calculating the phase function of the cross-correlation between the Fourier transform G(u,v) and the Fourier transform R(u,v) Among them, R * (u,v) is the complex conjugate function of R(u,v), u is the spatial frequency corresponding to x after Fourier transform, and v is the spatial frequency corresponding to y after Fourier transform, where x, y, u, and v are all integers, and x∈[0,N x -1],u∈[0,N x -1],y∈[0,N y -1],v∈[0,N y -1],N x N is the total number of pixels in the x direction of the image g(x,y) to be registered. y is the total number of pixels in the y direction of the image g(x,y) to be registered.

[0013] In combination with the first aspect, in some possible implementations, the Fourier transformed acquired image to be registered is subjected to band-limiting processing to filter out invalid information in the acquired image to be registered, including: based on Fourier transform C p (u,v) is band-limited; where W u is the bandwidth of u, W v is the bandwidth of v.

[0014] In combination with the first aspect, in some possible implementations, the band-limited acquired image to be registered is registered based on the reference image, including: determining a translation amount from the acquired image to be registered to the reference image, where the translation amount includes an integer part and a decimal part.

[0015] In conjunction with the first aspect, in some possible implementations, the process of determining the integer part of the translation amount includes: L (u,v) is inversely transformed to obtain c L (x,y); the highest point in the inverse Fourier transform result (x I ,y I ), determined as the integer part, where x I is the integer part of the translation in the x direction, y I The integer part of the translation in the y direction.

[0016] In conjunction with the first aspect, in some possible implementations, the process of determining the fractional part of the translation amount includes: L (u,v) is inversely transformed to obtain cL (x,y); Determine the fractional part x based on the highest point and adjacent points in the inverse Fourier transform result F and F :

[0017]

[0018]

[0019] Among them, x F is the decimal part of the x-direction translation, y F The decimal part of the y-direction translation.

[0020] In conjunction with the first aspect, in some possible implementations, the process of determining the fractional part of the translation amount includes: L (u,v) is inverse Fourier transformed to obtain c L (x,y); Determine the fractional part x based on the peak point and the kth point on both sides of the peak point in the inverse Fourier transform result F and F :

[0021]

[0022]

[0023] Among them, k X for Round up, k Y for Round up.

[0024] In a second aspect, an embodiment of the present application provides a sub-pixel image registration method and apparatus, including:

[0025] An image acquisition module is used to acquire a reference image of a target object and an image to be registered of the target object;

[0026] A computing module, used for performing Fourier transform on the acquired image to be registered and the reference image;

[0027] The band-limited processing module is used to perform band-limited processing on the acquired image to be registered after Fourier transformation, and filter out the bandwidth that does not contain valid information in the acquired image to be registered;

[0028] The registration module is used to register the acquired image to be registered after the band-limited processing based on the reference image.

[0029] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the sub-pixel image registration method as described in any one of the first aspects is implemented.

[0030] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the sub-pixel image registration method as described in any one of the first aspects is implemented.

[0031] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the sub-pixel image registration method described in any one of the above-mentioned first aspects.

[0032] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0033] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0034] In the embodiment of the present application, invalid information in the acquired image to be registered is filtered out through frequency domain band limiting processing, and then the acquired image to be registered that has undergone band limiting processing is registered based on the reference image to solve the problem of algorithm failure. This can support testing under high magnification conditions and can observe more temperature distribution details, thereby improving the reference value of the test results.

[0035] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0037] Figure 1 is a flowchart of a sub-pixel image registration method provided in an embodiment of the present application;

[0038] Figure 2 is a schematic diagram of the structure of a sub-pixel image registration device provided in an embodiment of the present application;

[0039] Figure 3 It is a structural schematic diagram of a sub-pixel image registration terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0041] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0042] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0043] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0044] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0045] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0046] Phase correlation method is a commonly used image registration algorithm. Hassan Foroosh et al. extended the correlation phase method to sub-pixel image registration. The algorithm has good registration accuracy and anti-noise ability for the case where only translation transformation exists, and the amount of calculation is small, which can be used for real-time position drift compensation. Its characteristics are very suitable for position drift correction in photothermal reflection microscopy. The above method was introduced into the photothermal reflection microscopy thermal imaging system, and temperature tests were carried out on several devices including GaN HEMT at a relatively low magnification, with satisfactory results. However, when the magnification was further increased to observe the details of channel heating, the algorithm sometimes failed and the displacement could not be detected.

[0047] To facilitate understanding of the problems existing in the related art, the principle of estimating the translation amount using the phase correlation method is first introduced below.

[0048] Assume that the reading of each point in the reference image is s(x,y), and the image to be registered is g(x,y). There is only a translation transformation between the two images, that is, g(x,y)=s(x-Δx,y-Δy), where Δx,Δy are the translation amounts.

[0049] Shifting in the spatial domain is equivalent to convolution with δ(x-Δx, y-Δy), which corresponds to multiplying a phase in the frequency domain.

[0050] Among them, u and v are the spatial frequencies corresponding to x and y after Fourier transform.

[0051]

[0052] Among them, M and Q are the number of pixels in the x and y directions respectively.

[0053]

[0054] Spatial cross-correlation corresponds to frequency domain

[0055]

[0056] Among them, S * (u,v) is the complex conjugate function of S(u,v), S(u,v) is the Fourier transform of s(x,y), and G(u,v) is the Fourier transform of g(x,y).

[0057] because

[0058]

[0059]

[0060]

[0061] Therefore, the Fourier transform of the cross-correlation function is

[0062]

[0063] After normalization, only the phase information is left, which is called phase correlation.

[0064]

[0065] The inverse transform obtains an impulse function in the spatial domain

[0066] c p (x,y)=δ(x-Δx,y-Δy)

[0067] The coordinates corresponding to the function peak are the translation amount

[0068]

[0069] For simplicity, the above principle is discussed under the condition of continuous function and unrestricted domain. For digital images, each variable should be discrete and have a limited range of values. Hassan Foroosh et al. found through analysis that for digital images, the actual c p (x, y) is similar to the translated sinc sampled at integer points, so it can be based on c p The function value near the (x, y) peak point is used to estimate the sub-pixel translation to obtain (Δx, Δy). Taking the case where only sub-pixel translation exists as an example, assuming that the main peak is c p (0,0), the two secondary peaks are c p (1,0) and c p (0,1), then the translation Δx in the x direction and the translation Δy in the y direction can be calculated as follows

[0070]

[0071]

[0072] The ± sign should be chosen so that the estimated translation falls between the main peak and the secondary peak. In the above example, the translations in both directions should be in the interval (0,1).

[0073] When the principle of estimating the translation amount using the above-mentioned phase correlation method is used to further increase the magnification to observe the details of channel heating, the algorithm sometimes fails and the displacement cannot be detected.

[0074] The specific phenomena and principles are as follows:

[0075] The above algorithm works normally when used at a lower magnification, such as 20X. For the same area of ​​the same object under test (the center position of the field of view is the same, that is, the image at high magnification is a capture of the center part of the image at low magnification), when the magnification is increased (in the hope of further observing details), such as 50X, the algorithm fails. It can be observed that when obvious translation occurs, the estimated value of the translation output by the algorithm is always close to zero.

[0076] Further investigation of the algorithm's intermediate data revealed that when the algorithm fails, c p (x,y) has a peak at the position corresponding to the actual translation, but there is another larger peak at the origin (0,0) that masks it, causing the algorithm to make a wrong judgment. Further explore the source of the origin peak and observe C p (u,v) found that the phase of effective information is concentrated in the low-frequency part, and the high-frequency part is dominated by noise. Considering c p The peak value of (x,y) at the origin, in the frequency domain C p (u,v) should be a fixed zero phase that is independent of (u,v). Therefore, further observation of the phase in the noise region revealed that it was not a uniform distribution corresponding to white noise, but a bell-shaped distribution with zero as the center. This was determined to be the direct cause of the algorithm failure. As a verification, S(u,v) and C were extracted. p The high frequency parts of (u,v) are inversely transformed to obtain the result C p After the inverse transformation of the high-frequency part of (u,v), there is an obvious peak at the origin, while after the inverse transformation of the high-frequency part of S(u,v), the value at the origin is not much different from that of the adjacent points.

[0077] It is speculated that the mechanism of this phenomenon is related to the inconsistent response of each pixel of the camera. From the spatial domain, the response of each pixel has a small random fluctuation, and its influence is similar to white noise. Therefore, the phase corresponding to its influence in S(u,v) is uniformly distributed. However, since the pixel position is fixed, the correlation between the two images is p As the magnification of the optical system increases, the high-frequency components in the resulting image decrease, resulting in C p The proportion of effective phase signals in (u,v) decreases, and the above-mentioned influence becomes increasingly prominent, causing the algorithm to fail.

[0078] Based on the above problems, the sub-pixel image registration method in the embodiment of the present application filters out invalid information in the acquired image to be registered through frequency domain band limiting processing, and then aligns the acquired image to be registered after band limiting processing based on the reference image to solve the algorithm failure problem. It can support testing under high magnification conditions and can observe more temperature distribution details, thereby improving the reference value of the test results.

[0079] To facilitate understanding of the sub-pixel image registration method provided in the embodiment of the present application, the principle of the sub-pixel image registration method is described below.

[0080] According to the previous analysis, the direct cause of the algorithm failure is the lack of effective high-frequency components in the images collected at high magnification. Therefore, band-limited processing can be performed in the frequency domain to filter out the high-frequency part without effective signals, thereby suppressing c p (x,y) interferes with the signal at the origin. p (u,v) will result in c p The corresponding sinc of (x, y) is broadened, and a new relationship is derived below to calculate the translation amount based on the values ​​of the peak value and its two adjacent points (-1, 0, 1).

[0081] Also, let the reference image be

[0082]

[0083] Where S(u,v) is the Fourier transform of s(x,y)

[0084] The image after translation is

[0085]

[0086] Among them, x, y, u, v are all integers, and x∈[0,N x -1],u∈[0,N x -1],y∈[0,N y -1],v∈[0,N y -1],N x N is the total number of pixels in the x direction of the image g(x,y) to be registered. y is the total number of pixels in the y direction of the image g(x,y) to be registered.

[0087] The following analysis takes one dimension as an example, and the two-dimensional case is similar.

[0088] C p (u) Band-limited processing is performed, the total number of points is N, and the reserved bandwidth is W, then

[0089]

[0090] The spatial domain function after inverse transformation is as follows, that is, the frequency domain band limitation causes the spatial domain sinc function to be broadened.

[0091]

[0092] c p The coordinates of the (x) peak are the integer part of the estimated translation value.

[0093] The method for obtaining the decimal part is that the pixel peak point is c p (0), the adjacent points on both sides are c p (-1) and c p (1), Δx∈(-0.5,0.5), then

[0094]

[0095] Arrangement available

[0096]

[0097] Further sorting and solving can get

[0098]

[0099] According to this, c p (x) The function values ​​at the peak point and its two sides are used to calculate the sub-pixel translation.

[0100] Figure 2 is a schematic flow chart of a sub-pixel image registration method provided in an embodiment of the present application, referring to Figure 2 , the sub-pixel image registration method is described in detail as follows:

[0101] In step 101, a reference image of a target object and an acquired image to be registered are obtained.

[0102] Exemplarily, obtaining a reference image of a target object includes: obtaining a temporary reference image, which is any frame image of the target object, or is obtained by averaging multiple frames of unregistered images; collecting multiple frames of images of the target object, and aligning the multiple frames based on the temporary reference image; and generating a reference image based on averaging the registered multiple frames.

[0103] In step 102, Fourier transform is performed on the acquired image to be registered and the reference image.

[0104] Exemplarily, performing Fourier transform on the image to be registered includes: calculating the Fourier transform G(u,v) of the image to be registered g(x,y) and the Fourier transform R(u,v) of the reference image r(x,y); calculating the phase function of the cross-correlation between the Fourier transform G(u,v) and the Fourier transform R(u,v) Among them, R * (u,v) is the complex conjugate function of R(u,v), u is the spatial frequency corresponding to x after Fourier transform, v is the spatial frequency corresponding to y after Fourier transform, and x∈[0,N x -1],u∈[0,N x -1],y∈[0,N y-1],v∈[0,N y -1],N x N is the total number of pixels in the x direction of the image g(x,y) to be registered. y is the total number of pixels in the y direction of the image g(x,y) to be registered.

[0105] In step 103, the Fourier-transformed acquired image to be registered is subjected to band-limiting processing to filter out invalid information in the acquired image to be registered.

[0106] Exemplarily, the invalid information in the acquired image to be registered is: a fixed zero phase that is independent of the frequency domain and distributed approximately in a bell shape with the zero point as the center in the frequency domain.

[0107] Exemplarily, the Fourier transformed acquired image to be registered is subjected to band-limiting processing to filter out invalid information in the acquired image to be registered, including: based on Fourier transform C p (u,v) is band-limited; where W u is the bandwidth of u, W v is the bandwidth of v.

[0108] In step 104, the acquired image to be registered that has undergone band-limiting processing is registered based on the reference image.

[0109] Exemplarily, registering the acquired image to be registered after band-limiting processing based on the reference image includes: determining a translation amount from the acquired image to be registered to the reference image, where the translation amount includes an integer part and a decimal part.

[0110] Exemplarily, the process of determining the integer part of the translation includes: L (u,v) is inversely transformed to obtain c L (x,y); the highest point in the inverse Fourier transform result (x I ,y I ), determined as the integer part, where x I is the integer part of the translation in the x direction, y I The integer part of the translation in the y direction.

[0111] Exemplarily, the process of determining the fractional part of the translation includes: L (u,v) is inversely transformed to obtain c L (x,y); Determine the fractional part x based on the highest point and adjacent points in the inverse Fourier transform result F and F :

[0112]

[0113]

[0114] Among them, x F is the decimal part of the x-direction translation, y F The decimal part of the y-direction translation.

[0115] Optionally, the process of determining the fractional part of the translation amount further includes: L (u,v) is inversely transformed to obtain c L (x,y); Based on the peak point and the kth point on both sides of the peak point in the inverse Fourier transform result, determine the fractional part x F and F :

[0116]

[0117]

[0118] Among them, k X For Round up, k Y For Round up.

[0119] The above sub-pixel image registration method solves the algorithm failure problem through frequency domain band-limited processing, and provides an operational formula for calculating the estimated value of the translation, which is used for translation correction of photothermal reflection microscopy imaging. It can support testing under high magnification conditions and can observe more temperature distribution details, thereby improving the reference value of the test results.

[0120] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0121] Corresponding to the sub-pixel image registration method described in the above embodiment, Figure 3 A structural block diagram of a sub-pixel image registration device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0122] See also Figure 2 The sub-pixel image registration device in the embodiment of the present application may include: an image acquisition module 301, a calculation module 302, a band-limited processing module 303, and a registration module 304.

[0123] Optionally, the image acquisition module 301 is specifically used to acquire a reference image of the target object and an acquired image to be registered.

[0124] Exemplarily, obtaining a reference image of a target object includes: obtaining a temporary reference image, where the temporary reference image is any frame image, or is obtained by averaging multiple frames of unregistered images; collecting multiple frames of images of the target object, and aligning the multiple frames of images based on the temporary reference image; and generating a reference image based on averaging the registered multiple frames of images.

[0125] Optionally, the operation module 302 is specifically used to perform Fourier transform on the acquired image to be registered and the reference image.

[0126] Exemplarily, performing Fourier transform on the image to be registered includes: calculating the Fourier transform G(u,v) of the image to be registered g(x,y) and the Fourier transform R(u,v) of the reference image r(x,y); calculating the phase function of the cross-correlation between the Fourier transform G(u,v) and the Fourier transform R(u,v) Among them, R * (u,v) is the complex conjugate function of R(u,v), u is the spatial frequency corresponding to x after Fourier transform, and v is the spatial frequency corresponding to y after Fourier transform, where x, y, u, and v are all integers, and x∈[0,N x -1],u∈[0,N x -1],y∈[0,N y -1],v∈[0,N y -1],N x N is the total number of pixels in the x direction of the image g(x,y) to be registered. y is the total number of pixels in the y direction of the image g(x,y) to be registered.

[0127] Optionally, the band-limited processing module 303 is specifically used to perform band-limited processing on the acquired image to be registered after Fourier transformation, so as to filter out invalid information in the acquired image to be registered.

[0128] Exemplarily, the Fourier-transformed acquired image to be registered is subjected to band-limiting processing to filter out the bandwidth containing no valid information in the acquired image to be registered, including: based on Fourier transform C p (u,v) is band-limited; where W u is the bandwidth of u, W v is the bandwidth of v.

[0129] Optionally, the registration module 304 is specifically used to: register the acquired image to be registered that has undergone band-limiting processing based on the reference image.

[0130] Exemplarily, registering the acquired image to be registered after band-limiting processing based on the reference image includes: determining a translation amount from the acquired image to be registered to the reference image, where the translation amount includes an integer part and a decimal part.

[0131] Exemplarily, the process of determining the integer part of the translation includes: L (u,v) is inverse Fourier transformed to obtain c L (x,y); the highest point in the inverse Fourier transform result (x I ,y I ), determined as the integer part, where x I is the integer part of the translation in the x direction, y I The integer part of the translation in the y direction.

[0132] Exemplarily, the process of determining the fractional part of the translation includes: L (u,v) is inverse Fourier transformed to obtain c L (x,y); Determine the fractional part x based on the highest point and adjacent points in the inverse Fourier transform result F and F :

[0133]

[0134]

[0135] Among them, x F is the decimal part of the x-direction translation, y F The decimal part of the y-direction translation.

[0136] Optionally, the process of determining the fractional part of the translation amount further includes: L (u,v) is inverse Fourier transformed to obtain c L (x,y); Based on the peak point and the kth point on both sides of the peak point in the inverse Fourier transform result, determine the fractional part x F and F :

[0137]

[0138]

[0139] Among them, k X For Round up, k Y For Round up.

[0140] For example, the translation amount in the x direction Δx=x I +x F , the translation in the y direction Δy = y I +y F

[0141] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0142] The present application also provides a terminal device, see Figure 3 The terminal device 500 may include: at least one processor 510, a memory 520, and a computer program stored in the memory 520 and executable on the at least one processor 510. When the processor 510 executes the computer program, the steps in any of the above-mentioned method embodiments are implemented, for example Figure 1 Steps 101 to 104 in the illustrated embodiment. Alternatively, when the processor 510 executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Figure 2 Functions of modules 301 to 304 are shown.

[0143] Exemplarily, the computer program may be divided into one or more modules / units, one or more modules / units are stored in the memory 520, and executed by the processor 510 to complete the present application. The one or more modules / units may be a series of computer program segments capable of completing specific functions, and the program segments are used to describe the execution process of the computer program in the terminal device 500.

[0144] Those skilled in the art will understand that Figure 3 It is only an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, such as input and output devices, network access devices, buses, etc.

[0145] The processor 510 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0146] The memory 520 may be an internal storage unit of the terminal device, or an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. The memory 520 is used to store the computer program and other programs and data required by the terminal device. The memory 520 may also be used to temporarily store data that has been output or is to be output.

[0147] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0148] The sub-pixel image registration method provided in the embodiment of the present application can be applied to terminal devices such as computers, tablet computers, laptop computers, and mobile phones. The embodiment of the present application does not impose any restrictions on the specific type of terminal devices.

[0149] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in each embodiment of the above-mentioned sub-pixel image registration method can be implemented.

[0150] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in each embodiment of the above-mentioned sub-pixel image registration method when executing the computer program product.

[0151] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0152] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0153] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0154] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0155] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0156] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A sub-pixel image registration method, characterized in that: include: Acquire a reference image of the target object and an acquired image to be registered; Performing Fourier transform on the acquired image to be registered and the reference image; Performing band-limiting processing on the acquired image to be registered after Fourier transformation to filter out invalid information in the acquired image to be registered; Based on the reference image, register the acquired image to be registered after the band-limited processing to obtain a fractional part of the translation amount; The process of determining the fractional portion of the translation includes: C L (u,v) is inverse Fourier transformed to obtain c L (x,y); Based on the function values ​​of the highest point and the adjacent points in the inverse Fourier transform result, the fractional part x is determined. F and F : Among them, x F is the decimal part of the x-direction translation, y F is the fractional part of the translation in the y direction; Alternatively, the process of determining the fractional part of the translation comprises: C L (u,v) is inverse Fourier transformed to obtain c L (x,y); Based on the peak point in the inverse Fourier transform result and the kth point on both sides of the peak point, the fractional part x is determined. F and F : Among them, k X For Round up, k Y For Round up.

2. The sub-pixel image registration method according to claim 1, characterized in that: The obtaining of a reference image of the target object comprises: Acquire a temporary reference image, where the temporary reference image is any frame image of the target object, or is obtained by averaging multiple frames of unregistered images; Acquire multiple frames of images of the target object, and register the multiple frames of images based on the temporary reference image; The reference image is generated based on the average of the multiple frames of images after registration.

3. The sub-pixel image registration method according to claim 1, characterized in that: The performing Fourier transform on the acquired image to be registered comprises: Calculate the Fourier transform G(u,v) of the acquired image to be registered g(x,y) and the Fourier transform R(u,v) of the reference image r(x,y); Calculate the phase function of the cross-correlation of the Fourier transform G(u,v) and the Fourier transform R(u,v) Among them, R * (u,v) is the complex conjugate function of R(u,v), u is the spatial frequency corresponding to x after Fourier transform, and v is the spatial frequency corresponding to y after Fourier transform, where x, y, u, and v are all integers, and x∈[0,N x -1],u∈[0,N x -1],y∈[0,N y -1],v∈[0,N y -1],N x N is the total number of pixels in the x direction of the image g(x,y) to be registered. y is the total number of pixels in the y direction of the image g(x,y) to be registered; The band-limiting processing is performed on the Fourier-transformed acquired image to be registered to filter out invalid information in the acquired image to be registered, including: based on The Fourier transform C p (u, v) is subjected to the band-limiting processing; wherein W u is the bandwidth of u, W v is the bandwidth of v.

4. The sub-pixel image registration method according to claim 3, characterized in that: The registering the acquired image to be registered after the band-limited processing based on the reference image comprises: The translation amount from the acquired image to be registered to the reference image is determined, where the translation amount includes an integer part and a decimal part.

5. The sub-pixel image registration method according to claim 4, characterized in that: The process of determining the integer part of the translation includes: C L (u,v) is inverse Fourier transformed to obtain c L (x,y); The highest point (x I ,y I ), determined as the integer part, where x I is the integer part of the translation in the x direction, y I The integer part of the translation in the y direction.

6. A sub-pixel image registration device, characterized in that: include: An image acquisition module, used to acquire a reference image of a target object and an image to be registered of the target object; A computing module, used for performing Fourier transform on the acquired image to be registered and the reference image; A band-limited processing module is used to perform band-limited processing on the acquired image to be registered after Fourier transformation, so as to filter out invalid information in the acquired image to be registered; A registration module, used for registering the acquired image to be registered after the band-limiting processing based on the reference image to obtain a fractional part of the translation amount; The process of determining the fractional portion of the translation includes: C L (u,v) is inverse Fourier transformed to obtain c L (x,y); Based on the function values ​​of the highest point and the adjacent points in the inverse Fourier transform result, the fractional part x is determined. F and F : Among them, x F is the decimal part of the x-direction translation, y F is the fractional part of the translation in the y direction; Alternatively, the process of determining the fractional part of the translation comprises: C L (u,v) is inverse Fourier transformed to obtain c L (x,y); Based on the peak point in the inverse Fourier transform result and the kth point on both sides of the peak point, the fractional part x is determined. F and F : Among them, k X For Round up, k Y For Round up.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images