Image registration processing method and device

Abstract

The present application discloses an image registration processing method and device, belonging to the field of image processing technology. The method comprises: obtaining a global translation vector between an image to be registered and a reference image in a multi-exposure image sequence; performing block processing on the image to be registered to obtain multiple image sub-blocks; calculating a local translation vector of each of the multiple image sub-blocks to obtain multiple local translation vectors; and performing registration processing on the image to be registered based on the global translation vector and the multiple local translation vectors.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image registration processing method and device. Background Art

[0002] In order to meet people's increasingly high shooting requirements for electronic devices such as mobile phones and tablets, such as high dynamic range, low illumination, large depth of field and wide field of view, multiple exposure technology is applied to the shooting function of electronic devices. Since the electronic device and the target scene are prone to relative movement during the multiple exposure time period, the multiple exposure images generally cannot achieve pixel-level or even sub-pixel matching, so it is necessary to align the images in the multiple exposure image sequence. However, the current registration of images in the multiple exposure image sequence usually has the problem of low registration accuracy. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide an image registration processing method and device, which can solve the problem of low registration accuracy in the current registration of images in a multi-exposure image sequence.

[0004] In a first aspect, an embodiment of the present application provides an image registration processing method, the method comprising:

[0005] Obtaining a global translation vector between the image to be registered and the reference image in a multi-exposure image sequence;

[0006] Performing block processing on the image to be registered to obtain multiple image sub-blocks;

[0007] Calculating a local translation vector of each of the plurality of image sub-blocks to obtain a plurality of local translation vectors;

[0008] Based on the global translation vector and the multiple local translation vectors, the image to be registered is registered.

[0009] In a second aspect, an embodiment of the present application provides an image registration processing device, the device comprising:

[0010] A global translation vector acquisition module is used to acquire a global translation vector between the image to be registered and the reference image in a multi-exposure image sequence;

[0011] A block division module, used for performing block processing on the image to be registered to obtain multiple image sub-blocks;

[0012] A local translation vector calculation module, used to calculate the local translation vector of each of the multiple image sub-blocks to obtain multiple local translation vectors;

[0013] A registration module is used to perform registration processing on the image to be registered based on the global translation vector and the multiple local translation vectors.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0017] In an embodiment of the present application, during the process of the electronic device performing registration processing on a multi-exposure image sequence, the global translation vector of the image to be registered is calculated, and the feature sequence of the image to be registered is updated based on the global translation vector, so as to obtain a first image after the first registration processing of the image to be registered, and on the basis of the first image, the first image after the first registration processing is further registered at least once, and each registration processing is implemented by the translation vector of the image sub-region of the image to be registered, so as to obtain the final registered image. In this way, through the image registration processing method provided by the present application, through multiple registration processing, and taking both the overall and the local into consideration in the multiple registration processing, the accuracy of the registration of the multi-exposure image sequence can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of an image registration processing method provided in an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of removing a local displacement vector with noise provided by an embodiment of the present application;

[0020] Figure 3 This is one of the distribution diagrams of the center points of the image sub-blocks provided in the embodiment of the present application;

[0021] Figure 4 This is the second distribution diagram of the center points of the image sub-blocks provided in the embodiment of the present application;

[0022] Figure 5 This is the third distribution diagram of the center points of the image sub-blocks provided in the embodiment of the present application;

[0023] Figure 6 is a schematic diagram of an image before iterative registration provided in an embodiment of the present application;

[0024] Figure 7 is a schematic diagram of an image after iterative registration provided in an embodiment of the present application;

[0025] Figure 8 It is a structural schematic diagram of an image registration processing device provided in an embodiment of the present application;

[0026] Fig. 9 This is one of the hardware structure diagrams of the electronic device provided in the embodiment of the present application;

[0027] Fig.10 This is the second schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0029] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0030] The image registration processing method provided by the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0031] See also Figure 1 , is a flow chart of an image registration processing method provided by an embodiment of the present application, which is applied to electronic devices such as Figure 1 As shown, the above image registration processing method includes the following steps:

[0032] Step 101, obtaining a global translation vector between an image to be registered and a reference image in a multi-exposure image sequence;

[0033] Step 102, performing block processing on the image to be registered to obtain multiple image sub-blocks;

[0034] Step 103: Calculate the local translation vector of each of the multiple image sub-blocks to obtain multiple local translation vectors;

[0035] Step 104: Perform registration processing on the image to be registered based on the global translation vector and the multiple local translation vectors.

[0036] Based on this, in the process of the electronic device performing registration processing on the multi-exposure image sequence, by obtaining the global translation vector between the image to be registered and the reference image, and performing block processing on the image to be registered, a plurality of image sub-blocks are obtained, and then the image to be registered is registered based on the global translation vector and the local translation vectors of the plurality of image sub-blocks. In this way, the image registration processing method provided by the present application can effectively improve the accuracy of the registration of the multi-exposure image sequence.

[0037] In the above step 101, when the electronic device registers the multi-exposure image sequence, the electronic device may obtain a global translation vector between the image to be registered and the reference image.

[0038] The multi-exposure image sequence may include at least two images captured under different exposure parameters, and the exposure parameter may include at least one of exposure value, depth of field value, spectrum type, etc.

[0039] In addition, the reference image may be any one of the at least two images in the multi-exposure image sequence, and the image to be registered may be any one of the at least two images except the reference image.

[0040] In an embodiment of the present application, the above-mentioned acquisition of the global translation vector between the image to be registered and the reference image in the multi-exposure image sequence can be to determine the matching feature points or feature lines in the reference image and the image to be registered, and extract the coordinate information of the matching feature points or feature lines from the feature sequence of the reference image and the feature sequence of the image to be registered, and calculate the above-mentioned global translation vector through the extracted coordinate information.

[0041] Alternatively, the above step 101 may include:

[0042] Acquire a first bitmap sequence of images to be registered in a multi-exposure image sequence, and acquire a second bitmap sequence of reference images in the multi-exposure image sequence;

[0043] Based on the first bitmap sequence and the second bitmap sequence, a global translation vector between the image to be registered and the reference image is calculated, wherein the global translation vector represents: when the bitmap of the first bitmap sequence is translated, the translation amount of the bitmap of the first bitmap sequence when the similarity between the bitmap of the first bitmap sequence and the bitmap of the second bitmap sequence after translation is the largest.

[0044] Based on this, through the above-mentioned first bitmap sequence of the image to be registered and the second bitmap sequence of the reference image, when the first bitmap sequence is translated and the similarity between the translated first bitmap sequence and the second bitmap sequence is the largest, the translation amount of the bitmap of the first bitmap sequence is used as the above-mentioned global translation vector, thereby making the global translation vector between the image to be registered and the reference image more accurate, further improving the registration accuracy.

[0045] The first bitmap sequence of the image to be registered may be obtained by converting pixels of the image to be registered into bitmap values ​​through any bitmap conversion method.

[0046] Specifically, the above-mentioned obtaining of the first bitmap sequence of the image to be registered in the multi-exposure image sequence may include:

[0047] Compare the brightness of each pixel in the image to be registered with the target brightness to determine the bitmap value of each pixel in the image to be registered, wherein the target brightness is the brightness of the image to be registered at the Kth percentile, and K is an integer greater than 1;

[0048] A first bitmap sequence of the image to be registered is generated through the bitmap value of each pixel in the image to be registered.

[0049] Based on this, the brightness of each pixel in the image to be registered is compared with the brightness of the image to be registered at the K percentile to determine the bitmap value of each pixel in the image to be registered, and then the bitmap sequence of the image to be registered is generated by determining the bitmap value of each pixel, so that the generated bitmap sequence is more accurate and the registration accuracy is further improved.

[0050] Among them, the brightness of each pixel in the image to be registered is compared with the target brightness to determine the bitmap value of each pixel in the image to be registered. When the brightness of the pixel is greater than or equal to the target brightness, the bitmap value of the pixel is set to 1; and when the brightness of the pixel is less than the target brightness, the bitmap value of the pixel is set to 0, that is, the image to be registered is subjected to K-bit binarization processing.

[0051] In addition, the above-mentioned generation of the first bitmap sequence of the image to be registered by the bitmap value of each pixel in the image to be registered can be understood as replacing the brightness of each pixel in the image to be registered with its bitmap value to obtain the first bitmap sequence of the image to be registered.

[0052] It should be noted that the above-mentioned target brightness is the brightness of the above-mentioned percentile in the image to be registered, which can be understood as sorting the brightness of all pixel points in the image to be registered, and the brightness of the position between the first K percentiles and the last K percentiles in the sorting is determined as the above-mentioned target brightness (if the image to be registered is underexposed, it is the first K percentiles, and if the image to be registered is overexposed, it is the last K percentiles).

[0053] For example, in the above case of K=2, the middle value in the brightness ranking can be determined as the target brightness; in the above case of K=4, the brightness at one quarter or three quarters of the brightness ranking can be determined as the above target brightness, and so on.

[0054] In addition, the above K may be a value pre-set according to actual needs; or, before comparing the brightness of each pixel in the image to be registered with the target brightness, the above method may further include:

[0055] Calculate the area ratio of the abnormal exposure image region in the image to be registered in the image to be registered;

[0056] A value corresponding to the area ratio is determined as the K.

[0057] Based on this, the above K is determined by the area ratio of the abnormal exposure image area in the image to be registered in the registered image, so that the determined K value is more reasonable and the acquired bitmap sequence is further improved to be more accurate.

[0058] The abnormal exposure image region may be at least one of an overexposed image region and an underexposed image region.

[0059] In addition, the above-mentioned value corresponding to the area ratio is determined as K. It can be that the electronic device has a preset correspondence between different K and different preset area ratio ranges, and the value that has a corresponding relationship with the preset area ratio range in which the calculated area ratio is located is determined as the above-mentioned K.

[0060] It should be noted that the implementation process of acquiring the second bitmap sequence of the reference image is the same as the implementation process of acquiring the first bitmap sequence of the image to be registered, which will not be described in detail here.

[0061] In addition, in order to reduce the influence of overexposed image areas and underexposed image areas on registration, the above-mentioned image to be registered can be an image whose target brightness is greater than the underexposed threshold and less than the overexposed threshold, and an image whose target brightness is less than or equal to the underexposed threshold or greater than or equal to the overexposed threshold can be eliminated during the registration of the multi-exposure image sequence and not be registered as the above-mentioned image to be registered. Here, the underexposed threshold is the brightness used to determine whether the image is underexposed, and the overexposed threshold is the brightness used to determine whether the image is overexposed.

[0062] To facilitate understanding of the process of obtaining a bitmap sequence through the above-mentioned K-percentile binarization, an example is given here for explanation. The example is to obtain a multi-exposure image sequence {I k} of the bitmap sequence {B k The process of} is as follows:

[0063] If the input multi-exposure image sequence is a color image, then the brightness channel sequence {L k}; If the input image is a single-channel grayscale image, it is directly recorded as {L k Thanks to the monotonicity of the camera response function, the brightness relationship between pixels on the image will not change under different exposures. Based on this, the following formula (1) is used for K-quantile binarization:

[0064]

[0065] For the brightness channel sequence {L k}, and binarize the images with different exposure values ​​to obtain the bitmap sequence {B k}. Where s(K) represents the brightness of the extracted target image (i.e., the image to be registered) at the Kth percentile. For example, if K=2, s(K) represents the middle value of the target image brightness; if K=4, s(K) represents the brightness of the first quarter (underexposure) or the last three quarters (overexposure) of the target image brightness ranking.

[0066] The value of K can be derived according to the image content. H (overexposure threshold) and t L (Underexposure threshold) detects the area ratio of overexposure and underexposure regions in the multi-exposure image, as shown in formula (2):

[0067]

[0068] Where Ω (·) represents the area of ​​the region that meets a certain condition, r k That is, the ratio of the overexposed area to the underexposed area to the entire image area. kWhen it is less than 50%, that is, when the normal exposure area of ​​the image exceeds half, K=2 can be used; when r k When it is greater than 50%, that is, more than half of the image area is overexposed or underexposed, the value of K can be appropriately increased, typically K = 4; when r k When it is larger, it means that most areas in the image are overexposed or underexposed. In this case, the value of K can be increased or the registration process can be stopped according to the actual situation. In addition, in order to reduce the impact of overexposed or underexposed areas on the registration, if s(K)>t H Or s(K)<t L , you can directly stop the registration process.

[0069] In an embodiment of the present application, after obtaining the above-mentioned first bitmap sequence and second bitmap sequence, the electronic device can calculate the above-mentioned global translation vector through the first bitmap sequence and the second bitmap sequence, that is, the global translation vector is: when the bitmap of the first bitmap sequence is translated, the translation amount when the similarity between the bitmap of the translated first bitmap sequence and the bitmap of the second bitmap sequence is the largest.

[0070] For example, through the above K-bit binarization, a bitmap sequence of multi-exposure images {B k According to the monotonicity of the camera response function, the displacement vector between multiple exposure images can be calculated by the following formula (3):

[0071]

[0072] Where T(·) represents the translation operation, and the subscript c represents the reference bitmap number. The physical meaning of formula (3) is that the target bitmap B k d x ,d y ) After translation, it is consistent with the reference bitmap B c As similar as possible. Since this operation is done on the bitmap, the optimal displacement vector can be directly obtained by using the bitmap XOR method.

[0073] In the above step 102, the electronic device may perform block processing on the image to be registered to obtain a plurality of image sub-blocks of the image to be registered.

[0074] The above-mentioned block processing of the image to be registered may be to divide the image to be registered into a preset number of image sub-blocks, and the preset number may be set according to actual needs. Specifically, the image to be registered may be divided into n×n image sub-blocks, where n may be a positive integer greater than 2.

[0075] It should be noted that the above-mentioned block processing of the image to be registered is only a kind of image area division, and does not actually segment or transform the image itself.

[0076] In addition, the above step 102 may be performed after the above step 101, or before the above step 101, or may be performed simultaneously with the step 101. Figure 1 Only the case where step 102 is performed after step 101 is shown for illustration, and is not limited here.

[0077] Of course, when performing block processing on the above-mentioned image to be registered, the electronic device can also perform block processing on the above-mentioned reference image, dividing the reference image into the same number of image sub-blocks as the image to be registered, that is, dividing the reference image into multiple image sub-blocks.

[0078] In step 103, after the above-mentioned multiple image sub-blocks are obtained by division, the electronic device may calculate the local translation vector of each image sub-block in the multiple image sub-blocks to obtain the local translation vectors of the multiple image sub-blocks.

[0079] Among them, the above-mentioned calculation of the local translation vector of each image sub-block in the multiple image sub-blocks can be performed by calculating the translation vector of each image sub-block in the image to be registered relative to the matching image sub-block in the reference image through the coordinate information of the image sub-blocks in the image to be registered and the reference image, and using the calculated translation vector as the local translation vector of the image sub-block in the image to be registered.

[0080] Alternatively, calculating the local translation vector of each image sub-block in the plurality of image sub-blocks may include:

[0081] Acquire a first block boundary coordinate set of multiple image sub-blocks of the reference image, and acquire a second block boundary coordinate set of multiple image sub-blocks of the image to be registered based on the global translation vector; and calculate the local translation vectors of the multiple image sub-blocks of the image to be registered based on the first block boundary marking and the second block boundary coordinate set.

[0082] For example, the brightness channel sequence {L k} performs a block operation to divide a single image into n×n sub-blocks, where n=3, 4, 5…. In particular, the block of the reference brightness image (i.e., image sub-block) can be described as:

[0083] {P c}=S(L c )

[0084] Where S(·) represents the block operation, the subscript c represents the reference bitmap number, {P c} is the block boundary coordinate set of the reference brightness image, which does not represent the real image information. The block of the target brightness image can be described as:

[0085] k≠c

[0086] The difference from the reference brightness map is that after the block operation, a translation operation needs to be performed on the block boundary coordinate set. The purpose is to gradually accumulate the registration process to gradually reach a higher accuracy result.

[0087] After the n×n block operation described above, each pair of blocks P can be calculated in parallel k and P c The displacement vector between them, that is, n×n local displacement vectors are obtained, which are recorded as:

[0088]

[0089] The superscript i represents the number of the local displacement vector, and there are n 2 indivual.

[0090] In the above step 104, after acquiring the global translation vector of the above image to be registered and the above multiple local translation vectors, the electronic device may perform registration processing on the above image to be registered based on the global translation vector and the multiple local translation vectors.

[0091] Among them, the above-mentioned registration processing of the image to be registered based on the global translation vector and the multiple local translation vectors can be performed by calculating the spatial transformation matrix of the image to be registered through the global translation vector and the multiple local translation vectors, and updating the feature sequence of the image to be registered through the spatial transformation matrix to realize the registration processing of the image to be registered.

[0092] Alternatively, since there may be noise influence in the image capture (such as the sensor is affected by the outside world when working, or there are moving targets in the multiple exposure images, etc.), the spatial transformation matrix may be affected. Therefore, in order to eliminate the influence of noise, the above step 103 may include:

[0093] Based on the multiple local translation vectors, image sub-blocks with noise are removed from the multiple image sub-blocks to obtain multiple target image sub-blocks;

[0094] Determining a spatial transformation matrix of the image to be registered based on the global translation vector and the local translation vectors of the plurality of target image sub-blocks;

[0095] The image to be registered is registered using the space transformation matrix.

[0096] Based on this, the spatial transformation matrix of the image to be registered is determined by eliminating noisy image sub-blocks from multiple image sub-blocks and using the local translation vectors and global translation vectors of the remaining multiple target image sub-blocks, thereby making the determined spatial transformation matrix more accurate and further improving the registration accuracy.

[0097] Among them, the above-mentioned eliminating noisy image sub-blocks from multiple image sub-blocks based on the multiple local translation vectors can be carried out by a preset noisy image sub-block determination method, according to the local translation vectors of the multiple image sub-blocks, determining the noisy image sub-blocks from the multiple image sub-blocks, and then eliminating the noisy image sub-blocks.

[0098] For example, in Figure 2 As shown, assuming that there are 6 local displacement vectors from v1 to v6, by observing their distribution in the coordinate system, vectors v2 to v5 can be considered to have similar effects, while vectors v1 and v6 deviate from the main force and can be considered to be affected by noise, so vectors v1 and v6 are eliminated.

[0099] Specifically, the removing of the image sub-blocks having noise from the multiple image sub-blocks based on the multiple local translation vectors may include:

[0100] Calculating a first deviation of each local translation vector in the plurality of local translation vectors on a translation association index, wherein the translation association index includes at least one of a magnitude and a direction;

[0101] The image sub-blocks corresponding to the local translation vectors whose first deviations do not satisfy the first preset condition are eliminated.

[0102] Based on this, the image subblock with noise among multiple image subblocks is determined by the deviation of each local translation vector in at least one of the amplitude and direction, so that the image subblock with noise can be accurately determined, thereby making the determined spatial transformation matrix more accurate.

[0103] Among them, after calculating the above-mentioned first deviation, the electronic device can compare the first deviation with the preset deviation, and when it is determined that the first deviation is greater than or equal to the preset deviation, it is determined that the first deviation does not meet the first preset condition; and when it is determined that the first deviation is less than the preset deviation, it is determined that the first deviation meets the first preset condition.

[0104] Exemplarily, when the local displacement vectors of the n×n image sub-blocks are calculated, the amplitudes η of the n×n local displacement vectors are calculated. i and direction θ i(i is the sequence number of the image sub-block), and through the probability statistics of direction and amplitude, the displacement vectors that do not conform to the main force of the local displacement vector and have abnormal amplitude are eliminated. Specifically, the amplitude mean of all local displacement vectors is calculated. and standard deviation σ η , Directional Mean and standard deviation σ η , calculate the amplitude deviation t by formula (4) η The deviation from the direction t θ ,as follows:

[0105]

[0106] The deviation of the amplitude t η The deviation from the direction t θ , determine each local displacement vector Is it a noise vector? If t η and t θ If at least one of the items exceeds its corresponding threshold, the local displacement vector is the noise vector.

[0107] Alternatively, the removing of image sub-blocks having noise from the multiple image sub-blocks based on the multiple local translation vectors may include:

[0108] Calculating an average displacement vector of the multiple local translation vectors;

[0109] Calculating a second deviation between each of the local translation vectors and the average displacement vector;

[0110] The image sub-blocks corresponding to the local translation vectors whose second deviation is greater than or equal to the average deviation are eliminated.

[0111] Based on this, the image sub-block with noise in multiple image sub-blocks can also be determined by the deviation between each local translation vector and the average displacement vector, so that not only the image sub-block with noise can be accurately determined, but also the method is more flexible.

[0112] Exemplarily, the average displacement vector of the n×n local displacement vectors is calculated by formula (5):

[0113]

[0114] The physical meaning of formula (5) is to synthesize n×n local displacement vectors into a global displacement vector Then, based on the global displacement vector, the deviation e of each local displacement vector is calculated by formula (6): i :

[0115]

[0116] When the deviation of any local translation vector e i If the following formula (7) is not satisfied, it is regarded as a noise vector.

[0117]

[0118] In an embodiment of the present application, after the multiple target image sub-blocks are obtained by screening the multiple local translation vectors, the electronic device can determine the spatial transformation matrix of the image to be registered based on the global translation vector and the local translation vectors of the multiple target image sub-blocks.

[0119] Among them, the above-mentioned determination of the spatial transformation matrix of the image to be registered based on the global translation vector and the local translation vectors of multiple target image sub-blocks can be performed by calculating the transformation matrix of the image to be registered through the local translation vectors of multiple target image sub-blocks, and then taking the average vector of the global translation vector and the calculated transformation matrix as the above-mentioned spatial transformation matrix.

[0120] Alternatively, the determining the spatial transformation matrix of the image to be registered based on the global translation vector and the local translation vectors of the multiple target image sub-blocks may include:

[0121] Based on the center coordinate information of the multiple target image sub-blocks, determining whether the associated center points among the center points of the multiple target image sub-blocks are on a preset straight line, and obtaining a determination result;

[0122] Based on the judgment result, the global translation vector and the number of target image sub-blocks that are not on a preset straight line, a spatial transformation matrix of the image to be registered is determined.

[0123] Based on this, by judging whether the associated center points among the center points of multiple target image sub-blocks are on the same straight line, as well as the global translation vector and the number of target image sub-blocks that are not on the preset straight line, the spatial transformation matrix of the image to be registered is determined, thereby making the determined spatial transformation matrix more accurate.

[0124] The associated center points among the center points of the plurality of target image sub-blocks may be center points of image sub-blocks in the same row. For example, among n×n image sub-blocks, the center points of n image sub-blocks in the nth row are associated.

[0125] In addition, the preset straight line may be, when the reference image and the image to be registered are arranged in parallel, an extension of a straight line determined by the center points of a plurality of image sub-blocks in the same row in the reference image. For example, when the plurality of target image sub-blocks include at least two image sub-blocks in the nth row, the preset straight line is an extension of a straight line where the image sub-blocks in the nth row in the reference image are located.

[0126] Furthermore, the determining of the spatial transformation matrix of the image to be registered based on the judgment result, the global translation vector and the number of target image sub-blocks not on the same straight line may include:

[0127] In a case where the judgment result indicates that the associated central point is on a preset straight line, determining the global translation vector as a spatial transformation matrix of the image to be registered;

[0128] When the judgment result indicates that the associated center points are not on a preset straight line and the number is less than or equal to a preset value, the calculated affine transformation matrix is ​​used as the spatial transformation matrix;

[0129] When the judgment result indicates that the associated central points are not on a preset straight line and the number is greater than a preset value, the calculated projection transformation matrix is ​​used as the space transformation matrix.

[0130] Based on this, the corresponding spatial transformation matrix can be determined in the global translation vector, affine transformation matrix and projection transformation matrix according to whether the associated center points are on a preset straight line and the number of target image sub-blocks that are not on the same straight line, thereby further making the determined spatial transformation matrix more appropriate.

[0131] Exemplarily, the local displacement vector after local displacement vector screening is recorded as:

[0132] i=1,2…M,

[0133] There are M non-noise local displacement vectors, namely qualified local displacement vectors. Based on this, the spatial transformation relationship between the multi-exposure images can be estimated. The specific implementation method is as follows.

[0134] According to the reference brightness image block {P c}, combined with the displacement vector refinement, the centers of the M block areas corresponding to the M qualified local displacement vectors can be recorded as:

[0135] i=1,2…M

[0136] Similarly, according to the block {P k}, combined with the above screening of local displacement vectors, the centers of the M block areas corresponding to the M qualified local displacement vectors can be recorded as:

[0137] i=1,2…M

[0138] Then, M pairs of matching feature points can be established in the n×n block framework:

[0139]

[0140] Based on the distribution characteristics of the above M pairs of matching feature point pairs in the n×n block framework, select a suitable spatial transformation matrix. The specific selection method is as follows:

[0141] Case 1: M centers If a straight line is formed in space (ie, the judgment result indicates that the associated center point is on the preset straight line), the adopted global displacement vector is regarded as a suitable spatial transformation matrix.

[0142] For example, Figure 3 As shown, the center points of the blocks in the same row among the M center points are all located on the straight line where the center points of the blocks in the corresponding row in the reference image are located, and the global displacement vector is used as the spatial transformation matrix.

[0143] Case 2: M centers When a straight line is not formed in space (i.e., the judgment result indicates that the associated center point is not on the preset straight line), but the number of center points that do not form a straight line is less than or equal to 4, the affine transformation matrix is ​​used as the appropriate spatial transformation matrix.

[0144] For example, Figure 4 As shown, the two center points of the first row (represented by black solid points) are not located on the straight line (represented by white dotted lines) where the center points of the blocks in the first row of the reference image are located; and the two center points of the second row are not located on the straight line where the center points of the blocks in the second row of the reference image are located, and the affine transformation matrix is ​​used as the spatial transformation matrix.

[0145] Case 3: M centers When a straight line is not formed in space, but the number of center points that do not form a straight line is greater than 4, a projection transformation matrix is ​​used as a suitable space transformation matrix.

[0146] For example, Figure 5 As shown, the three center points of the first row are not located on the straight line where the center points of the blocks in the first row of the reference image are located; and the three center points of the third row are not located on the straight line where the center points of the blocks in the second row of the reference image are located, and the affine transformation matrix is ​​used as the spatial transformation matrix.

[0147] It should be noted that, after the registration process is performed on the image to be registered, the electronic device can directly use the image output after the registration as the registered image and stop processing the image to be registered.

[0148] Alternatively, after step 103, the method may further include:

[0149] When it is determined that the output image after the registration processing does not meet the preset registration condition, the output image is used as the image to be registered, and the step of dividing the image to be registered into blocks to obtain multiple image sub-blocks is re-executed;

[0150] When it is determined that the output image meets the preset registration condition, the output image is used as the registration image.

[0151] Based on this, the output image is used as the final registered image only when it meets the preset registration conditions. If the output image does not meet the preset registration conditions, the output image is repeatedly segmented and subsequently registered, so as to improve the registration accuracy through iterative registration processing.

[0152] Among them, the above-mentioned preset registration conditions can be any conditions set according to actual needs. Specifically, the above-mentioned preset registration conditions can be that the projection error of the output image is less than or equal to the preset error, or that the output image is the image output after a preset number of registration processes, thereby making the registration process more flexible and effectively ensuring the registration accuracy.

[0153] In addition, in the above-mentioned iterative registration process, the image to be registered is divided into blocks in different registration processes, which may be divided into the same number of image sub-blocks, or may be divided into different number of image sub-blocks.

[0154] Specifically, before using the output image as the image to be registered, the method may further include:

[0155] Calculating a first projection error of an output image obtained by the Nth registration process, where N is an integer greater than 1;

[0156] Comparing the first projection error with a second projection error of the output image obtained by the N-1th registration process;

[0157] The step of performing block processing on the image to be registered to obtain a plurality of image sub-blocks includes:

[0158] When the first projection error is less than the second projection error, dividing the image to be registered obtained by updating after the N-th registration process into a first number of image sub-blocks, wherein the first number is the number of image sub-blocks divided in the N-th registration process;

[0159] When the first projection error is greater than or equal to the second projection error, the image to be registered obtained by updating after the N-th registration process is divided into a second number of image sub-blocks, wherein the second number is greater than the first number.

[0160] Based on this, the number of image blocks in the next registration process can be determined by comparing the projection error of the image to be registered in the current registration process with the projection error of the image to be registered in the full registration process, so that the number of image blocks in the registration process is more reasonable and the registration accuracy is further improved.

[0161] Exemplarily, after determining the above-mentioned appropriate spatial transformation matrix H, the projection error of the above-mentioned target image is calculated by using formula (8) through H and M qualified local displacement vectors, as follows:

[0162]

[0163] where H(·) represents the appropriate spatial transformation operation, E k It represents the projection error between the target image and the reference image, that is, the registration error.

[0164] In the registration iteration process, through double-layer loop iteration, the n×n block operation will cyclically start from 3×3 blocks, and the number of blocks will continue to increase, and the loop iteration will be repeated until a satisfactory registration result is achieved. The above double-layer loop iteration includes an outer loop iteration and an inner loop iteration. The outer loop iteration refers to the increase in the number of blocks (for example, the last iteration is 3×3 blocks, and the next iteration is 4×4 blocks), and the inner loop refers to the increase in the number of iterations under the same number of blocks (for example, the last iteration is 3×3 blocks, and the next iteration is still 3×3 blocks). The specific statements are as follows:

[0165] In the inner loop iteration, the registration is performed under the condition of n×n blocks to obtain the projection error E k 1 , perform the registration again and get the projection error Maximum number of loops executed is N IN The condition for the loop execution is that the projection error keeps decreasing and at most one false decrease is allowed (that is, the current projection error is smaller than the previous projection error).

[0166] In the outer loop iteration, under the condition of n×n blocks, the registration is performed cyclically, and the number of blocks is continuously increased until the block limit N. The projection error under the condition of n×n blocks in the previous round is recorded as The projection error under the (n+1)×(n+1) block condition in this round is like Then continue the outer loop, otherwise stop the outer loop.

[0167] For example, if the projection error in the second registration is greater than the projection error in the first registration, the number of blocks in the third registration is the same as the number of blocks in the second registration (such as 3×3); if the projection error in the third registration is less than the projection error in the second registration, the number of blocks in the fourth registration (such as 4×4) is greater than the number of blocks in the third registration; if the projection error in the fifth registration is less than the projection error in the fourth registration, the number of blocks in the sixth registration is equal to the number of blocks in the fifth registration (such as 4×4), ... and so on.

[0168] It should be noted that, at the end of the Nth registration process, if its projection error is greater than the projection error of the N-1th registration process, the output image of the N-1th registration process is taken as the image to be registered in the N+1th registration process.

[0169] In addition, when the output image in the above registration process meets the preset registration condition, the electronic device can use the output image after the registration process as the final registration image and stop the registration iteration process. Figure 6 In the case shown in the figure, after the above iterative registration process, we can get Figure 7 The registered images.

[0170] It should be noted that the image registration processing method provided in the embodiment of the present application can be executed by an image registration processing device, or a control module in the image registration processing device for executing the image registration processing method. In the embodiment of the present application, the image registration processing device provided in the embodiment of the present application is described by taking the image registration processing device executing the image registration processing method as an example.

[0171] See also Figure 8 , is a schematic diagram of the structure of the image registration processing device provided in the embodiment of the present application, such as Figure 8 As shown, the device 800 includes:

[0172] A global translation vector acquisition module 801 is used to acquire a global translation vector between the image to be registered and the reference image in a multi-exposure image sequence;

[0173] A block division module 802 is used to perform block processing on the image to be registered to obtain multiple image sub-blocks;

[0174] A local translation vector calculation module 803, configured to calculate a local translation vector of each of the plurality of image sub-blocks to obtain a plurality of local translation vectors;

[0175] The registration module 804 is used to perform registration processing on the image to be registered based on the global translation vector and the multiple local translation vectors.

[0176] Optionally, the global translation vector acquisition module 801 includes:

[0177] A bitmap sequence acquisition submodule, used to acquire a first bitmap sequence of images to be registered in a multi-exposure image sequence, and to acquire a second bitmap sequence of reference images in the multi-exposure image sequence;

[0178] A global translation vector calculation submodule is used to calculate the global translation vector between the image to be registered and the reference image based on the first bitmap sequence and the second bitmap sequence, wherein the global translation vector represents: when the bitmap of the first bitmap sequence is translated, the similarity between the bitmap of the first bitmap sequence and the bitmap of the second bitmap sequence after the translation is the largest, and the translation amount of the bitmap of the first bitmap sequence is the same as the bitmap of the second bitmap sequence.

[0179] Optionally, the bitmap sequence acquisition submodule includes:

[0180] A bitmap value determination unit, used for comparing the brightness of each pixel in the image to be registered with the target brightness, and determining the bitmap value of each pixel in the image to be registered, wherein the target brightness is the brightness of the image to be registered at the Kth percentile, and K is an integer greater than 1;

[0181] The bitmap sequence generating unit is used to generate a first bitmap sequence of the image to be registered according to the bitmap value of each pixel point in the image to be registered.

[0182] Optionally, the bitmap sequence acquisition submodule further includes:

[0183] An area ratio calculation unit, used for calculating the area ratio of the abnormal exposure image region in the image to be registered in the image to be registered;

[0184] The K determination unit is used to determine a value corresponding to the area ratio as the K.

[0185] Optionally, the registration module 804 includes:

[0186] An image sub-block elimination sub-module, configured to eliminate image sub-blocks with noise from the multiple image sub-blocks based on the multiple local translation vectors, to obtain multiple target image sub-blocks;

[0187] A transformation matrix determination submodule, used to determine the spatial transformation matrix of the image to be registered based on the global translation vector and the local translation vectors of the multiple target image sub-blocks;

[0188] The registration submodule is used to perform registration processing on the image to be registered through the space transformation matrix.

[0189] Optionally, the image sub-block elimination sub-module includes:

[0190] a first deviation calculation unit, configured to calculate a first deviation of each local translation vector in the plurality of local translation vectors on a translation association index, wherein the translation association index includes at least one of a magnitude and a direction;

[0191] The first elimination unit is used to eliminate the image sub-blocks corresponding to the local translation vectors whose first deviations do not satisfy the first preset condition.

[0192] Optionally, the image sub-block elimination sub-module includes:

[0193] an average displacement vector calculation unit, used to calculate an average displacement vector of the multiple local translation vectors;

[0194] A second deviation calculation unit, used for calculating a second deviation between each of the local translation vectors and the average displacement vector;

[0195] The second elimination unit is used to eliminate the image sub-blocks corresponding to the local translation vectors whose second deviation is greater than or equal to the average deviation.

[0196] Optionally, the transformation matrix determination submodule includes:

[0197] a judging unit, configured to judge, based on the central coordinate information of the plurality of target image sub-blocks, whether the associated central points among the central points of the plurality of target image sub-blocks are on a preset straight line, and obtain a judging result;

[0198] The transformation matrix determining unit is used to determine the spatial transformation matrix of the image to be registered based on the judgment result, the global translation vector and the number of target image sub-blocks that are not on a preset straight line.

[0199] Optionally, the transformation matrix determining unit is specifically used to:

[0200] In a case where the judgment result indicates that the associated central point is on a preset straight line, determining the global translation vector as a spatial transformation matrix of the image to be registered;

[0201] When the judgment result indicates that the associated center points are not on a preset straight line and the number is less than or equal to a preset value, the calculated affine transformation matrix is ​​used as the spatial transformation matrix;

[0202] When the judgment result indicates that the associated central points are not on a preset straight line and the number is greater than a preset value, the calculated projection transformation matrix is ​​used as the space transformation matrix.

[0203] Optionally, the device 800 further includes:

[0204] an iterative module, configured to, when it is determined that the output image after the registration processing does not meet the preset registration condition, use the output image as the image to be registered, and re-execute the step of dividing the image to be registered into blocks to obtain a plurality of image sub-blocks;

[0205] The registration image determination module is used to use the output image as the registration image when it is determined that the output image meets the preset registration condition.

[0206] Optionally, the device 800 further includes:

[0207] A projection error calculation module, used to calculate a first projection error of an output image obtained by the Nth registration process, wherein N is an integer greater than 1;

[0208] A projection error comparison module, used for comparing the first projection error with a second projection error of the output image obtained by the N-1th registration process;

[0209] The block division module 802 includes:

[0210] A first block division submodule, configured to divide the image to be registered obtained by updating after the N-th registration process into a first number of image sub-blocks when the first projection error is less than the second projection error, wherein the first number is the number of image sub-blocks divided in the N-th registration process;

[0211] The second block submodule is used to divide the image to be registered obtained by updating after the Nth registration process into a second number of image sub-blocks when the first projection error is greater than or equal to the second projection error, wherein the second number is greater than the first number.

[0212] Optionally, the preset registration condition is that the projection error of the output image is less than or equal to a preset error, or that the output image is an image output after a preset number of registration processes.

[0213] The image registration processing device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0214] The image registration processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0215] The image registration processing device provided in the embodiment of the present application can achieve Figures 1 to 6 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0216] Alternatively, if Fig. 9 As shown, an embodiment of the present application also provides an electronic device 900, including a processor 901, a memory 902, and a program or instruction stored in the memory 902 and executable on the processor 901. When the program or instruction is executed by the processor 901, each process of the above-mentioned image registration processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0217] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0218] Fig.10 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0219] The electronic device 1000 includes but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010 and other components.

[0220] Those skilled in the art will appreciate that the electronic device 1000 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1010 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.10 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0221] The processor 1010 is configured to:

[0222] Obtaining a global translation vector between the image to be registered and the reference image in a multi-exposure image sequence;

[0223] Performing block processing on the image to be registered to obtain multiple image sub-blocks;

[0224] Calculating a local translation vector of each of the plurality of image sub-blocks to obtain a plurality of local translation vectors;

[0225] Based on the global translation vector and the multiple local translation vectors, the image to be registered is registered.

[0226] Optionally, the processor 1010 is configured to:

[0227] Acquire a first bitmap sequence of images to be registered in a multi-exposure image sequence, and acquire a second bitmap sequence of reference images in the multi-exposure image sequence;

[0228] Based on the first bitmap sequence and the second bitmap sequence, a global translation vector between the image to be registered and the reference image is calculated, wherein the global translation vector represents: when the bitmap of the first bitmap sequence is translated, the similarity between the bitmap of the first bitmap sequence and the bitmap of the second bitmap sequence after the translation is the largest, and the translation amount of the bitmap of the first bitmap sequence is the same as the bitmap of the second bitmap sequence.

[0229] Optionally, the processor 1010 is configured to:

[0230] Compare the brightness of each pixel in the image to be registered with the target brightness to determine the bitmap value of each pixel in the image to be registered, wherein the target brightness is the brightness of the image to be registered at the Kth percentile, and K is an integer greater than 1;

[0231] A first bitmap sequence of the image to be registered is generated through the bitmap value of each pixel in the image to be registered.

[0232] Optionally, the processor 1010 is configured to:

[0233] Calculate the area ratio of the abnormal exposure image region in the image to be registered in the image to be registered;

[0234] A value corresponding to the area ratio is determined as the K.

[0235] Optionally, the processor 1010 is configured to:

[0236] Based on the multiple local translation vectors, image sub-blocks with noise are removed from the multiple image sub-blocks to obtain multiple target image sub-blocks;

[0237] Determining a spatial transformation matrix of the image to be registered based on the global translation vector and the local translation vectors of the plurality of target image sub-blocks;

[0238] The image to be registered is registered using the space transformation matrix.

[0239] Optionally, the processor 1010 is configured to:

[0240] Calculating a first deviation of each local translation vector in the plurality of local translation vectors on a translation association index, wherein the translation association index includes at least one of a magnitude and a direction;

[0241] The image sub-blocks corresponding to the local translation vectors whose first deviations do not satisfy the first preset condition are eliminated.

[0242] Optionally, the processor 1010 is configured to:

[0243] Calculating an average displacement vector of the multiple local translation vectors;

[0244] Calculating a second deviation between each of the local translation vectors and the average displacement vector;

[0245] The image sub-blocks corresponding to the local translation vectors whose second deviation is greater than or equal to the average deviation are eliminated.

[0246] Optionally, the processor 1010 is configured to:

[0247] Based on the center coordinate information of the multiple target image sub-blocks, determining whether the associated center points among the center points of the multiple target image sub-blocks are on a preset straight line, and obtaining a determination result;

[0248] Based on the judgment result, the global translation vector and the number of target image sub-blocks that are not on a preset straight line, a spatial transformation matrix of the image to be registered is determined.

[0249] Optionally, the processor 1010 is configured to:

[0250] In a case where the judgment result indicates that the associated central point is on a preset straight line, determining the global translation vector as a spatial transformation matrix of the image to be registered;

[0251] When the judgment result indicates that the associated center points are not on a preset straight line and the number is less than or equal to a preset value, the calculated affine transformation matrix is ​​used as the spatial transformation matrix;

[0252] When the judgment result indicates that the associated central points are not on a preset straight line and the number is greater than a preset value, the calculated projection transformation matrix is ​​used as the space transformation matrix.

[0253] Optionally, the processor 1010 is configured to:

[0254] When it is determined that the output image after the registration processing does not meet the preset registration condition, the output image is used as the image to be registered, and the step of dividing the image to be registered into blocks to obtain multiple image sub-blocks is re-executed;

[0255] When it is determined that the output image meets the preset registration condition, the output image is used as the registration image.

[0256] Optionally, the processor 1010 is configured to:

[0257] Calculating a first projection error of an output image obtained by the Nth registration process, where N is an integer greater than 1;

[0258] Comparing the first projection error with a second projection error of the output image obtained by the N-1th registration process;

[0259] When the first projection error is less than the second projection error, dividing the image to be registered obtained by updating after the N-th registration process into a first number of image sub-blocks, wherein the first number is the number of image sub-blocks divided in the N-th registration process;

[0260] When the first projection error is greater than or equal to the second projection error, the image to be registered obtained by updating after the N-th registration process is divided into a second number of image sub-blocks, wherein the second number is greater than the first number.

[0261] Optionally, the preset registration condition is that the projection error of the output image is less than or equal to a preset error, or that the output image is an image output after a preset number of registration processes.

[0262] It should be understood that in the embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042, and the graphics processor 10041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1007 includes a touch panel 10071 and other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include two parts: a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here. The memory 1009 can be used to store software programs and various data, including but not limited to applications and operating systems. The processor 1010 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 1010.

[0263] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image registration processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0264] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0265] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image registration processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0266] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0267] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0268] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0269] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for image registration, characterized in that: include: Acquire a first bitmap sequence of the images to be registered in the multi-exposure image sequence, and acquire a second bitmap sequence of the reference images in the multi-exposure image sequence; Based on the first bitmap sequence and the second bitmap sequence, a global translation vector between the image to be registered and the reference image is calculated, wherein the global translation vector represents: when the bitmap of the first bitmap sequence is translated, the similarity between the bitmap of the first bitmap sequence and the bitmap of the second bitmap sequence after the translation is the largest, and the translation amount of the bitmap of the first bitmap sequence is the same as the bitmap of the second bitmap sequence. Performing block processing on the image to be registered to obtain multiple image sub-blocks; Calculating a local translation vector of each of the plurality of image sub-blocks to obtain a plurality of local translation vectors; Based on the global translation vector and the multiple local translation vectors, the image to be registered is registered.

2. The method according to claim 1, characterized in that The step of obtaining a first bitmap sequence of images to be registered in a multi-exposure image sequence comprises: Compare the brightness of each pixel in the image to be registered with the target brightness to determine the bitmap value of each pixel in the image to be registered, wherein the target brightness is the brightness of the image to be registered at the Kth percentile, and K is an integer greater than 1; A first bitmap sequence of the image to be registered is generated through the bitmap value of each pixel in the image to be registered.

3. The method according to claim 2, characterized in that Before comparing the brightness of each pixel in the image to be registered with the target brightness, the method further includes: Calculate the area ratio of the abnormal exposure image region in the image to be registered in the image to be registered; A value corresponding to the area ratio is determined as the K.

4. The method according to claim 1, characterized in that: The performing registration processing on the image to be registered based on the global translation vector and the multiple local translation vectors includes: Based on the multiple local translation vectors, image sub-blocks with noise are removed from the multiple image sub-blocks to obtain multiple target image sub-blocks; Determining a spatial transformation matrix of the image to be registered based on the global translation vector and the local translation vectors of the plurality of target image sub-blocks; The image to be registered is registered using the space transformation matrix.

5. The method according to claim 4, characterized in that The removing of the image sub-blocks having noise from the multiple image sub-blocks based on the multiple local translation vectors comprises: Calculating a first deviation of each local translation vector in the plurality of local translation vectors on a translation association index, wherein the translation association index includes at least one of a magnitude and a direction; The image sub-blocks corresponding to the local translation vectors whose first deviations do not satisfy the first preset condition are eliminated.

6. The method according to claim 4, characterized in that The removing of the image sub-blocks having noise from the multiple image sub-blocks based on the multiple local translation vectors comprises: Calculating an average displacement vector of the multiple local translation vectors; Calculating a second deviation between each of the local translation vectors and the average displacement vector; The image sub-blocks corresponding to the local translation vectors whose second deviation is greater than or equal to the average deviation are eliminated.

7. The method according to claim 4, characterized in that The step of determining the spatial transformation matrix of the image to be registered based on the global translation vector and the local translation vectors of the plurality of target image sub-blocks comprises: Based on the center coordinate information of the multiple target image sub-blocks, determining whether the associated center points among the center points of the multiple target image sub-blocks are on a preset straight line, and obtaining a determination result; Based on the judgment result, the global translation vector and the number of target image sub-blocks that are not on a preset straight line, a spatial transformation matrix of the image to be registered is determined.

8. The method according to claim 7, characterized in that The determining, based on the judgment result, the global translation vector and the number of target image sub-blocks not on the same straight line, the spatial transformation matrix of the image to be registered comprises: In a case where the judgment result indicates that the associated central point is on a preset straight line, determining the global translation vector as a spatial transformation matrix of the image to be registered; When the judgment result indicates that the associated center points are not on a preset straight line and the number is less than or equal to a preset value, the calculated affine transformation matrix is ​​used as the spatial transformation matrix; When the judgment result indicates that the associated central points are not on a preset straight line and the number is greater than a preset value, the calculated projection transformation matrix is ​​used as the space transformation matrix.

9. The method according to claim 1, characterized in that: After performing registration processing on the image to be registered based on the global translation vector and the multiple local translation vectors, the method further includes: When it is determined that the output image after the registration processing does not meet the preset registration condition, the output image is used as the image to be registered, and the step of dividing the image to be registered into blocks to obtain multiple image sub-blocks is re-executed; When it is determined that the output image meets the preset registration condition, the output image is used as the registration image.

10. The method according to claim 9, characterized in that Before using the output image as the image to be registered, the method further includes: Calculating a first projection error of an output image obtained by the Nth registration process, where N is an integer greater than 1; Comparing the first projection error with a second projection error of the output image obtained by the N-1th registration process; The step of performing block processing on the image to be registered to obtain a plurality of image sub-blocks includes: When the first projection error is less than the second projection error, dividing the image to be registered obtained by updating after the N-th registration process into a first number of image sub-blocks, wherein the first number is the number of image sub-blocks divided in the N-th registration process; When the first projection error is greater than or equal to the second projection error, the image to be registered obtained by updating after the N-th registration process is divided into a second number of image sub-blocks, wherein the second number is greater than the first number.

11. The method according to claim 9, characterized in that The preset registration condition is that the projection error of the output image is less than or equal to a preset error, or that the output image is an image output after a preset number of registration processes.

12. An image registration processing device, characterized in that: include: A bitmap sequence acquisition submodule, used to acquire a first bitmap sequence of images to be registered in a multi-exposure image sequence, and to acquire a second bitmap sequence of reference images in the multi-exposure image sequence; A global translation vector calculation submodule, configured to calculate a global translation vector between the image to be registered and the reference image based on the first bitmap sequence and the second bitmap sequence, wherein the global translation vector represents: when the bitmaps of the first bitmap sequence are translated, the translation amount of the bitmaps of the first bitmap sequence when the similarity between the bitmaps of the first bitmap sequence and the bitmaps of the second bitmap sequence after the translation is the largest; A block division module, used for performing block processing on the image to be registered to obtain multiple image sub-blocks; A local translation vector calculation module, used to calculate the local translation vector of each of the multiple image sub-blocks to obtain multiple local translation vectors; A registration module is used to perform registration processing on the image to be registered based on the global translation vector and the multiple local translation vectors.

13. The device according to claim 12, characterized in that The bitmap sequence acquisition submodule includes: A bitmap value determination unit, used for comparing the brightness of each pixel in the image to be registered with the target brightness, and determining the bitmap value of each pixel in the image to be registered, wherein the target brightness is the brightness of the image to be registered at the Kth percentile, and K is an integer greater than 1; The bitmap sequence generating unit is used to generate a first bitmap sequence of the image to be registered according to the bitmap value of each pixel point in the image to be registered.

14. The device according to claim 13, characterized in that The bitmap sequence acquisition submodule further includes: An area ratio calculation unit, used for calculating the area ratio of the abnormal exposure image region in the image to be registered in the image to be registered; The K determination unit is used to determine a value corresponding to the area ratio as the K.

15. The device according to claim 12, characterized in that The registration module comprises: An image sub-block elimination sub-module, configured to eliminate image sub-blocks with noise from the multiple image sub-blocks based on the multiple local translation vectors, to obtain multiple target image sub-blocks; A transformation matrix determination submodule, used to determine the spatial transformation matrix of the image to be registered based on the global translation vector and the local translation vectors of the multiple target image sub-blocks; The registration submodule is used to perform registration processing on the image to be registered through the space transformation matrix.

16. The device according to claim 15, characterized in that The image sub-block removal sub-module comprises: a first deviation calculation unit, configured to calculate a first deviation of each local translation vector in the plurality of local translation vectors on a translation association index, wherein the translation association index includes at least one of a magnitude and a direction; The first elimination unit is used to eliminate the image sub-blocks corresponding to the local translation vectors whose first deviations do not satisfy the first preset condition.

17. The device according to claim 15, characterized in that The image sub-block removal sub-module comprises: an average displacement vector calculation unit, used to calculate an average displacement vector of the multiple local translation vectors; A second deviation calculation unit, used for calculating a second deviation between each of the local translation vectors and the average displacement vector; The second elimination unit is used to eliminate the image sub-blocks corresponding to the local translation vectors whose second deviation is greater than or equal to the average deviation.

18. The device according to claim 17, characterized in that The transformation matrix determination submodule includes: a judging unit, configured to judge, based on the central coordinate information of the plurality of target image sub-blocks, whether the associated central points among the central points of the plurality of target image sub-blocks are on a preset straight line, and obtain a judging result; The transformation matrix determining unit is used to determine the spatial transformation matrix of the image to be registered based on the judgment result, the global translation vector and the number of target image sub-blocks that are not on a preset straight line.

19. The device according to claim 18, characterized in that The transformation matrix determination unit is specifically used for: In a case where the judgment result indicates that the associated central point is on a preset straight line, determining the global translation vector as a spatial transformation matrix of the image to be registered; When the judgment result indicates that the associated center points are not on a preset straight line and the number is less than or equal to a preset value, the calculated affine transformation matrix is ​​used as the spatial transformation matrix; When the judgment result indicates that the associated central points are not on a preset straight line and the number is greater than a preset value, the calculated projection transformation matrix is ​​used as the space transformation matrix.

20. The device according to claim 12, characterized in that The device further comprises: an iterative module, configured to, when it is determined that the output image after the registration processing does not meet the preset registration condition, use the output image as the image to be registered, and re-execute the step of dividing the image to be registered into blocks to obtain a plurality of image sub-blocks; The registration image determination module is used to use the output image as the registration image when it is determined that the output image meets the preset registration condition.

21. The device according to claim 20, characterized in that The device further comprises: A projection error calculation module, used to calculate a first projection error of an output image obtained by the Nth registration process, where N is an integer greater than 1; A projection error comparison module, used for comparing the first projection error with a second projection error of the output image obtained by the N-1th registration process; The block module comprises: A first block division submodule, configured to divide the image to be registered obtained by updating after the N-th registration process into a first number of image sub-blocks when the first projection error is less than the second projection error, wherein the first number is the number of image sub-blocks divided in the N-th registration process; The second block submodule is used to divide the image to be registered obtained by updating after the Nth registration process into a second number of image sub-blocks when the first projection error is greater than or equal to the second projection error, wherein the second number is greater than the first number.

Images