Image registration processing method and device

Abstract

The present application discloses an image registration processing method and device, belonging to the field of image processing technology. The method includes: extracting a feature sequence of a reference image and a feature sequence of a target image, wherein the target image is any image in a multiple exposure image sequence except the reference image; based on the feature sequence of the reference image and the feature sequence of the target image, calculating the global translation vector between the target image and the reference image; based on the global translation vector, updating the feature sequence of the target image to obtain a first image after registration processing; performing at least one registration processing based on the first image to obtain a registered image; wherein, in each registration processing of at least one registration processing, the image area of ​​the image to be registered is divided into multiple image sub-areas, and the feature sequence of the image to be registered is updated by the local translation vectors of the multiple image sub-areas, and the image to be registered is the image obtained after the last registration processing.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image registration processing method and device. Background Art

[0002] In order to meet people's increasingly high shooting requirements for electronic devices such as mobile phones and tablets, such as high dynamic range, low illumination, large depth of field and wide field of view, multiple exposure technology is applied to the shooting function of electronic devices. Since the electronic device and the target scene are prone to relative movement during the multiple exposure time period, the multiple exposure images generally cannot achieve pixel-level or even sub-pixel matching, so it is necessary to align the images in the multiple exposure image sequence. However, the current registration of images in the multiple exposure image sequence usually has the problem of low registration accuracy. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide an image registration processing method and device, which can solve the problem of low registration accuracy in the current registration of images in a multiple exposure image sequence.

[0004] In a first aspect, an embodiment of the present application provides an image registration processing method, the method comprising:

[0005] Extracting a feature sequence of a reference image and a feature sequence of a target image, wherein the target image is any image in a multiple exposure image sequence except the reference image;

[0006] Calculating a global translation vector between the target image and the reference image based on a feature sequence of the reference image and a feature sequence of the target image;

[0007] Based on the global translation vector, the feature sequence of the target image is updated to obtain a first image after registration processing;

[0008] Perform at least one registration process based on the first image to obtain a registered image;

[0009] Wherein, in each registration process of the at least one registration process, the image area of ​​the image to be registered is divided into multiple image sub-regions, and the feature sequence of the image to be registered is updated by the local translation vectors of the multiple image sub-regions, and the image to be registered is the image obtained after the last registration process.

[0010] In a second aspect, an embodiment of the present application provides an image registration processing device, the device comprising:

[0011] A feature extraction module, used to extract a feature sequence of a reference image and a feature sequence of a target image, wherein the target image is any image in a multiple exposure image sequence except the reference image;

[0012] A global translation vector calculation module, used for calculating a global translation vector between the target image and the reference image based on a feature sequence of the reference image and a feature sequence of the target image;

[0013] A first registration processing module, used for updating the feature sequence of the target image based on the global translation vector to obtain a first image after registration processing;

[0014] A second registration processing module, used for performing at least one registration process based on the first image to obtain a registered image;

[0015] Wherein, in each registration process of the at least one registration process, the image area of ​​the image to be registered is divided into multiple image sub-regions, and the feature sequence of the image to be registered is updated by the local translation vectors of the multiple image sub-regions, and the image to be registered is the image obtained after the last registration process.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0019] In an embodiment of the present application, in the process of the electronic device performing registration processing on a multiple exposure image sequence, the global translation vector of the target image is calculated, and the feature sequence of the target image is updated based on the global translation vector, so as to obtain the first image after the first registration processing of the target image, and on the basis of the first image, the first image after the first registration processing is further registered at least once, and each registration processing is implemented by the translation vectors of multiple image sub-regions of the image to be registered, so as to obtain the final registered image. In this way, through the image registration processing method provided by the present application, through multiple registration processing, and taking into account both the whole and the local in the multiple registration processing, the accuracy of the registration of the multiple exposure image sequence can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flowchart of an image registration processing method provided in an embodiment of the present application;

[0021] Figure 2 is a schematic diagram of a person photo to be saved provided in an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of facial feature information provided by an embodiment of the present application;

[0023] Figure 4 It is one of the schematic diagrams of the generated photo provided in the embodiment of the present application;

[0024] Figure 5 This is the second schematic diagram of the generated photo provided in the embodiment of the present application;

[0025] Figure 6 It is a structural schematic diagram of an image registration processing device provided in an embodiment of the present application;

[0026] Figure 7 This is one of the hardware structure diagrams of the electronic device provided in the embodiment of the present application;

[0027] Figure 8 This is the second schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0029] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0030] The image registration processing method provided by the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0031] See also Figure 1 , is a flow chart of an image registration processing method provided in an embodiment of the present application, which is applied to electronic devices such as Figure 1 As shown, the above image registration processing method includes the following steps:

[0032] Step 101, extracting a feature sequence of a reference image and a feature sequence of a target image, wherein the target image is any image in a multiple exposure image sequence except the reference image;

[0033] Step 102: Calculate a global translation vector between the target image and the reference image based on the feature sequence of the reference image and the feature sequence of the target image;

[0034] Step 103: based on the global translation vector, updating the feature sequence of the target image to obtain a first image after registration processing;

[0035] Step 104: performing at least one registration process based on the first image to obtain a registered image;

[0036] Wherein, in each registration process of the at least one registration process, the image area of ​​the image to be registered is divided into multiple image sub-regions, and the feature sequence of the image to be registered is updated by the local translation vectors of the multiple image sub-regions, and the image to be registered is the image obtained after the last registration process.

[0037] Based on this, in the process of the electronic device performing registration processing on the multiple exposure image sequence, by calculating the global translation vector of the target image, and updating the feature sequence of the target image based on the global translation vector, the first image after the first registration processing of the target image is obtained, and on the basis of the first image, the first image after the first registration processing is further registered at least once, and each registration processing is realized by the translation vectors of multiple image sub-regions of the image to be registered, so as to obtain the final registered image. In this way, through the image registration processing method provided by the present application, through multiple registration processing, and taking into account both the whole and the local in the multiple registration processing, the accuracy of the registration of the multiple exposure image sequence can be effectively improved.

[0038] In the above step 101, when the electronic device performs registration on the multi-exposure image sequence, the electronic device may extract a feature sequence of the reference image and a feature sequence of the target image.

[0039] The multiple exposure image sequence may include at least two images captured under different exposure parameters, and the exposure parameter may include at least one of exposure value, depth of field value, spectrum type, etc.

[0040] In addition, the reference image may be any one of the at least two images in the multiple exposure image sequence, and the target image may be any one of the at least two images except the reference image.

[0041] In the embodiment of the present application, the feature sequences of the reference image and the target image may be extracted by using an image feature extraction algorithm to extract point features or line features of the reference image and the target image respectively for storage.

[0042] For example, feature extraction and feature preservation can be performed on the input image through a point feature or line feature extraction algorithm to obtain a feature sequence {Ε i}, i represents the sequence number of the input image in the multiple exposure image sequence.

[0043] It should be noted that the image feature extraction algorithm is preset in the electronic device, and it can be any algorithm that can realize the extraction of point features or line features. Since this application does not improve the feature extraction algorithm, it will not be described in detail here.

[0044] For example, point features can be extracted using a feature point extraction algorithm such as Scale-Invariant Feature Transform (SIFT) that has scale, translation and orientation invariance; or, a feature point extraction algorithm such as Oriented FAST and Rotated BRIEF (ORB) that does not have scale, translation and orientation invariance can be used, and so on.

[0045] In the above step 102, after extracting the feature sequence of the reference image and the feature sequence of the target image, the electronic device can calculate the global translation vector between the target image and the reference image based on the feature sequence of the reference image and the feature sequence of the target image.

[0046] In an embodiment of the present application, the global translation vector between the target image and the reference image is calculated based on the feature sequence of the reference image and the feature sequence of the target image. Matching feature points or feature lines may be determined in the reference image and the target image, and coordinate information of the matching feature points or feature lines may be extracted from the feature sequence of the reference image and the feature sequence of the target image, and the global translation vector may be calculated using the extracted coordinate information.

[0047] Specifically, the coordinate information of the jth feature (point feature or line feature) is extracted from the feature sequence of the target image. And, extract the coordinate information of the jth feature (point feature or line feature) from the feature sequence of the reference image Where i and r represent the serial numbers of the target image and the reference image in the multiple exposure image sequence, respectively, and i, r and j are integers. By formula (1), the global translation (Δx, Δy) between the target image and the reference image can be calculated as follows:

[0048]

[0049] Here, n represents the total number of features.

[0050] In the above step 103, after calculating the above global translation vector, the electronic device can update the feature sequence of the target image based on the global translation vector to obtain the first image after registration processing, that is, to achieve primary registration processing of the target image.

[0051] Specifically, the feature sequence of the target image can be updated by formula (2), as follows:

[0052]

[0053] in, represents the feature sequence obtained after the primary registration process (i.e., the feature sequence of the first image); C(E i ) represents the calculation of the corresponding coordinate set for the features of the i-th image;

[0054] r represents the reference image number in the multiple exposure image sequence, and i≠r means that the feature update is performed on all images except the reference image.

[0055] For example, Figure 2 As shown, the electronic device may be provided with a primary registration module, a secondary registration module, ..., a K-level registration module, ..., and through the primary registration module, the electronic device may register the following Figure 3 The reference image 31 and the target image 32 are subjected to feature extraction, and their feature sequences are extracted respectively; the global translation estimation is then performed through the extracted feature sequences, that is, the global translation vector of the image to be registered 32 and the reference image 31 is calculated; finally, the feature sequence of the target image 32 is updated through the calculated global translation vector, so as to realize the primary registration processing of the target image 32.

[0056] In the above step 104, after obtaining the updated first image after the above primary registration process, the electronic device may perform at least one registration process based on the first image to obtain a registered image.

[0057] It should be noted that, in the process of each registration process performed by the electronic device based on the above-mentioned first image, the image area of ​​the image to be registered is divided into multiple image sub-regions, and the feature sequence of the image to be registered is updated through the local translation vectors of the multiple image sub-regions. The above-mentioned image to be registered includes: in the first registration process of at least one registration process, it is the first image; in the Kth registration process of at least one registration process, it is the image obtained after the previous registration process, and K is a positive integer.

[0058] In one embodiment, the at least one registration process may be only one registration process. Specifically, the step 104 may include:

[0059] Acquire a feature sequence of the first image;

[0060] Dividing the image area of ​​the first image into a plurality of image sub-areas, and calculating a translation vector of each image sub-area based on a feature series of each image sub-area in a feature sequence of the first image;

[0061] Based on the translation vectors of the plurality of image sub-regions of the first image, the feature sequence of the first image is updated to obtain a second image after registration processing;

[0062] The second image is determined as the registered image.

[0063] Based on this, by dividing the image area of ​​the first image into multiple image sub-regions, and based on the translation vectors of the multiple image sub-regions (i.e., local translation vectors), the feature sequence of the first image after the first-level update is updated to realize the secondary registration processing of the target image, which can not only improve the registration accuracy but also ensure the processing speed.

[0064] The acquiring of the feature sequence of the first image may be that after the electronic device updates the feature sequence of the target image in the primary registration process, the feature sequence of the first image is obtained and saved, and the electronic device may extract the saved feature sequence of the first image.

[0065] In addition, the above-mentioned dividing the image area of ​​the first image into a plurality of image sub-areas may be that the electronic device divides the image area of ​​the first image into an arbitrary number of image sub-areas, where the arbitrary number is greater than or equal to 2.

[0066] For example, it can be a feature sequence of the first image A block operation is performed, such as block division through a block feature extraction algorithm such as MTB that does not have scale, translation and direction invariance, so as to divide the image area of ​​the first image into four 2×2 image sub-areas in a two-dimensional coordinate space, and obtain a feature sequence of each image sub-area.

[0067] It should be noted that while the electronic device divides the image area of ​​the first image, it also divides the image area of ​​the reference image into the same number of image sub-regions. For example, when the image sub-region of the first image is divided into four image sub-regions of 2×2, the electronic device also divides the reference image into four matching image sub-regions of 2×2.

[0068] In the embodiment of the present application, after the electronic device obtains the feature sequences of the multiple image sub-regions, the electronic device may calculate the translation vectors of the multiple sub-image regions based on the feature sequence of the first image.

[0069] Among them, the above-mentioned calculation of the translation vectors of multiple sub-image regions based on the feature sequence of the first image can be based on the feature sequence of the matching image sub-regions in the first image and the reference image, and the translation vector of each image sub-region in the first image and the reference image is calculated, that is, the translation vector represents the translation amount generated by the matching image sub-regions in the matching first image and the reference image.

[0070] Specifically, the translation vector of the image sub-region can be obtained by formula (3): as follows:

[0071]

[0072] Wherein, the above represents the kth block area (i.e., image sub-area);

[0073] and represent the feature sequences of the k-th block region in the reference image and the image to be registered after the primary registration process (i.e., the first image mentioned above);

[0074] L represents the number of image sub-regions. For example, in the case of four image sub-regions divided into 2×2, L is 4, and so on.

[0075] In an embodiment of the present application, after the electronic device obtains the translation vectors of the above-mentioned multiple image sub-regions, the electronic device can update the feature sequence of the first image based on the translation vectors of the multiple image sub-regions to obtain a second image after registration processing.

[0076] The updating of the feature sequence of the first image based on the translation vectors of the plurality of image sub-regions may be performed by calculating the average of the translation vectors of the plurality of image sub-regions and updating the feature sequence of the first image using the calculated average. For example, the average is used as (Δx, Δy) in formula (2).

[0077] Alternatively, the updating of the feature sequence of the first image based on the translation vectors of the plurality of image sub-regions of the first image may include:

[0078] Based on the translation vectors of the plurality of image sub-regions of the first image, a rotation vector and a translation vector between the first image and the reference image are calculated;

[0079] Based on a rotation vector and a translation vector between the first image and the reference image, a feature sequence of the first image is updated.

[0080] Based on this, the rotation vector and translation vector between the first image and the reference image are calculated through the translation vectors of multiple image sub-areas of the first image, and then the feature sequence of the first image is updated through the calculated rotation vector and translation vector, thereby making the alignment more accurate.

[0081] Among them, the above-mentioned translation vectors of multiple image sub-regions based on the first image are used to calculate the rotation vector and translation vector between the first image and the reference image. It can be that the translation vectors of multiple image sub-regions are used to construct matching feature point pairs with the same number as the image sub-regions, and based on the matching feature point pairs, the rotation vector R and translation vector T between the first image and the reference image are estimated by using image space transformation constraints.

[0082] Specifically, when the number of the above image sub-regions is L, L feature point pairs can be constructed through the translation vectors of the L image sub-regions, as follows:

[0083]

[0084] Its physical meaning can be understood as four pairs of virtual matching feature point pairs corresponding to the feature sequence after the first-level registration processing. Therefore, based on the L pairs of virtual matching feature point pairs, the rotation vector R and the translation vector T between the first image and the reference image can be estimated by using the image space transformation constraints.

[0085] In addition, after the electronic device calculates the rotation vector and translation vector between the first image and the reference image, the electronic device can update the feature sequence of the first image based on the calculated rotation vector and translation vector to achieve secondary registration processing of the target image.

[0086] Specifically, the feature sequence of the first image may be updated by formula (5), as follows:

[0087]

[0088] Where R and T are the rotation vector and translation vector between the first image and the reference image respectively;

[0089] It is the feature sequence after secondary registration processing.

[0090] For example, Figure 4 As shown, after the electronic device completes the primary registration processing of the target image through the primary registration module, the electronic device can also perform secondary registration processing on the target image through its secondary registration module. In the secondary registration processing, the electronic device can first perform 2×2 block division, that is, divide the image areas of the reference image 31 and the first image 41 into 2×2 image sub-areas respectively; then perform block translation estimation, that is, calculate the translation vector of the 2×2 image sub-area through the feature sequence of the reference image 31 and the feature sequence of the first image 41; then perform rotation and translation solution, that is, calculate the rotation vector R and translation vector T based on the translation vector of the 2×2 image sub-area; finally perform feature update, that is, update the feature sequence of the first image 41 through the rotation vector R and translation vector T.

[0091] It should be noted that, after the electronic device updates the feature sequence of the first image, ie, completes the secondary registration process, the electronic device may directly use the second image obtained after the secondary registration process as the registration image.

[0092] Alternatively, determining the second image as the registration image may include:

[0093] When the first registration error is less than or equal to a preset error threshold, the second image is determined as the registered image, wherein the first registration error is calculated based on the feature sequence of the reference image and the feature sequence of the updated first image.

[0094] Based on this, the calculated first registration error is compared with the preset error threshold, and when the first registration error is less than or equal to the preset error threshold, the second image is determined as the registration image, thereby ensuring that the registration accuracy of the final registered image can reach the actual required accuracy.

[0095] The first registration error is calculated based on the feature sequence of the reference image and the feature sequence of the updated first image, and this may be achieved by a preset registration error calculation algorithm.

[0096] Specifically, the first registration error can be calculated by formula (6), as follows:

[0097]

[0098] in, represents mean calculation;

[0099] The physical meaning of formula (6) is that the feature sequence after the secondary registration processing is The feature sequence E of the reference image r The average difference in spatial position.

[0100] In another embodiment, the at least one registration process based on the first image may be to continue to perform multiple registration processes after registering the target image once. For example, the registration process may be continued for a preset number of times based on the first image, and the preset number of times may be greater than or equal to two times, and the multiple registration processes are performed only when the registration process for the preset number of times is completed, and the image obtained by the registration process for the preset number of times is used as the registered image.

[0101] Alternatively, determining the second image as the registration image may include:

[0102] When the first registration error is less than or equal to a preset error threshold, updating the first image to the second image;

[0103] Performing N registration processes based on the second image to obtain an image processed by the Nth registration process, wherein the registered image is an image obtained after the Nth registration process, and N is a positive integer;

[0104] The Mth registration process in the N registration processes includes:

[0105] Extracting a feature sequence of the image to be registered, where M is 1, the image to be registered is the second image; and M is greater than 1, the image to be registered is the image obtained after the last registration process;

[0106] Dividing an image region of the image to be registered into a plurality of image sub-regions, and calculating translation vectors of the plurality of image sub-regions of the image to be registered based on a feature sequence of the image to be registered;

[0107] Based on the translation vectors of the multiple image sub-regions of the image to be registered, the feature sequence of the image to be registered is updated to obtain the image after the M-th registration process.

[0108] Based on this, when the registration accuracy of the secondary registration process does not meet the requirements, that is, the registration error exceeds the preset error threshold, the image after the secondary registration process can be registered N times to further improve the accuracy of the registration process.

[0109] In an embodiment of the present application, in the Mth registration processing of the above-mentioned N registration processings, the electronic device extracts a feature sequence of the image to be registered, and when M is 1, the image to be registered is the second image; when M is greater than 1, the image to be registered is the image obtained after the previous registration processing; the image area of ​​the image to be registered is divided into multiple image sub-regions, and based on the feature sequence of the image to be registered, the translation vectors of the multiple image sub-regions of the image to be registered are calculated; based on the translation vectors of the multiple image sub-regions of the image to be registered, the feature sequence of the image to be registered is updated to obtain the image after the Mth registration processing.

[0110] Among them, in the Mth registration process, the electronic device can extract the feature sequence of the image obtained after the previous registration process, that is, when M=1, the previous registration process is the above-mentioned secondary registration process, then the electronic device extracts the feature sequence of the above-mentioned second picture (at this time the second picture is the picture to be registered); and when M is greater than 1, the previous registration process is the M-1th registration process in the above-mentioned N registration processes (that is, the M+1 level registration process), at this time, the electronic device extracts the feature sequence of the picture after the M-1th registration process (at this time the picture to be registered).

[0111] In addition, after extracting the feature sequence of the image to be registered, the electronic device can divide the image area of ​​the image to be registered into multiple image sub-areas, and calculate the translation vectors of the multiple image sub-areas of the image to be registered based on the feature sequence of the image to be registered.

[0112] It should be noted that the number of image sub-regions obtained by dividing the image to be registered may be the same as the number of image sub-regions obtained by dividing the first image in the secondary registration process; or, it may be different from the number of image sub-regions obtained by dividing the first image in the secondary registration process.

[0113] Specifically, in the above two-level registration process, the image area of ​​the first image can be divided into 2×2 image sub-areas, and in each registration process of the above N registration processes, the image area of ​​the image to be registered can be divided into K×K image sub-areas, where K is an integer greater than 2, and K is equal to M+2. At this time, K can be obtained by formula (7): 2 The translation vector of the image sub-region is as follows:

[0114]

[0115] in, K 2 The kth image sub-region P k The translation vector of .

[0116] In the embodiment of the present application, after obtaining the translation vectors of multiple image sub-regions of the image to be registered, the electronic device can directly update the feature sequence of the image to be registered based on the translation vectors of all image sub-regions in the image to be registered.

[0117] Alternatively, since there may be noise influence in the image capture (such as the sensor is affected by the outside world when working, or there are moving targets in the multiple exposure images, etc.), noise may be generated in the previous feature extraction. Therefore, in order to eliminate the influence of noise, the above-mentioned M-th registration process may also include:

[0118] Based on the translation vectors of the multiple image sub-regions of the image to be registered, removing some image sub-regions that do not meet preset conditions from the multiple image sub-regions of the image to be registered;

[0119] The image after the M-th registration process is an image obtained by updating the feature sequence of the image to be registered based on the translation vector of the image sub-region retained in the image to be registered after elimination.

[0120] Based on this, by eliminating some image areas that do not meet the preset conditions in multiple image sub-areas of the image to be registered, the third feature sequence is updated only by relying on the translation vector of the retained image area in the image to be registered, thereby further improving the registration accuracy.

[0121] Wherein, based on the translation vectors of the multiple image sub-regions of the image to be registered, the partial image sub-regions that do not meet the preset conditions in the multiple image sub-regions of the image to be registered are eliminated, which may be to calculate the average of the translation vectors of the multiple image sub-regions of the image to be registered, and according to the absolute value of the difference between the translation vector of each image sub-region and the average, at least one image sub-region with a larger absolute value is regarded as an image sub-region with noise and is deleted. For example, the image sub-region with the largest absolute value may be eliminated.

[0122] In addition, after the electronic device removes some of the multiple image subregions of the image to be registered, the electronic device may update the feature sequence of the image to be registered based on the translation vectors of all the image subregions of the image to be registered that are retained after the removal.

[0123] In an embodiment of the present application, the above-mentioned translation vectors based on multiple image sub-regions of the image to be registered (which may be the image sub-regions after being eliminated) are used to update the feature sequence of the image to be registered. It can be that in the reference secondary registration process, the rotation vector and the translation vector between the image to be registered and the reference image are calculated through the translation vectors of multiple image sub-regions of the image to be registered, and the feature sequence of the image to be registered is updated through the rotation vector and the translation vector.

[0124] Alternatively, the M-th registration process may further include:

[0125] Generate a translation vector sequence of translation vectors of a plurality of image sub-regions of the image to be registered, and calculate a spatial transformation matrix corresponding to the translation vector sequence;

[0126] The image after the M-th registration process is an image obtained by updating the feature sequence of the image to be registered based on the spatial transformation matrix.

[0127] Based on this, by generating a translation vector sequence of translation vectors of multiple image sub-regions of the image to be registered, and then calculating the spatial transformation matrix corresponding to the translation vector sequence, and updating the feature sequence of the image to be registered based on the calculated spatial transformation matrix, the registration accuracy can be further improved.

[0128] It should be noted that the above calculation of the space transformation matrix corresponding to the translation vector sequence may be to select the best method to estimate the stable space transformation matrix, that is, to achieve the best transformation H * The optimal method for selecting the optimal transformation is to make a judgment based on the quantity and quality of the appropriate translation vector sequence. * , which reflects the local correspondence between the multiple exposure images to be registered. When the number of image blocks is H *When the size is changed from large to small, the registration force on the local part of the image is gradually strengthened, and then the purpose of gradual refinement is achieved. The specific solution method is the same as the rotation and translation solution steps in the secondary registration process. The optimal transformation can be estimated by constructing a virtual matching feature point pair using the appropriate translation vector.

[0129] In addition, the above-mentioned updating of the feature sequence of the image to be registered based on the spatial transformation matrix can be implemented by formula (8), as follows:

[0130]

[0131] in, is the feature sequence after K (ie, M+2) levels of registration processing (ie, the Mth registration processing in the above-mentioned N registration processing).

[0132] For example, Figure 5 As shown, in the K-level registration processing (i.e., K=M+2), the electronic device first performs K×K block processing, that is, divides the reference image 31 and the image to be registered 51 into K×K image sub-regions respectively; then performs block translation estimation, that is, calculates the translation vector of each image sub-region in the K×K image sub-regions; then performs translation vector screening, that is, eliminates the image sub-regions with noise in the K×K image sub-regions; then performs optimal transformation solution, that is, calculates the spatial transformation matrix through the translation vector of the image sub-region after removing the noise; finally, the feature sequence of the image to be registered 51 is updated through the spatial transformation matrix.

[0133] In the embodiment of the present application, the M-th registration process may further include:

[0134] Calculating a second registration error between the feature sequence of the image after the M-th registration process and the feature sequence of the reference image, and when the second registration error is less than or equal to the preset error threshold, using the M-th registration process as the N-th registration process; or,

[0135] If it is determined that the M-th registration process is a preset number of registration processes, the M-th registration process is used as the N-th registration process.

[0136] Based on this, it is possible to stop the registration process in time by judging whether the registration error after each registration process is less than or equal to the preset error threshold, or judging whether the current registration process has reached the preset number of registration processes, thereby ensuring both the registration accuracy and the processing speed.

[0137] The above calculation of the second registration error can be implemented by formula (9), as follows:

[0138]

[0139] The physical meaning of formula (9) is that the updated feature sequence of the M+2 level registration module is The feature E of the reference image r The average difference in spatial position.

[0140] Then, after the image to be registered is updated in the above-mentioned Mth registration process, the electronic device determines the calculated second registration error τ i Whether the preset error threshold is reached, or whether the Mth registration process is the preset number of registration processes, if so, stop the next registration process and use the image after the Mth registration process as the registration image; otherwise, continue the next registration process, that is, Figure 2 shown.

[0141] It should be noted that each of the N registration processes performed on the image after the secondary registration process may be a process of referring to the secondary registration process, in which the image area of ​​the image obtained by the previous registration process is divided, and the translation vector of the divided image sub-area is calculated, and based on the translation vector of the divided image sub-area, the rotation vector and translation vector of the image obtained by the previous registration process and the reference image are calculated, and finally, the feature sequence of the image obtained by the previous registration process is updated through the calculated rotation vector and translation vector.

[0142] It should be noted that the image registration processing method provided in the embodiment of the present application can be executed by an image registration processing device, or a control module in the image registration processing device for executing the image registration processing method. In the embodiment of the present application, the image registration processing device provided in the embodiment of the present application is described by taking the image registration processing device executing the image registration processing method as an example.

[0143] See also Figure 6 , is a schematic diagram of the structure of the image registration processing device provided in the embodiment of the present application, such as Figure 6 As shown, the device 600 includes:

[0144] A feature extraction module 601 is used to extract a feature sequence of a reference image and a feature sequence of a target image, wherein the target image is any image in a multiple exposure image sequence except the reference image;

[0145] A global translation vector calculation module 602, configured to calculate a global translation vector between the target image and the reference image based on a feature sequence of the reference image and a feature sequence of the target image;

[0146] A first registration processing module 603, used to update the feature sequence of the target image based on the global translation vector to obtain a first image after registration processing;

[0147] A second registration processing module 604, configured to perform at least one registration process based on the first image to obtain a registered image;

[0148] Wherein, in each registration process of the at least one registration process, the image area of ​​the image to be registered is divided into multiple image sub-regions, and the feature sequence of the image to be registered is updated by the local translation vectors of the multiple image sub-regions, and the image to be registered is the image obtained after the last registration process.

[0149] Based on this, in the process of the electronic device performing registration processing on the multiple exposure image sequence, by calculating the global translation vector of the target image, and updating the feature sequence of the target image based on the global translation vector, the first image after the first registration processing of the target image is obtained, and on the basis of the first image, the first image after the first registration processing is further registered at least once, and each registration processing is realized by the translation vectors of multiple image sub-regions of the image to be registered, so as to obtain the final registered image. In this way, through the image registration processing method provided by the present application, through multiple registration processing, and taking into account both the whole and the local in the multiple registration processing, the accuracy of the registration of the multiple exposure image sequence can be effectively improved.

[0150] Optionally, the second registration processing module 604 includes:

[0151] A first feature sequence acquisition unit, used to acquire a feature sequence of the first image;

[0152] a first translation vector calculation unit, configured to divide an image region of the first image into a plurality of image sub-regions, and calculate translation vectors of the image sub-regions based on a feature sequence of the first image;

[0153] A registration processing unit, configured to update a feature sequence of the first image based on translation vectors of a plurality of image sub-regions of the first image, so as to obtain a second image after registration processing;

[0154] A registration image determining unit is used to determine the second image as the registration image.

[0155] Based on this, by dividing the image area of ​​the first image into multiple image sub-regions, and based on the translation vectors of the multiple image sub-regions (i.e., local translation vectors), the feature sequence of the first image obtained after the primary registration processing is updated to realize the secondary registration processing of the target image, thereby not only improving the registration accuracy but also ensuring the processing speed.

[0156] Optionally, the registration processing unit includes:

[0157] a rotation and translation subunit, configured to calculate a rotation vector and a translation vector between the first image and the reference image based on the translation vectors of the plurality of image sub-regions of the first image;

[0158] The registration processing subunit is used to update the feature sequence of the first image based on the rotation vector and the translation vector between the first image and the reference image.

[0159] Based on this, the rotation vector and translation vector between the first image and the reference image are calculated through the translation vectors of multiple image sub-areas of the first image, and then the feature sequence of the first image is updated through the calculated rotation vector and translation vector, thereby making the alignment more accurate.

[0160] Optionally, the registration image determination unit is specifically used to:

[0161] When the first registration error is less than or equal to a preset error threshold, the second image is determined as the registered image.

[0162] Based on this, the calculated first registration error is compared with the preset error threshold, and when the first registration error is less than or equal to the preset error threshold, the second image is determined as the registration image, thereby ensuring that the registration accuracy of the final registered image can reach the actual required accuracy.

[0163] Optionally, the registration image determination unit comprises:

[0164] A registration processing subunit, configured to, when a first registration error is less than or equal to a preset error threshold, continue to perform N registration processes based on the second image to obtain an image after the Nth registration process, where N is a positive integer;

[0165] A registration image determination subunit, used to determine the image obtained after the N registration processes as a registration image;

[0166] The Mth registration process in the N registration processes includes:

[0167] Extracting a feature sequence of the image to be registered, where when M is 1, the image to be registered is the second image; and when M is greater than 1, the image to be registered is the image obtained after the last registration process;

[0168] Dividing an image region of the image to be registered into a plurality of image sub-regions, and calculating translation vectors of the plurality of image sub-regions of the image to be registered based on a feature sequence of the image to be registered;

[0169] Based on the translation vectors of the multiple image sub-regions of the image to be registered, the feature sequence of the image to be registered is updated to obtain the image after the M-th registration process.

[0170] Based on this, when the registration accuracy of the secondary registration process does not meet the requirements, that is, the registration error exceeds the preset error threshold, the image after the secondary registration process can be registered N times to further improve the accuracy of the registration process.

[0171] Optionally, the M-th registration process further includes:

[0172] Calculating a second registration error between the feature sequence of the image after the M-th registration process and the feature sequence of the reference image, and when the second registration error is less than or equal to the preset error threshold, using the M-th registration process as the N-th registration process; or,

[0173] If it is determined that the M-th registration process is a preset number of registration processes, the M-th registration process is used as the N-th registration process.

[0174] Based on this, by judging whether the registration error after each registration process is less than or equal to the preset error threshold, or judging whether the current registration process has reached the preset number of registration processes, the registration process can be stopped in time, thereby ensuring both the registration accuracy and the processing speed.

[0175] Optionally, the M-th registration process further includes:

[0176] Generate a translation vector sequence of translation vectors of a plurality of image sub-regions of the image to be registered, and calculate a spatial transformation matrix corresponding to the translation vector sequence;

[0177] The image after the M-th registration process is an image obtained by updating the feature sequence of the image to be registered based on the spatial transformation matrix.

[0178] Based on this, by generating a translation vector sequence of translation vectors of multiple image sub-regions of the image to be registered, and then calculating the spatial transformation matrix corresponding to the translation vector sequence, and updating the feature sequence of the image to be registered based on the calculated spatial transformation matrix, the registration accuracy can be further improved.

[0179] Optionally, the M-th registration process further includes:

[0180] Based on the translation vectors of the multiple image sub-regions of the image to be registered, removing some image sub-regions that do not meet preset conditions from the multiple image sub-regions of the image to be registered;

[0181] The image after the M-th registration process is an image obtained by updating the feature sequence of the image to be registered based on the translation vector of the image sub-region retained in the image to be registered after elimination.

[0182] Based on this, by eliminating some image areas that do not meet the preset conditions in multiple image sub-areas of the image to be registered, the third feature sequence is updated only by relying on the translation vector of the retained image area in the image to be registered, thereby further improving the registration accuracy.

[0183] The image registration processing device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0184] The image registration processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0185] The image registration processing device provided in the embodiment of the present application can achieve Figures 1 to 5 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0186] Alternatively, if Figure 7As shown, the embodiment of the present application also provides an electronic device 700, including a processor 701, a memory 702, and a program or instruction stored in the memory 702 and executable on the processor 701. When the program or instruction is executed by the processor 701, each process of the above-mentioned image registration processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0187] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0188] Figure 8 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0189] The electronic device 800 includes but is not limited to: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810 and other components.

[0190] Those skilled in the art will appreciate that the electronic device 800 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 810 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0191] The processor 810 is configured to:

[0192] Extracting a feature sequence of a reference image and a feature sequence of a target image, wherein the target image is any image in a multiple exposure image sequence except the reference image;

[0193] Calculating a global translation vector between the target image and the reference image based on a feature sequence of the reference image and a feature sequence of the target image;

[0194] Based on the global translation vector, the feature sequence of the target image is updated to obtain a first image after registration processing;

[0195] Perform at least one registration process based on the first image to obtain a registered image;

[0196] Wherein, in each registration process of the at least one registration process, the image area of ​​the image to be registered is divided into multiple image sub-regions, and the feature sequence of the image to be registered is updated by the local translation vectors of the multiple image sub-regions, and the image to be registered is the image obtained after the last registration process.

[0197] Based on this, in the process of the electronic device performing registration processing on the multiple exposure image sequence, by calculating the global translation vector of the target image, and updating the feature sequence of the target image based on the global translation vector, the first image after the first registration processing of the target image is obtained, and on the basis of the first image, the first image after the first registration processing is further registered at least once, and each registration processing is realized by the translation vectors of multiple image sub-regions of the image to be registered, so as to obtain the final registered image. In this way, through the image registration processing method provided by the present application, through multiple registration processing, and taking into account both the whole and the local in the multiple registration processing, the accuracy of the registration of the multiple exposure image sequence can be effectively improved.

[0198] Optionally, the processor 810 is further configured to:

[0199] Acquire a feature sequence of the first image;

[0200] Dividing an image region of the first image into a plurality of image sub-regions, and calculating translation vectors of the image sub-regions based on a feature sequence of the first image;

[0201] Based on the translation vectors of the plurality of image sub-regions of the first image, the feature sequence of the first image is updated to obtain a second image after registration processing;

[0202] The second image is determined as the registered image.

[0203] Based on this, by dividing the image area of ​​the first image into multiple image sub-regions, and based on the translation vectors of the multiple image sub-regions (i.e., local translation vectors), the feature sequence of the first image after the first-level update is updated to realize the secondary registration processing of the target image, which can not only improve the registration accuracy but also ensure the processing speed.

[0204] Optionally, the processor 810 is further configured to:

[0205] Based on the translation vectors of the plurality of image sub-regions of the first image, a rotation vector and a translation vector between the first image and the reference image are calculated;

[0206] Based on a rotation vector and a translation vector between the first image and the reference image, a feature sequence of the first image is updated.

[0207] Based on this, the rotation vector and translation vector between the first image and the reference image are calculated through the translation vectors of multiple image sub-areas of the first image, and then the feature sequence of the first image is updated through the calculated rotation vector and translation vector, thereby making the alignment more accurate.

[0208] Optionally, the processor 810 is further configured to:

[0209] When the first registration error is less than or equal to a preset error threshold, the second image is determined as the registered image, wherein the first registration error is calculated based on the feature sequence of the reference image and the feature sequence of the updated first image.

[0210] Based on this, the calculated first registration error is compared with the preset error threshold, and when the first registration error is less than or equal to the preset error threshold, the second image is determined as the registration image, thereby ensuring that the registration accuracy of the final registered image can reach the actual required accuracy.

[0211] Optionally, the processor 810 is further configured to:

[0212] When the first registration error is less than or equal to a preset error threshold, continue to perform N registration processes based on the second image to obtain an image after the Nth registration process, where N is a positive integer;

[0213] Determine the image obtained after the N-times registration processing as the registered image;

[0214] The Mth registration process in the N registration processes includes:

[0215] Extracting a feature sequence of the image to be registered, where, when M is 1, the image to be registered is the second image;

[0216] In the case where M is greater than 1, the image to be registered is an image obtained after the last registration process; the image area of ​​the image to be registered is divided into a plurality of image sub-areas, and based on the feature sequence of the image to be registered, the translation vectors of the plurality of image sub-areas of the image to be registered are calculated;

[0217] Based on the translation vectors of the multiple image sub-regions of the image to be registered, the feature sequence of the image to be registered is updated to obtain the image after the M-th registration process.

[0218] Based on this, when the registration accuracy of the secondary registration process does not meet the requirements, that is, the registration error exceeds the preset error threshold, the image after the secondary registration process can be registered N times to further improve the accuracy of the registration process.

[0219] Optionally, the M-th registration process further includes:

[0220] Calculating a second registration error between the feature sequence of the image after the M-th registration process and the feature sequence of the reference image, and when the second registration error is less than or equal to the preset error threshold, using the M-th registration process as the N-th registration process; or,

[0221] If it is determined that the M-th registration process is a preset number of registration processes, the M-th registration process is used as the N-th registration process.

[0222] Based on this, by judging whether the registration error after each registration process is less than or equal to the preset error threshold, or judging whether the current registration process has reached the preset number of registration processes, the registration process can be stopped in time, thereby ensuring both the registration accuracy and the processing speed.

[0223] Optionally, the M-th registration process further includes:

[0224] Generate a translation vector sequence of translation vectors of a plurality of image sub-regions of the image to be registered, and calculate a spatial transformation matrix corresponding to the translation vector sequence;

[0225] The image after the M-th registration process is an image obtained by updating the feature sequence of the image to be registered based on the spatial transformation matrix.

[0226] Based on this, by generating a translation vector sequence of translation vectors of multiple image sub-regions of the image to be registered, and then calculating the spatial transformation matrix corresponding to the translation vector sequence, and updating the feature sequence of the image to be registered based on the calculated spatial transformation matrix, the registration accuracy can be further improved.

[0227] Optionally, the M-th registration process further includes:

[0228] Based on the translation vectors of the multiple image sub-regions of the image to be registered, removing some image sub-regions that do not meet preset conditions from the multiple image sub-regions of the image to be registered;

[0229] The image after the M-th registration process is an image obtained by updating the feature sequence of the image to be registered based on the translation vector of the image sub-region retained in the image to be registered after elimination.

[0230] Based on this, by eliminating some image areas that do not meet the preset conditions in multiple image sub-areas of the image to be registered, the third feature sequence is updated only by relying on the translation vector of the retained image area in the image to be registered, thereby further improving the registration accuracy.

[0231] It should be understood that in the embodiment of the present application, the input unit 804 may include a graphics processor (Graphics Processing Unit, GPU) 8041 and a microphone 8042, and the graphics processor 8041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 806 may include a display panel 8061, and the display panel 8061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 807 includes a touch panel 8071 and other input devices 8072. The touch panel 8071 is also called a touch screen. The touch panel 8071 may include two parts: a touch detection device and a touch controller. Other input devices 8072 may include but are not limited to a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, and a joystick, which will not be repeated here. The memory 809 can be used to store software programs and various data, including but not limited to applications and operating systems. The processor 810 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, the user interface, and the application program, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 810.

[0232] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image registration processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0233] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0234] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image registration processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0235] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0236] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be noted that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0237] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0238] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for image registration, characterized in that: include: Extracting a feature sequence of a reference image and a feature sequence of a target image, wherein the target image is any image in a multiple exposure image sequence except the reference image; Calculating a global translation vector between the target image and the reference image based on a feature sequence of the reference image and a feature sequence of the target image; Based on the global translation vector, the feature sequence of the target image is updated to obtain a first image after registration processing; Perform at least one registration process based on the first image to obtain a registered image; Wherein, in each registration process of the at least one registration process, the image area of ​​the image to be registered is divided into multiple image sub-regions, and the feature sequence of the image to be registered is updated by the local translation vectors of the multiple image sub-regions, and the image to be registered is the image obtained after the last registration process.

2. The method according to claim 1, characterized in that The performing at least one registration process based on the first image to obtain a registered image includes: Acquire a feature sequence of the first image; Dividing an image region of the first image into a plurality of image sub-regions, and calculating translation vectors of the image sub-regions based on a feature sequence of the first image; Based on the translation vectors of the plurality of image sub-regions of the first image, the feature sequence of the first image is updated to obtain a second image after registration processing; The second image is determined as a registered image.

3. The method according to claim 2, characterized in that The updating of the feature sequence of the first image based on the translation vectors of the plurality of image sub-regions of the first image comprises: Based on the translation vectors of the plurality of image sub-regions of the first image, a rotation vector and a translation vector between the first image and the reference image are calculated; Based on a rotation vector and a translation vector between the first image and the reference image, a feature sequence of the first image is updated.

4. The method according to claim 2, characterized in that: The step of determining the second image as a registration image comprises: When a first registration error is less than or equal to a preset error threshold, the second image is determined as a registered image, wherein the first registration error is calculated based on a feature sequence of the reference image and a feature sequence of the updated first image.

5. The method according to claim 2, characterized in that: The step of determining the second image as a registration image comprises: When the first registration error is less than or equal to a preset error threshold, continue to perform N registration processes based on the second image to obtain an image after the Nth registration process, where N is a positive integer; Determine the image obtained after the N registration processes as the registered image; The Mth registration process in the N registration processes includes: Extracting a feature sequence of the image to be registered, where when M is 1, the image to be registered is the second image; and when M is greater than 1, the image to be registered is the image obtained after the last registration process; Dividing an image region of the image to be registered into a plurality of image sub-regions, and calculating translation vectors of the plurality of image sub-regions of the image to be registered based on a feature sequence of the image to be registered; Based on the translation vectors of the multiple image sub-regions of the image to be registered, the feature sequence of the image to be registered is updated to obtain the image after the M-th registration process.

6. The method according to claim 5, characterized in that The M-th registration process further includes: Calculating a second registration error between the feature sequence of the image after the M-th registration process and the feature sequence of the reference image, and when the second registration error is less than or equal to the preset error threshold, using the M-th registration process as the N-th registration process; or, If it is determined that the M-th registration process is a preset number of registration processes, the M-th registration process is used as the N-th registration process.

7. The method according to claim 5, characterized in that The M-th registration process further includes: Generate a translation vector sequence of translation vectors of a plurality of image sub-regions of the image to be registered, and calculate a spatial transformation matrix corresponding to the translation vector sequence; The image after the M-th registration process is an image obtained by updating the feature sequence of the image to be registered based on the spatial transformation matrix.

8. The method according to claim 5, characterized in that The M-th registration process further includes: Based on the translation vectors of the multiple image sub-regions of the image to be registered, removing some image sub-regions that do not meet preset conditions from the multiple image sub-regions of the image to be registered; The image after the M-th registration process is an image obtained by updating the feature sequence of the image to be registered based on the translation vector of the image sub-region retained in the image to be registered after elimination.

9. An image registration processing device, characterized in that: include: A feature extraction module, used to extract a feature sequence of a reference image and a feature sequence of a target image, wherein the target image is any image in a multiple exposure image sequence except the reference image; A global translation vector calculation module, used for calculating a global translation vector between the target image and the reference image based on a feature sequence of the reference image and a feature sequence of the target image; A first registration processing module, used for updating the feature sequence of the target image based on the global translation vector to obtain a first image after registration processing; A second registration processing module, used for performing at least one registration process based on the first image to obtain a registered image; Wherein, in each registration process of the at least one registration process, the image area of ​​the image to be registered is divided into multiple image sub-regions, and the feature sequence of the image to be registered is updated by the local translation vectors of the multiple image sub-regions, and the image to be registered is the image obtained after the last registration process.

10. The device according to claim 9, characterized in that The second registration processing module includes: A first feature sequence acquisition unit, used to acquire a feature sequence of the first image; a first translation vector calculation unit, configured to divide an image region of the first image into a plurality of image sub-regions, and calculate translation vectors of the image sub-regions based on a feature sequence of the first image; A registration processing unit, configured to update a feature sequence of the first image based on translation vectors of a plurality of image sub-regions of the first image, to obtain a second image after registration processing; A registration image determining unit is used to determine the second image as the registration image.

11. The device according to claim 10, characterized in that The registration processing unit comprises: a rotation and translation subunit, configured to calculate a rotation vector and a translation vector between the first image and the reference image based on the translation vectors of the plurality of image sub-regions of the first image; The registration processing subunit is used to update the feature sequence of the first image based on the rotation vector and the translation vector between the first image and the reference image.

12. The device according to claim 10, characterized in that The registration image determination unit is specifically used for: When a first registration error is less than or equal to a preset error threshold, the second image is determined as the registered image, wherein the first registration error is calculated based on a feature sequence of the reference image and a feature sequence of the updated first image.

13. The device according to claim 12, characterized in that The registration image determination unit comprises: A registration processing subunit, configured to, when a first registration error is less than or equal to a preset error threshold, continue to perform N registration processes based on the second image to obtain an image after the Nth registration process, where N is a positive integer; A registration image determination subunit, used to determine the image obtained after the N registration processes as a registration image; The Mth registration process in the N registration processes includes: Extracting a feature sequence of the image to be registered, where when M is 1, the image to be registered is the second image; and when M is greater than 1, the image to be registered is the image obtained after the last registration process; Dividing an image region of the image to be registered into a plurality of image sub-regions, and calculating translation vectors of the plurality of image sub-regions of the image to be registered based on a feature sequence of the image to be registered; Based on the translation vectors of the multiple image sub-regions of the image to be registered, the feature sequence of the image to be registered is updated to obtain the image after the M-th registration process.

14. The device according to claim 13, characterized in that The M-th registration process further includes: Calculating a second registration error between the feature sequence of the image to be registered and the feature sequence of the reference image, and when the second registration error is less than or equal to the preset error threshold, treating the Mth registration process as the Nth registration process; or, If it is determined that the M-th registration process is a preset number of registration processes, the M-th registration process is used as the N-th registration process.

15. The device according to claim 13, characterized in that The M-th registration process further includes: Generate a translation vector sequence of translation vectors of a plurality of image sub-regions of the image to be registered, and calculate a spatial transformation matrix corresponding to the translation vector sequence; The image after the M-th registration process is an image obtained by updating the feature sequence of the image to be registered based on the spatial transformation matrix.

16. The device according to claim 13, characterized in that The M-th registration process further includes: Based on the translation vectors of the multiple image sub-regions of the image to be registered, removing some image sub-regions that do not meet preset conditions from the multiple image sub-regions of the image to be registered; The image after the M-th registration process is an image obtained by updating the feature sequence of the image to be registered based on the translation vector of the image sub-region retained in the image to be registered after elimination.

Images