Image registration method, terminal and computer storage medium

By combining depth information and clustering technology in image registration, the problem of low image registration accuracy in the prior art is solved, and high-precision processing of moving objects and edge information is achieved, thereby improving the registration effect.

CN114586059BActive Publication Date: 2024-11-26GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201980101531.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-11-13
Publication Date
2024-11-26
Estimated Expiration
2039-11-13

Smart Images

  • Figure CN114586059B_ABST
    Figure CN114586059B_ABST
Patent Text Reader

Abstract

The embodiment of the present application discloses an image registration method, a terminal and a computer storage medium, comprising: acquiring a first image and a second image, dividing the first image into blocks to obtain image blocks of the first image, calling a preset image block similarity algorithm according to pixel values ​​of the image blocks of the first image, determining similar image blocks of the image blocks of the first image from the second image, calculating the movement amount of the image blocks of the first image according to position information of the image blocks of the first image and position information of similar image blocks of the image blocks of the first image, clustering the first image according to the depth value of the first image, the movement amount of the image blocks of the first image and the position information of the first image, generating clustered image blocks, and when the clustered image blocks meet preset conditions, moving the image blocks of the first image according to the movement amount of the image blocks of the first image to obtain a registered image, thereby improving the accuracy of the depth value of the image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to image registration technology, and more particularly to an image registration method, a terminal, and a computer storage medium. Background Art

[0002] Image registration is the process of matching and superimposing two or more images acquired at different times, with different sensors or under different conditions. It has been widely used in remote sensing data analysis, computer vision, image processing and other fields.

[0003] In image registration, block matching is often used to detect the movement of pixels on the original image based on feature point matching or optical flow method, and then the obtained movement amount is used to move the pixels of the original image again to generate a registered image.

[0004] Block matching and feature point-based matching can only obtain sparse pixel movement. For scenes with moving objects, this method can only obtain a global motion model and cannot simultaneously match local moving objects, which will have a certain impact on the image registration results. When using the optical flow method, the optical flow often does not process the image edge information accurately, resulting in blurred edges after matching, which will also reduce the accuracy of image registration. It can be seen that the registration accuracy of existing image registration methods is low. Summary of the invention

[0005] In order to solve the above technical problems, the embodiments of the present application provide an image registration method, a terminal and a computer storage medium, which can improve the accuracy of image registration.

[0006] The technical solution of the embodiment of the present application can be implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides an image registration method, comprising:

[0008] Acquire a first image and a second image;

[0009] Dividing the first image into blocks to obtain image blocks of the first image;

[0010] According to the pixel values ​​of the image blocks of the first image, calling a preset image block similarity algorithm, and determining similar image blocks of the image blocks of the first image from the second image;

[0011] calculating a movement amount of the image block of the first image according to position information of the image block of the first image and position information of a similar image block to the image block of the first image;

[0012] Clustering the first image according to the depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate clustered image blocks;

[0013] When the clustered image blocks meet a preset condition, the image blocks of the first image are moved according to the movement amount of the image blocks of the first image to obtain a registered image.

[0014] In a second aspect, an embodiment of the present application provides a terminal, including:

[0015] An acquisition module, configured to acquire a first image and a second image;

[0016] A block division module, configured to divide the first image into blocks to obtain image blocks of the first image;

[0017] a determination module configured to call a preset image block similarity algorithm according to pixel values ​​of the image blocks of the first image, and determine similar image blocks of the image blocks of the first image from the second image;

[0018] a calculation module configured to calculate a movement amount of the image block of the first image according to position information of the image block of the first image and position information of similar image blocks of the image block of the first image;

[0019] a clustering module configured to cluster the first image according to the depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate clustered image blocks;

[0020] The moving module is configured to move the image block of the first image according to the movement amount of the image block of the first image when the clustered image block meets a preset condition, so as to obtain a registered image.

[0021] In a third aspect, an embodiment of the present application provides a terminal, the terminal comprising:

[0022] A processor and a storage medium storing instructions executable by the processor, wherein the storage medium relies on the processor to perform operations through a communication bus, and when the instructions are executed by the processor, the image registration method described in one or more of the above embodiments is executed.

[0023] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores executable instructions. When the executable instructions are executed by one or more processors, the processor executes the image registration method described in one or more of the above embodiments.

[0024] The embodiment of the present application provides an image registration method, a terminal and a computer storage medium, which obtain a first image and a second image, divide the first image into blocks to obtain image blocks of the first image, call a preset image block similarity algorithm based on pixel values ​​of the image blocks of the first image, determine similar image blocks of the image blocks of the first image from the second image, calculate the movement amount of the image blocks of the first image based on position information of the image blocks of the first image and position information of similar image blocks of the image blocks of the first image, cluster the first image based on the depth value of the first image, the movement amount of the image blocks of the first image and the position information of the first image, generate clustered image blocks, and when the clustered image blocks meet preset conditions, move the image blocks of the first image based on the movement amount of the image blocks of the first image to obtain the registration. image; that is, in an embodiment of the present application, an image block similar to an image block of the first image is searched in the second image to calculate the amount of movement of the image block of the first image, and then the depth value of the first image, the amount of movement of the image block of the first image and the position information of the first image are used to cluster the first image, thereby generating clustered image blocks. In this way, when the clustered image blocks meet the preset conditions, the first image is moved by the movement of the image blocks of the first image, thereby obtaining a registered image. In this way, the depth information of the first image is taken into account in the image registration to form the clustered image blocks. Only when the clustered image blocks meet the preset conditions are they moved according to the amount of movement of the image blocks of the first image to obtain the registered image, thereby improving the accuracy of the registered image, and thus making the effect of the obtained registered image better. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of a flow chart of an optional image registration method provided in an embodiment of the present application;

[0026] Figure 2 It is a flowchart of the matching method based on feature points;

[0027] Figure 3 A schematic flow chart of an example of an optional image registration method provided in an embodiment of the present application;

[0028] Figure 4 A schematic diagram of an optional process of dividing an image to be registered into blocks provided in an embodiment of the present application;

[0029] Figure 5 A schematic diagram of an optional structure for searching a reference image provided in an embodiment of the present application;

[0030] Figure 6a A schematic diagram of an optional structure of an image to be registered provided in an embodiment of the present application;

[0031] Figure 6b A schematic diagram of the structure of an optional reference image provided in an embodiment of the present application;

[0032] Figure 7a A schematic diagram of another optional structure of an image to be registered provided in an embodiment of the present application;

[0033] Figure 7b A schematic diagram of the structure of another optional reference image provided in an embodiment of the present application;

[0034] Figure 7c A schematic diagram of the structure of another optional reference image provided in an embodiment of the present application;

[0035] Figure 8a A schematic diagram of the structure of an optional image block of an image to be registered provided in an embodiment of the present application;

[0036] Figure 8b A schematic diagram of the structure of an optional clustered image block provided in an embodiment of the present application;

[0037] Figure 9a A schematic diagram of the structure of overlapping pixels of an optional registration image provided in an embodiment of the present application;

[0038] Figure 9b A schematic diagram of the structure of vacant pixels of an optional registration image provided in an embodiment of the present application;

[0039] Fig.10 A schematic diagram of an optional terminal structure provided in an embodiment of the present application Figure 1 ;

[0040] Fig.11 A schematic diagram of an optional terminal structure provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0041] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It is understood that the specific embodiments described herein are only used to explain the related applications, rather than to limit the applications. It should also be noted that, for the convenience of description, only the parts related to the related applications are shown in the accompanying drawings.

[0042] The present application embodiment provides an image registration method. Figure 1 A schematic diagram of a flow chart of an optional image registration method provided in an embodiment of the present application, such as Figure 1 As shown, the above image registration method may include:

[0043] S101: Acquire a first image and a second image;

[0044] At present, the most commonly used image registration methods include block matching, feature point-based matching and optical flow method. Figure 2 It is a flowchart of the matching method based on feature points, such as Figure 2 As shown, the matching method based on feature points may include:

[0045] S201: input image;

[0046] Specifically, an input image is obtained, the input image includes an image to be registered and a reference image;

[0047] S202: feature detection;

[0048] Specifically, feature detection is performed on the image to be registered and the reference image respectively to obtain feature points of the image to be registered and feature points of the reference image;

[0049] S203: mapping function;

[0050] Here, a mapping function is used to find feature points of the reference image that match the feature points of the image to be registered;

[0051] S204: conversion and resampling;

[0052] S205: Output image.

[0053] Then, the image space coordinate transformation parameters are obtained through the matching feature point pairs, and finally the image is registered by the coordinate transformation parameters to obtain the registered image, that is, the output image.

[0054] In this way, the registered image is obtained through a matching method based on feature points.

[0055] It can be seen from the above methods that the matching method based on feature points can only obtain sparse pixel movement, and the same is true for block matching. However, for scenes with moving objects, the above two methods can only obtain a global motion model but cannot take into account local moving objects at the same time. The accuracy of the image registration result is not high. In addition, the optical flow method is used. Since the optical flow often does not process the edge information of the image accurately, the edge is blurred after matching, resulting in poor accuracy of the registered image.

[0056] In order to improve the accuracy of image registration, here, firstly, the image to be registered and the reference image are obtained, wherein in S101, the first image obtained is the image to be registered, and the second image obtained is the reference image, so that the first image and the second image are used for registration to obtain the registered image.

[0057] Here, it should be noted that the acquired first image includes not only the pixel values ​​and position information of the first image, but also the depth value of the first image. Similarly, the acquired second image includes not only the pixel values ​​of the second image, but also the depth value of the second image.

[0058] Among them, the pixel values ​​of the above-mentioned first image and the pixel values ​​of the second image can be color information of the image, such as RGB information, or grayscale information. Here, the embodiment of the present application does not make specific limitations on this.

[0059] S102: Divide the first image into blocks to obtain image blocks of the first image;

[0060] After the first image and the second image are acquired through S101, the first image needs to be divided into blocks. Here, the first image can be divided into ordinary rectangular blocks. For example, the first image block is divided into several blocks according to a given size to obtain image blocks of the first image.

[0061] S103: calling a preset image block similarity algorithm according to pixel values ​​of the image blocks of the first image, and determining similar image blocks of the image blocks of the first image from the second image;

[0062] After the image blocks of the first image are obtained through S102, for each image block of the first image, the size of each image block of the first image is used as a reference block, and the second image is searched and traversed to obtain each image block traversed by the search, and the similarity between the reference block and each image block traversed by the search is calculated, and the image block with the largest calculated similarity value is determined as the similar image block of the first image; here, in calculating the similarity, the sum of absolute difference (SAD) method can be used, and the normalized cross correlation image matching (NCC) method can also be used. Here, the embodiment of the present application does not make specific limitations on this.

[0063] It should be noted that when the grayscale information of the first image and the second image differ greatly, the NCC method is often used to calculate the similarity between the two image blocks.

[0064] In this way, through S103, similar image blocks can be found in the second image for the image blocks of the first image.

[0065] S104: calculating a movement amount of the image block of the first image according to position information of the image block of the first image and position information of similar image blocks of the image block of the first image;

[0066] After similar image blocks of the image blocks of the first image are determined, the movement amount of the image blocks of the first image may be calculated according to the position information of the image blocks of the first image and the position information of the image blocks of the first image.

[0067] In order to calculate the movement amount of the image block of the first image, in an optional embodiment, S104 may include:

[0068] Determine the difference between the horizontal and vertical coordinates of the image block of the first image and the horizontal and vertical coordinates of the image block similar to the image block of the first image as the initial movement amount of the pixel point of the image block of the first image;

[0069] The initial movement amount of the pixel points of the image block of the first image is corrected to obtain the movement amount of the image block of the first image.

[0070] Specifically, for the first image and the second image, the lower left corner of the image can be set as the origin of the two-dimensional coordinates, and the length and width of the image block and the distance between the horizontal and vertical coordinates can be recorded as the position information of the image block. In this way, assuming that the horizontal and vertical coordinates in the position information of the image block of the first image are (x1, y1), and the horizontal and vertical coordinates in the position information of the similar image block of the image block of the first image are (x2, y2), then the difference between the two is calculated to be (x1-x2, y1-y2), and the calculation result is recorded as the initial movement of the pixel points of the image block of the first image. Here, it should be noted that the image block of the first image includes a number of pixel points, and the initial movement of the pixel points in the same image block is the same, which is the difference between the horizontal and vertical coordinates of the image block of the first image and the horizontal and vertical coordinates of the similar image block of the image block of the first image.

[0071] After the initial movement amount of the pixel points of the image block of the first image is calculated, the value needs to be further corrected to obtain the movement amount of the image block of the first image; then, to obtain the movement amount of the image block of the first image, in an optional embodiment, the initial movement amount of the pixel points of the image block of the first image is corrected to obtain the movement amount of the image block of the first image, including:

[0072] Taking the position information of any selected pixel point in the image block of the first image as a reference, sampling is performed from a preset range of the first image to obtain a preset number of pixel points;

[0073] According to the initial movement amount of a preset number of pixel points, the position information of the selected pixel points is moved to generate the moved pixel points;

[0074] Obtaining from the second image the pixel value of the pixel point at the same position as the moved pixel point;

[0075] Determine the weight of the moved pixel based on the pixel value of the pixel at the same position;

[0076] According to the weights of the moved pixels, the initial movement amounts of the moved pixels are weighted summed to obtain the movement amount of the image block of the first image.

[0077] Specifically, for the image block of the first image, a pixel point is selected therein, and based on the horizontal and vertical coordinates of the selected pixel point, within a preset range from the selected pixel point, for example, with the selected pixel point as the center, pixel points are sampled in the adjacent image blocks of the image block where the selected pixel point is located. Here, in order to enrich the types of sampling data, different adjacent image blocks are usually selected for sampling respectively, so that a preset number of pixel points can be obtained.

[0078] After sampling a preset number of pixels, the initial movement amount of the preset number of pixels can be obtained. At this time, the position information of the selected pixels is moved according to the movement amount of the preset number of pixels to generate the moved pixels.

[0079] For the moved pixel points, the weight of the moved pixel points can be determined according to the pixel values ​​of the pixels at the same position as the moved pixel points. That is to say, each moved pixel point corresponds to a weight value. In this way, the initial movement amount of the moved pixel points can be weighted summed according to the weight of the moved pixel points, and the value obtained by the weighted summation is determined as the movement amount of the image block of the first image.

[0080] In order to determine the weight value of the moved pixel point, in an optional embodiment, the weight of the moved pixel point is determined according to the pixel value of the moved pixel point, including:

[0081] Calculate the absolute value of the difference between the pixel value of the pixel point at the same position and the pixel value of the selected pixel point;

[0082] The weight corresponding to the absolute value of the difference between the pixel value of the pixel point at the same position and the pixel value of the selected pixel point is determined as the weight of the pixel point after the move.

[0083] Specifically, after obtaining the moved pixel points, the pixel values ​​of the pixel points at the same position of each moved pixel point in the second image are obtained, and the absolute value of the difference between the pixel value of each pixel point at the same position and the pixel value of the selected pixel point is calculated. The absolute value of the difference between the pixel value of each pixel point at the same position and the selected pixel point corresponds to a weight value, so the corresponding weight value can be determined as the weight value of the moved pixel point.

[0084] Among them, the weight value corresponding to the absolute value of the difference between the pixel value of each pixel at the same position and the selected pixel can be set in advance or input in real time. Here, the embodiment of the present application does not make any specific limitation on this.

[0085] S105: clustering the first image according to the depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate clustered image blocks;

[0086] After obtaining the movement amount of the image block of the first image through S104, it is necessary to determine whether the movement amount of the image block of the first image is available. Here, in order to determine whether it is available, the first image is clustered first, and whether the movement amount of the image block of the first image is available is determined based on the clustered image blocks.

[0087] Furthermore, when the resolution of the pixel value of the first image is different from the resolution of the depth value of the first image, it is necessary to process the resolution of the depth value of the first image. In an optional embodiment, before S105, the method may further include:

[0088] Scaling the resolution of the depth value of the first image according to the resolution of the pixel value of the first image to obtain a depth value of the first image after processing;

[0089] Accordingly, S105 may include:

[0090] The first image is clustered according to the processed depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate clustered image blocks.

[0091] That is to say, when the resolution of the depth value of the first image is smaller than the resolution of the pixel value of the first image, the depth value of the first image is enlarged so that the resolution of the depth value of the enlarged first image is equal to the resolution of the pixel value of the first image; when the resolution of the depth value of the first image is larger than the resolution of the pixel value of the first image, the depth value of the first image is reduced so that the resolution of the depth value of the reduced first image is equal to the resolution of the pixel value of the first image, so that the resolution of the depth value of the processed first image is equal to the resolution of the pixel value of the first image.

[0092] Thus, after obtaining the processed depth value of the first image, the first image needs to be clustered according to the processed depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate clustered image blocks.

[0093] Here, it should be noted that when scaling the depth value, the depth value of the first image can also be preprocessed and corrected. For the first image that represents the depth value in a bitmap, the surface information of the object can be restored by triangular plane reconstruction, and smoothing and interpolation can be performed. For the first image that represents the depth value in a depth map, it is necessary to detect and denoise bad pixels of the first image to ensure that the obtained depth value has good accuracy.

[0094] In order to obtain clustered image blocks, in an optional embodiment, S105 may include:

[0095] Determine the movement amount of the image block of the first image as the movement amount of the pixel points of the image block of the first image;

[0096] Using the depth value corresponding to the pixel point of the first image, the movement amount of the pixel point of the first image and the position information of the pixel point of the first image, multiplying by the corresponding weight value, respectively, to form an array of the first image;

[0097] According to a preset clustering algorithm, the array of the first image is clustered to generate clustered image blocks.

[0098] First, the obtained movement amount of the image block of the first image is also the movement amount of the pixel points in the image block of the first image. For each pixel point of the first image, the corresponding depth value of the pixel point, the movement amount of the pixel point and the position coordinates of the pixel point are multiplied by the corresponding weight value to form an array, that is, the array of the first image.

[0099] Here, a preset clustering algorithm is used to cluster the array of the first image, wherein the preset clustering algorithm may be a mean-shift clustering algorithm or a K-means clustering algorithm. Here, the embodiment of the present application does not make any specific limitation on this.

[0100] In this way, clustered image blocks can be generated.

[0101] Furthermore, in order to obtain clustered image blocks, in an optional embodiment, clustering the array of the first image according to a preset clustering algorithm to generate clustered image blocks includes:

[0102] Clustering the array of the first image according to a preset clustering algorithm to obtain an initial clustered image block;

[0103] Selecting an image block whose number of pixels is less than a first preset threshold from the initial clustered image blocks;

[0104] The selected image blocks are reclassified to obtain clustered image blocks.

[0105] After clustering using a preset clustering algorithm, initial clustered image blocks can be obtained. Since there may be some image blocks with very few pixels in the initial clustered image blocks, these image blocks have no reference significance, so the categories of such image blocks need to be discarded.

[0106] Specifically, a first preset threshold is preset, and image blocks with a number of pixels less than the first preset threshold are selected from the initial clustered image blocks, and the category is discarded and reclassified.

[0107] In order to reclassify the categories of the image blocks whose number of pixels is less than the first preset threshold, in an optional embodiment, the selected image blocks are reclassified to obtain clustered image blocks, including:

[0108] Determine, from the initial clustered image blocks, an image block that is closest to the selected image block;

[0109] The category of the selected image block is assigned to the image block that is closest to the selected image block to obtain a clustered image block.

[0110] It should be noted that, in the clustered image blocks, there is a distance between each class of image blocks. Here, the category of the selected image block is attributed to the image block closest to it. In this way, the category with a smaller number of pixels is discarded to obtain the clustered image block.

[0111] S106: When the clustered image blocks meet a preset condition, the image blocks of the first image are moved according to the movement amount of the image blocks of the first image to obtain a registered image.

[0112] Here, the clustered image blocks are judged to determine whether the clustered image blocks meet the preset conditions. Only when the clustered image blocks meet the preset conditions, the precision sign requirement of the calculated movement amount of the image blocks of the first image is met. Therefore, at this time, the image blocks of the first image are moved according to the movement amount of the image blocks of the first image to obtain the final registered image.

[0113] For the clustered image blocks not meeting the preset conditions, in an optional embodiment, the image registration method may further include:

[0114] When the clustered image blocks do not meet the preset conditions, the clustered image blocks are determined as the image blocks of the first image to update the image blocks of the first image, and the process returns to execute based on the pixel values ​​of the image blocks of the first image, calling a preset image block similarity algorithm, and determining similar image blocks of the image blocks of the first image from the second image.

[0115] That is to say, when the clustered image blocks do not meet the preset conditions, it means that the calculated movement amount of the image blocks of the first image is not accurate enough, and the movement amount of the image blocks of the first image needs to be recalculated. At this time, the clustered image blocks are determined as the image blocks of the first image. In this way, the blocking method of the first image is updated, and then S103 to S106 are executed until the obtained clustered image blocks meet the preset conditions to obtain a registered image with higher accuracy.

[0116] With respect to the above preset conditions, in an optional embodiment, the clustered image blocks satisfy the preset conditions, including:

[0117] Determine image blocks of the first image that correspond one-to-one to the clustered image blocks, compare the clustered image blocks with the corresponding image blocks of the first image, and determine the number of changed pixels in the clustered image blocks;

[0118] When the ratio between the number of changed pixels and the size of the first image is less than a second preset threshold, the clustered image blocks meet the preset conditions.

[0119] In an optional embodiment, the clustered image blocks do not meet a preset condition, including:

[0120] When the ratio between the number of changed pixels and the size of the first image is greater than or equal to the second preset threshold, the clustered image block does not meet the preset condition.

[0121] Specifically, first, the clustered image blocks are matched one by one with the image blocks of the first image. Specifically, for the first image block after clustering, an image block with the least number of changed pixels compared with the first image block is found in the image blocks of the first image, and the two are matched. Then, for the second image block, an image block with the least number of changed pixels compared with the second image block is found in the remaining image blocks of the first image, and the two are matched. And so on, the image blocks of the first image that correspond one by one to the clustered image blocks are determined, and then the number of changed pixels in the clustered image blocks is obtained by comparison. For example, for an image block in the clustered image blocks, the corresponding image block is found in the image blocks of the first image, and then the disappeared pixels and newly added pixels in the clustered image blocks, that is, the changed pixels, are determined, so that the number of changed pixels can be counted.

[0122] Then calculate the ratio between the number of changed pixels and the size of the first image, and compare the relationship between the ratio and the second preset threshold time. Only when the ratio is less than the second preset threshold, it is determined that the clustered image blocks meet the preset conditions, and the calculated movement amount of the image blocks of the first image can be used to move the first image to obtain the registered image; otherwise, it is determined that the clustered image blocks do not meet the preset conditions, and the image blocks of the first image are updated with the clustered image blocks, so as to update the blocking method, thereby re-determining the movement amount of the image blocks of the first image, until the clustered image blocks meet the preset conditions to obtain a more accurate registered image.

[0123] After obtaining the registered image, there may be overlapping pixels and vacant pixels. In an optional embodiment, after S106, the method may further include:

[0124] When there are overlapping pixels in the registered image, a pixel value is selected from the pixel values ​​of the first image corresponding to the overlapping pixel, and the selected pixel value is determined as the pixel value of the overlapping pixel.

[0125] When there are overlapping pixels in the registered image, that is, after the first image is moved, some pixels move to the pixel points at the same position. In this way, there are more than one pixel value for the overlapping pixels in the registered image. In order to determine the pixel value for the overlapping pixels, a pixel value can be arbitrarily selected from the pixel values ​​of the first image corresponding to the overlapping pixels as the pixel value of the overlapping pixel, or a pixel value can be selected from the pixel values ​​of the first image corresponding to the overlapping pixels in a preset manner as the pixel value of the overlapping pixel. Of course, the pixel values ​​of the overlapping pixels can also be calculated by some preset algorithms. Here, the embodiments of the present application do not make specific limitations on this.

[0126] In order to determine the pixel value of the overlapping pixel point, in an optional embodiment, selecting a pixel value from the pixel values ​​of the first image corresponding to the overlapping pixel point, and determining the selected pixel value as the pixel value of the overlapping pixel point includes:

[0127] Obtain at least two pixel values ​​corresponding to overlapping pixel points;

[0128] respectively determining differences between at least two pixel values ​​and pixel values ​​of the second image corresponding to the overlapping pixel points;

[0129] The pixel value of the overlapping pixel point is determined according to the relationship between the absolute values ​​of the differences between the at least two pixel values ​​and the pixel value of the second image corresponding to the overlapping pixel point.

[0130] Specifically, at least two pixel values ​​corresponding to the overlapping pixel points are first obtained, and then the difference between each pixel value and the pixel value of the second image corresponding to the overlapping pixel point is calculated, and the absolute value is calculated. Finally, the pixel value of the overlapping pixel point is determined based on the relationship between the absolute values ​​of the difference between the at least two pixel values ​​and the pixel value of the second image corresponding to the overlapping pixel point. For example, the pixel value with a smaller absolute value is determined as the overlapping pixel value.

[0131] For vacant pixels, in an optional embodiment, after S106, the method may further include:

[0132] When there are vacant pixels in the registration image, the pixel value of the vacant pixel is determined by using a preset algorithm according to the pixel values ​​of the adjacent pixels of the vacant pixel;

[0133] Among them, the vacant pixel refers to the pixel whose pixel value does not exist.

[0134] When there are vacant pixels in the registered image, that is, there are pixels whose pixel values ​​do not exist in the registered image, in order to obtain the pixel values ​​of the vacant pixels, the adjacent pixels of the vacant pixels can be obtained first, and the pixel values ​​of the vacant pixels can be calculated according to a preset interpolation algorithm or a global motion model, so as to obtain a more accurate and complete registered image.

[0135] The following example is used to illustrate the image registration method described in one or more of the above embodiments.

[0136] Figure 3 A flowchart of an example of an optional image registration method provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the image registration method may include:

[0137] S301: Correcting depth information and matching depth information with image information;

[0138] Specifically, two or more images (videos) and depth maps (depth information, equivalent to the above-mentioned depth value) are captured by the terminal's camera. When the two images are registered here, one image is used as the image to be registered and the other image is used as the reference image. The image after registration is the registered image.

[0139] After the image to be registered and the reference image are acquired, the image information and depth information of the image to be registered can be obtained, wherein the image information can be an RGB value or grayscale information. Here, grayscale information is taken as an example.

[0140] Since the depth information may be a dot matrix or a depth map, the depth information needs to be preprocessed and corrected. If it is a dot matrix, it is necessary to restore the surface information of the object through triangular plane reconstruction, and perform smoothing and interpolation processing; if it is a depth map, it is necessary to detect bad pixels and denoise to ensure that the obtained depth information has good accuracy.

[0141] After correcting the depth information, the depth map and grayscale information need to be matched. Since depth information often does not have a higher resolution than grayscale information, the depth information needs to be upsampled and then matched; if the depth information and grayscale information have not been calibrated, they need to be calibrated and calibrated to ensure the matching accuracy. When upsampling, edge accuracy can be ensured by guided filtering and other methods. After completing the matching, the depth information of all pixels in the image can be obtained.

[0142] S302: Divide the image to be registered into blocks;

[0143] After obtaining the depth information of each pixel, the image to be registered needs to be divided into blocks. The first block division can divide the image to be registered into ordinary rectangular blocks. The appropriate image block size can be given to evenly distribute the image blocks to the whole image. The size of the image block is not fixed during the algorithm. Figure 4 A schematic diagram of an optional process for dividing an image to be registered into blocks is provided in an embodiment of the present application, such as Figure 4 As shown in FIG. 1 , after obtaining the image to be registered and the reference image, the image to be registered is first divided into blocks. Here, the image to be registered is divided into blocks according to a fixed size to obtain image blocks of the image to be registered. Figure 4 The image pointed by the middle arrow is the image to be registered after segmentation. It can be seen that the image to be registered is segmented into several rectangular blocks.

[0144] S303: searching the reference image, finding the image block with the content closest to the block, and obtaining the movement amount of the image block;

[0145] Select the image blocks on the image to be registered one by one, and search for the image blocks of the same size and position on the moving reference image. Figure 5 This is a schematic diagram of an optional structure for searching a reference image provided in an embodiment of the present application, such as Figure 5 As shown, the current macroblock is searched in the search window, that is, the input reference image is searched in the search window until the image block with the content closest to the image block to be registered is found. The similarity of the image blocks can be evaluated by the SAD method. If the overall grayscale information of the image to be registered and the reference image is quite different, the NCC indicator can also be used to evaluate the similarity of the image blocks.

[0146] In addition, after the search is completed, the position difference between the image block to be registered and the similar reference image block is used as the movement amount of the image block to be registered. Figure 6a A schematic diagram of an optional structure of an image to be registered provided in an embodiment of the present application, Figure 6b A schematic diagram of an optional reference image structure provided in an embodiment of the present application, such as Figure 6a and Figure 6b As shown, in Figure 6a In the figure, for an image block in the image to be registered, the position coordinates of the image block are (x1, y1), and the position of the similar image block (in the reference image) of the image block is as follows: Figure 6b As shown, the position coordinates are (x2, y2), so the movement amount mv of the image block can be obtained:

[0147] mv = [x1-x2, y1-y2] (1)

[0148] Since there is no overlap between the image blocks, the movement amount can be used as the movement amount of all pixels in the image block, thus obtaining the movement amount of all pixels in the image to be registered.

[0149] S304: Correcting the movement amount of the image block;

[0150] Since the block search method has a low granularity, the obtained pixel movement amount has a low accuracy, and the movement amount of all pixels will be further corrected and refined.

[0151] Figure 7a A schematic diagram of another optional structure of the image to be registered provided in the embodiment of the present application, such as Figure 7a As shown in the figure, for a certain pixel point A, the position P and pixel value BV of the pixel point are obtained, and the movement amount of the pixel points within the radius r from the pixel point A is sampled to obtain N different movement amounts mv0~mv N-1 , move the pixel point A according to these movement amounts to obtain the new position P+mv, Figure 7b A schematic diagram of another optional reference image structure provided in an embodiment of the present application is shown in FIG. Figure 7b As shown, the pixel values ​​RV0~RV of the reference image at the new position (BCD position in the figure) are obtained. N-1 , Figure 7c A schematic diagram of another optional reference image structure provided in the embodiment of the present application is as follows: Figure 7c As shown, the pixel value difference diff = |RV-BV| is calculated, and N different movement amounts mv0~mv are calculated based on the pixel value difference. N-1 Perform weighted summation to get the new mv.

[0152] S305: integrating the position information, movement amount and depth information and re-clustering to obtain a new image block;

[0153] Through the above method, we can get the depth information d and the movement amount [mv x , mv y ] and position information [x, y], normalize all data, and multiply each data by weight w to get the high-dimensional array [x*w1, y*w2, mv x *w3,mv y *w4, d*w5], cluster the array. The clustering method can be the mean shift algorithm, or the number of clusters can be specified in advance and the K-means clustering algorithm can be used for clustering. Figure 8a A schematic diagram of the structure of an optional image block to be registered provided in an embodiment of the present application, Figure 8b A schematic diagram of the structure of an optional clustered image block provided in an embodiment of the present application, such as Figure 8a and Figure 8b As shown, the pixels clustered into one category are redefined as new image blocks (such as Figure 8b ), for all classes after clustering, define the threshold thresh2. If the number of pixels in this class is less than thresh2, discard this class and reclassify the pixels of this class into other classes that are closest to this pixel.

[0154] S306: Determine whether it converges. If it converges, execute S307. If it does not converge, return to execute S303.

[0155] The obtained new image block is compared with the old image block, and the number of pixel changes is counted. A counter is used for counting. First, the counter is reset to 0. For each pixel in the old image block, if it does not belong to the new image block, the counter is increased by 1. After counting all the pixels in the old image block, the number of changed pixels m is obtained. If the size of the old image block is n, the ratio of pixel change m / n can be obtained. A threshold thresh1 is set. If the ratio is less than thresh1, the algorithm is determined to have converged, and the movement amount of pixels in the entire image is obtained. S307 is executed. Otherwise, the algorithm is continued. If it is determined not to have converged, the search is performed again according to the obtained new image block in the manner of S303.

[0156] S307: Realigning the images to obtain the registered images.

[0157] According to the above method, the movement amount of all pixels is obtained, and the pixels are moved according to the movement amount of all pixels to obtain the registered image, that is, the registered image. The moved image may overlap. Figure 9a A schematic diagram of the structure of overlapping pixels of an optional registration image provided in an embodiment of the present application, such as Figure 9aAs shown, two or more pixels have moved to the same position. Figure 9b A schematic diagram of the structure of vacant pixels of an optional registration image provided in an embodiment of the present application is shown in FIG. Figure 9b As shown, there are holes in the registered image, that is, no pixels move to this position. For the overlapping case, such as Figure 9a As shown, for example, if two pixels with pixel values ​​of P1 and P2 are moved to the same position, the pixel value Pref of the reference image at that position is obtained, and the pixel value difference is calculated. If |P1-Pref|<|P2-Pref|, the pixel value of the registered image at that position is set to P1, otherwise it is set to P2. For the case of a hole (equivalent to an empty pixel point), the pixel at that location can be filled by interpolation, or the global movement model (such as affine matrix or homography matrix) can be calculated to calculate the pixel value of the hole.

[0158] From the above examples, we can see that using depth information to assist registration is more accurate than using only image information. For moving objects, model estimation is not used (the motion model of a local moving object is difficult to match with a single model), but the flow of a single pixel is calculated, which makes the registration of the moving object area more accurate. The search uses irregular block search, which makes the matching of objects with irregular edges more accurate.

[0159] That is to say, the embodiment of the present application adopts a method similar to dense optical flow estimation to provide motion estimates of all pixels and rematch the image. It does not rely on a single motion model, and introduces scene depth information and irregular block matching in the search, which is more adaptable to complex motion scenes.

[0160] The embodiment of the present application provides an image registration method, which obtains a first image and a second image, divides the first image into blocks to obtain image blocks of the first image, calls a preset image block similarity algorithm based on pixel values ​​of the image blocks of the first image, determines similar image blocks of the image blocks of the first image from the second image, calculates the movement amount of the image blocks of the first image based on position information of the image blocks of the first image and position information of similar image blocks of the image blocks of the first image, clusters the first image based on the depth value of the first image, the movement amount of the image blocks of the first image and the position information of the first image, generates clustered image blocks, and when the clustered image blocks meet preset conditions, moves the image blocks of the first image based on the movement amount of the image blocks of the first image to obtain a registered image; that is, In the embodiment of the present application, an image block similar to an image block of the first image is searched in the second image to calculate the movement amount of the image block of the first image, and then the first image is clustered using the depth value of the first image, the movement amount of the image block of the first image and the position information of the first image, so as to generate clustered image blocks. In this way, when the clustered image blocks meet the preset conditions, the first image is moved by the movement of the image blocks of the first image to obtain a registered image. In this way, the depth information of the first image is taken into account in the image registration to form the clustered image blocks. Only when the clustered image blocks meet the preset conditions are they moved according to the movement amount of the image blocks of the first image to obtain the registered image, thereby improving the accuracy of the registered image and making the effect of the obtained registered image better.

[0161] Embodiment 2

[0162] Based on the same inventive concept, an embodiment of the present application provides a terminal, Fig.10 A schematic diagram of an optional terminal structure provided in an embodiment of the present application Figure 1 ,like Fig.10 As shown, the terminal may include: an acquisition module 101, a block module 102, a determination module 103, a calculation module 104, a clustering module 105 and a movement module 106; wherein,

[0163] An acquisition module 101 is configured to acquire a first image and a second image;

[0164] A block division module 102 is configured to divide the first image into blocks to obtain image blocks of the first image;

[0165] The determination module 103 is configured to call a preset image block similarity algorithm according to the pixel value of the image block of the first image, and determine similar image blocks of the image block of the first image from the second image;

[0166] A calculation module 104, configured to calculate a movement amount of the image block of the first image according to position information of the image block of the first image and position information of similar image blocks of the image block of the first image;

[0167] The clustering module 105 is configured to cluster the first image according to the depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate clustered image blocks;

[0168] The moving module 106 is configured to move the image block of the first image according to the movement amount of the image block of the first image when the clustered image block meets a preset condition, so as to obtain a registered image.

[0169] In an optional embodiment, the terminal is further configured as:

[0170] When the clustered image blocks do not meet the preset conditions, the clustered image blocks are determined as the image blocks of the first image to update the image blocks of the first image, and the process returns to execute based on the pixel values ​​of the image blocks of the first image, calling a preset image block similarity algorithm, and determining similar image blocks of the image blocks of the first image from the second image.

[0171] In an optional embodiment, when the resolution of the pixel value of the first image is different from the resolution of the depth value of the first image, accordingly, the terminal is further configured as follows:

[0172] Before clustering the first image according to the depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate the clustered image block, scaling the resolution of the depth value of the first image according to the resolution of the pixel value of the first image to obtain the processed depth value of the first image;

[0173] Accordingly, the clustering module 105 is specifically configured as follows:

[0174] The first image is clustered according to the processed depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate clustered image blocks.

[0175] In an optional embodiment, the calculation module 104 is specifically configured as follows:

[0176] Determine the difference between the horizontal and vertical coordinates of the image block of the first image and the horizontal and vertical coordinates of the image block similar to the image block of the first image as the initial movement amount of the pixel point of the image block of the first image;

[0177] The initial movement amount of the pixel points of the image block of the first image is corrected to obtain the movement amount of the image block of the first image.

[0178] In an optional embodiment, the calculation module 104 corrects the initial movement amount of the pixel points of the image block of the first image to obtain the movement amount of the image block of the first image, including:

[0179] Taking the position information of any selected pixel point in the image block of the first image as a reference, sampling is performed from a preset range of the first image to obtain a preset number of pixel points;

[0180] According to the initial movement amount of a preset number of pixel points, the position information of the selected pixel points is moved to generate the moved pixel points;

[0181] Obtaining from the second image the pixel value of the pixel point at the same position as the moved pixel point;

[0182] Determine the weight of the moved pixel based on the pixel value of the pixel at the same position;

[0183] According to the weights of the moved pixels, the initial movement amounts of the moved pixels are weighted summed to obtain the movement amount of the image block of the first image.

[0184] In an optional embodiment, the calculation module 104 determines the weight of the moved pixel point according to the pixel value of the pixel point at the same position, including:

[0185] Calculate the absolute value of the difference between the pixel value of the pixel point at the same position and the pixel value of the selected pixel point;

[0186] The weight corresponding to the absolute value of the difference between the pixel value of the pixel point at the same position and the pixel value of the selected pixel point is determined as the weight of the pixel point after the move.

[0187] In an optional embodiment, the clustering module 105 is specifically configured as follows:

[0188] Determine the movement amount of the image block of the first image as the movement amount of the pixel points of the image block of the first image;

[0189] Using the depth value corresponding to the pixel point of the first image, the movement amount of the pixel point of the first image and the position information of the pixel point of the first image, multiplying by the corresponding weight value, respectively, to form an array of the first image;

[0190] According to a preset clustering algorithm, the array of the first image is clustered to generate clustered image blocks.

[0191] In an optional embodiment, the clustering module 105 clusters the array of the first image according to a preset clustering algorithm to generate clustered image blocks, including:

[0192] Clustering the array of the first image according to a preset clustering algorithm to obtain an initial clustered image block;

[0193] Selecting an image block whose number of pixels is less than a first preset threshold from the initial clustered image blocks;

[0194] The selected image blocks are reclassified to obtain clustered image blocks.

[0195] In an optional embodiment, the clustering module 105 reclassifies the selected image blocks to obtain clustered image blocks, including:

[0196] Determine, from the initial clustered image blocks, an image block that is closest to the selected image block;

[0197] The category of the selected image block is assigned to the image block that is closest to the selected image block to obtain a clustered image block.

[0198] In an optional embodiment, the clustered image blocks meet a preset condition, including:

[0199] Determine image blocks of the first image that correspond one-to-one to the clustered image blocks, compare the clustered image blocks with the corresponding image blocks of the first image, and determine the number of changed pixels in the clustered image blocks;

[0200] When the ratio between the number of changed pixels and the size of the first image is less than a second preset threshold, the clustered image blocks meet the preset conditions.

[0201] In an optional embodiment, the clustered image blocks do not meet a preset condition, including:

[0202] When the ratio between the number of changed pixels and the size of the first image is greater than or equal to a second preset threshold, the clustered image block does not meet the preset condition.

[0203] In an optional embodiment, the terminal is further configured as:

[0204] When the clustered image blocks meet the preset conditions, the image blocks of the first image are moved according to the movement amount of the image blocks of the first image to obtain the registered image. When there are overlapping pixels in the registered image, a pixel value is selected from the pixel values ​​of the first image corresponding to the overlapping pixels, and the selected pixel value is determined as the pixel value of the overlapping pixel.

[0205] In an optional embodiment, the terminal selects a pixel value from the pixel values ​​of the first image corresponding to the overlapping pixel points, and determines the selected pixel value as the pixel value of the overlapping pixel point, including:

[0206] Obtain at least two pixel values ​​corresponding to overlapping pixel points;

[0207] respectively determining differences between at least two pixel values ​​and pixel values ​​of the second image corresponding to the overlapping pixel points;

[0208] The pixel value of the overlapping pixel point is determined according to the relationship between the absolute values ​​of the differences between the at least two pixel values ​​and the pixel value of the second image corresponding to the overlapping pixel point.

[0209] In an optional embodiment, the terminal is further configured as:

[0210] When the clustered image blocks meet the preset conditions, the image blocks of the first image are moved according to the movement amount of the image blocks of the first image to obtain the registered image. When there are vacant pixels in the registered image, the pixel values ​​of the vacant pixels are determined by using a preset algorithm according to the pixel values ​​of the adjacent pixels of the vacant pixels;

[0211] Among them, the vacant pixel refers to the pixel whose pixel value does not exist.

[0212] In practical applications, the acquisition module 101, the blocking module 102, the determination module 103, the calculation module 104, the clustering module 105 and the mobile module 106 can be implemented by a processor located on the terminal, specifically a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA).

[0213] Fig.11 A schematic diagram of an optional terminal structure provided in an embodiment of the present application Figure 2 ,like Fig.11 As shown, the embodiment of the present application provides a terminal 1100, including:

[0214] A processor 111 and a storage medium 112 storing executable instructions of the processor 111, wherein the storage medium 112 relies on the processor 111 to perform operations through a communication bus 113, and when the instructions are executed by the processor 111, the image registration method executed by the above-mentioned image processor is executed.

[0215] It should be noted that in actual application, the various components in the terminal are coupled together through the communication bus 113. It is understandable that the communication bus 113 is used to realize the connection and communication between these components. In addition to the data bus, the communication bus 113 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Fig.11 Various buses are labeled as communication buses 113.

[0216] An embodiment of the present application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the image registration method executed by the terminal in one or more of the above embodiments.

[0217] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM). The memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0218] The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor to be executed. The software module may be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0219] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0220] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0221] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0222] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0223] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0224] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0225] Industrial Applicability

[0226] The embodiments of the present application provide an image registration method, a terminal, and a computer storage medium, which obtain a first image and a second image, divide the first image into blocks to obtain image blocks of the first image, call a preset image block similarity algorithm based on pixel values ​​of the image blocks of the first image, determine similar image blocks of the image blocks of the first image from the second image, calculate the movement amount of the image blocks of the first image based on position information of the image blocks of the first image and position information of similar image blocks of the image blocks of the first image, cluster the first image based on the depth value of the first image, the movement amount of the image blocks of the first image, and the position information of the first image, generate clustered image blocks, and when the clustered image blocks meet preset conditions, move the image blocks of the first image based on the movement amount of the image blocks of the first image to obtain a registered image, thereby improving the accuracy of image registration.

Claims

1. A method for image registration, wherein: include: Acquire a first image and a second image; Dividing the first image into blocks to obtain image blocks of the first image; According to the pixel values ​​of the image blocks of the first image, calling a preset image block similarity algorithm, and determining similar image blocks of the image blocks of the first image from the second image; calculating a movement amount of the image block of the first image according to position information of the image block of the first image and position information of a similar image block of the image block of the first image; Clustering the first image according to the depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate clustered image blocks; When the clustered image blocks meet a preset condition, the image blocks of the first image are moved according to the movement amount of the image blocks of the first image to obtain a registered image.

2. The method according to claim 1, wherein: The method further comprises: When the clustered image blocks do not meet the preset conditions, the clustered image blocks are determined as the image blocks of the first image to update the image blocks of the first image, and the process returns to execute the process according to the pixel values ​​of the image blocks of the first image, calling the preset image block similarity algorithm, and determining similar image blocks of the image blocks of the first image from the second image.

3. The method according to claim 1, wherein: When the resolution of the pixel value of the first image is different from the resolution of the depth value of the first image, accordingly, before clustering the first image according to the depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate the clustered image block, the method further includes: Scaling the resolution of the depth value of the first image according to the resolution of the pixel value of the first image to obtain a depth value of the first image after processing; Correspondingly, clustering the first image according to the depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate the clustered image blocks includes: The first image is clustered according to the processed depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate clustered image blocks.

4. The method according to claim 1, wherein: The calculating, according to the position information of the image block of the first image and the position information of the image block similar to the image block of the first image, the movement amount of the image block of the first image comprises: Determine the difference between the horizontal and vertical coordinates of the image block of the first image and the horizontal and vertical coordinates of the image block similar to the image block of the first image as the initial movement amount of the pixel point of the image block of the first image; The initial movement amount of the pixel points of the image block of the first image is corrected to obtain the movement amount of the image block of the first image.

5. The method according to claim 4, wherein: The correcting the initial movement amount of the pixel points of the image block of the first image to obtain the movement amount of the image block of the first image includes: Taking the position information of any selected pixel point in the image block of the first image as a reference, sampling is performed from a preset range of the first image to obtain a preset number of pixel points; According to the initial movement amount of the preset number of pixel points, the position information of the selected pixel points is moved to generate the moved pixel points; Acquire a pixel value of a pixel point at the same position as the moved pixel point from the second image; Determine the weight of the moved pixel based on the pixel value of the pixel at the same position; According to the weights of the moved pixels, the initial movement amounts of the moved pixels are weighted summed to obtain the movement amount of the image block of the first image.

6. The method according to claim 5, wherein: Determining the weight of the moved pixel point according to the pixel value of the pixel point at the same position includes: Calculating the absolute value of the difference between the pixel value of the pixel point at the same position and the pixel value of the selected pixel point; The weight corresponding to the absolute value of the difference between the pixel value of the pixel point at the same position and the pixel value of the selected pixel point is determined as the weight of the moved pixel point.

7. The method according to claim 1, wherein: The clustering of the first image according to the depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate the clustered image blocks includes: Determine the movement amount of the image block of the first image as the movement amount of the pixel points of the image block of the first image; Using the depth values ​​corresponding to the pixels of the first image, the movement amounts of the pixels of the first image and the position information of the pixels of the first image, respectively multiplied by corresponding weight values, to form an array of the first image; According to a preset clustering algorithm, the array of the first image is clustered to generate clustered image blocks.

8. The method according to claim 7, wherein: The step of clustering the array of the first image according to a preset clustering algorithm to generate clustered image blocks includes: Clustering the array of the first image according to a preset clustering algorithm to obtain an initial clustered image block; Selecting an image block whose number of pixels is less than a first preset threshold from the initial clustered image blocks; The selected image blocks are reclassified to obtain clustered image blocks.

9. The method according to claim 8, wherein: The step of reclassifying the selected image blocks to obtain clustered image blocks includes: Determine, from the initial clustered image blocks, an image block that is closest to the selected image block; The category of the selected image block is assigned to the image block closest to the selected image block to obtain a clustered image block.

10. The method according to claim 1 or 2, wherein: The clustered image blocks meet preset conditions, including: Determine image blocks of the first image that correspond one-to-one to the clustered image blocks, compare the clustered image blocks with the corresponding image blocks of the first image, and determine the number of changed pixels in the clustered image blocks; When the ratio between the number of changed pixels and the size of the first image is less than a second preset threshold, the clustered image blocks meet the preset condition.

11. The method according to claim 10, wherein: The clustered image blocks do not meet the preset condition, including: When the ratio between the number of changed pixels and the size of the first image is greater than or equal to the second preset threshold, the clustered image block does not meet the preset condition.

12. The method according to claim 1, wherein: When the clustered image blocks meet a preset condition, the image blocks of the first image are moved according to the movement amount of the image blocks of the first image to obtain the registered image, the method further includes: When there are overlapping pixels in the registered image, a pixel value is selected from the pixel values ​​of the first image corresponding to the overlapping pixel, and the selected pixel value is determined as the pixel value of the overlapping pixel.

13. The method according to claim 12, wherein: The step of selecting a pixel value from the pixel values ​​of the first image corresponding to the overlapping pixel points, and determining the selected pixel value as the pixel value of the overlapping pixel point, comprises: Obtaining at least two pixel values ​​corresponding to the overlapping pixel points; respectively determining differences between at least two pixel values ​​and a pixel value of the second image corresponding to the overlapping pixel point; The pixel value of the overlapping pixel point is determined according to the relationship between the absolute values ​​of the differences between at least two pixel values ​​and the pixel value of the second image corresponding to the overlapping pixel point.

14. The method according to claim 1, wherein: When the clustered image blocks meet a preset condition, the image blocks of the first image are moved according to the movement amount of the image blocks of the first image to obtain the registered image, the method further includes: When there are vacant pixels in the registered image, a preset algorithm is used to determine the pixel value of the vacant pixel according to the pixel values ​​of the adjacent pixels of the vacant pixel; The vacant pixel refers to a pixel whose pixel value does not exist.

15. An image registration terminal, wherein: include: An acquisition module, configured to acquire a first image and a second image; A block division module, configured to divide the first image into blocks to obtain image blocks of the first image; a determination module configured to call a preset image block similarity algorithm according to pixel values ​​of the image blocks of the first image, and determine similar image blocks of the image blocks of the first image from the second image; a calculation module configured to calculate a movement amount of the image block of the first image according to position information of the image block of the first image and position information of similar image blocks of the image block of the first image; a clustering module configured to cluster the first image according to the depth value of the first image, the movement amount of the image block of the first image and the position information of the first image to generate clustered image blocks; The moving module is configured to move the image block of the first image according to the movement amount of the image block of the first image when the clustered image block meets a preset condition, so as to obtain a registered image.

16. A terminal, wherein: include: A processor and a storage medium storing instructions executable by the processor, wherein the storage medium relies on the processor to perform operations via a communication bus, and when the instructions are executed by the processor, the image registration method described in any one of claims 1 to 14 is executed.

17. A computer-readable storage medium, wherein: Executable instructions are stored, and when the executable instructions are executed by one or more processors, the processors execute the image registration method described in any one of claims 1 to 14.