An image registration method, electronic device, and storage medium

Abstract

The application discloses an image registration method, an electronic device and a storage medium. The method comprises the following steps: determining a relative coordinate set; the relative coordinate set comprises relative coordinates of at least part of pixel points between a source image and a target image to be generated, and the target image represents an image obtained by registering the source image; and the relative coordinate set is sent to a registration device, so that the registration device performs image registration based on the relative coordinates in the relative coordinate set. In this way, the image registration efficiency can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image registration method, electronic device, and storage medium. Background Technology

[0002] Image registration is a crucial step in image processing and analysis, serving as a prerequisite for image comparison, data fusion, change analysis, and target recognition. Existing image registration methods fall into two main categories: region-based registration methods, which utilize the grayscale information of regions and do not rely on image features; and feature-based registration methods, which require extracting fixed features from the image and obtaining matching parameters by analyzing the matching degree between features. Both of these methods involve complex calibration processes (i.e., calibration is relatively infrequent) and are best suited for processing on a CPU (Central Processing Unit). The final fusion calculation using the registration parameters can be implemented on a FPGA (Field Programmable Gate Array).

[0003] After the CPU calculates the registration parameters, the registration parameters need to be sent to the FPGA. The existing technology sends the absolute coordinates to the FPGA, which has the problem of excessive time in sending the registration parameters, resulting in low efficiency of image registration. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide an image registration method, electronic device, and storage medium that can improve image registration efficiency.

[0005] To address the aforementioned technical problems, the first aspect of this application provides an image registration method, which includes: determining a relative coordinate set; the relative coordinate set includes the relative coordinates of at least some pixels between a source image and a target image to be generated, the target image representing an image obtained by registering the source image; and sending the relative coordinate set to a registration device, so that the registration device performs image registration based on the relative coordinates in the relative coordinate set.

[0006] To address the aforementioned technical problems, a second aspect of this application provides an image registration method. The method includes: acquiring a relative coordinate set sent by a sending device, the relative coordinate set including the relative coordinates of at least some pixels between a source image and a target image to be generated, the target image representing an image registered with the source image; determining the absolute coordinates of each first pixel in the target image within the source image based on the relative coordinate set; and obtaining the target image based on the absolute coordinates of each first pixel and the pixel values ​​of the source image.

[0007] To address the aforementioned technical problems, a third aspect of this application provides an electronic device comprising: a first memory and a sending device, wherein the first memory stores program instructions, and the sending device executes the program instructions to implement the image registration method described in the first aspect.

[0008] To address the aforementioned technical problems, a fourth aspect of this application provides an electronic device comprising: a second memory and a registration device; the second memory stores program instructions; and the registration device executes the program instructions to implement the image registration method described in the second aspect.

[0009] To address the aforementioned technical problems, the fifth aspect of this application provides a computer-readable storage medium storing program instructions executable by a processor, the program instructions being used to implement the image registration method in the first or second aspect described above.

[0010] The beneficial effects of this application are as follows: Unlike existing technologies, this application determines a relative coordinate set; the relative coordinate set includes the relative coordinates of at least some pixels between the source image and the target image to be generated, where the target image represents the image obtained by registering the source image; the relative coordinate set is sent to the registration device, enabling the registration device to perform image registration based on the relative coordinates in the relative coordinate set. In the above method, the sending device sends a relative coordinate set, which reduces the amount of data sent compared to existing technologies, thus reducing the time for sending registration parameters and improving image registration efficiency. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating the first embodiment of the image registration method provided in this application;

[0012] Figure 2 This is a flowchart illustrating the second embodiment of the image registration method provided in this application;

[0013] Figure 3 This is a flowchart illustrating the third embodiment of the image registration method provided in this application;

[0014] Figure 4 This is a simplified schematic diagram of one implementation method for obtaining a relative coordinate set;

[0015] Figure 5 This is a schematic diagram of a storage method for relative coordinate sets in memory;

[0016] Figure 6 This is a schematic diagram of obtaining the absolute coordinates of the target pixel by interpolating the absolute coordinates of at least one first associated pixel.

[0017] Figure 7This is a schematic diagram of the framework structure of one embodiment of the electronic device provided in this application;

[0018] Figure 8 This is a schematic diagram of another embodiment of the electronic device provided in this application;

[0019] Figure 9 This is a schematic diagram of a framework of one embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0021] It should be noted that the embodiments of this application contain descriptions involving "first," "second," etc., which are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] Please see Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the image registration method provided in this application. The method includes:

[0024] S110: Determine the relative coordinate set; the relative coordinate set includes the relative coordinates of at least some pixels between the source image and the target image to be generated, and the target image represents the image obtained by registering the source image.

[0025] In one embodiment, the relative coordinate set can be obtained using a first reference image and a second reference image. The first and second reference images are acquired from the same object under different acquisition conditions. Any pixel in the second reference image is taken as a third pixel, and the relative coordinates of each third pixel in the second reference image represent the relative coordinates between the corresponding first pixel in the target image and the corresponding second pixel in the source image. The relative coordinates of each third pixel are obtained by subtracting the absolute coordinates of each third pixel in the first reference image from their sequential coordinates in the second reference image. In other words, the relative coordinates between each first pixel in the target image and the corresponding second pixel in the source image are obtained by subtracting the absolute coordinates of each third pixel in the first reference image from their sequential coordinates in the second reference image. The relative coordinate set includes the relative coordinates of at least some pixels between the source image and the target image to be generated. That is, the relative coordinate set contains the relative coordinates of several third pixels. Among them, at least some pixels can be all pixels or some pixels. The sequential coordinates of each third pixel in the second reference image are obtained by device calibration. For example, the sequential coordinates of the third pixel in the first row and first column of the second reference image are labeled as (0,0), the sequential coordinates of the third pixel in the first row and second column are (0,1), and so on, to obtain the sequential coordinates of all third pixels in the second reference image.

[0026] The first reference image and the source image are acquired under the same conditions. The first reference image and the source image can be the same image or different images. The target image represents the image acquired using the same acquisition conditions as the second reference image. These acquisition conditions may include: acquisition angle, acquisition device, etc.

[0027] Specifically, obtaining the relative coordinate set using the first reference image and the second reference image includes: acquiring spatial transformation parameters between the first and second reference images; performing a spatial transformation on the second reference image using the spatial transformation parameters to obtain the absolute coordinates of each third pixel in the second reference image within the first reference image. Based on the deviation between the absolute coordinates of at least one third pixel in the first reference image and the sequential coordinates of at least one third pixel in the second reference image, the relative coordinates of at least one third pixel are determined. For example, if the absolute coordinates of the third pixel in the first reference image are (4094.5, 2149.8) and the sequential coordinates of the third pixel in the second reference image are (4090, 2159), then the relative coordinates of the third pixel are (+4.5, -9.2); as another example, if the absolute coordinates of the third pixel in the first reference image are (4.8, 11.5) and the sequential coordinates of the third pixel in the second reference image are (5.10), then the relative coordinates of the third pixel are (-0.2, +1.5). Therefore, even if the image resolution is high, the resulting relative coordinate values ​​are small, resulting in a smaller amount of second data for the relative coordinates. Storing this data in memory can significantly reduce the amount of storage required.

[0028] When obtaining the spatial transformation parameters between the first and second reference images, the device-related parameters of the acquisition devices for the first and second reference images can be obtained first. These device-related parameters may include intrinsic parameter matrices, distortion matrices, projection matrices, rotation matrices, etc., and then the spatial transformation parameters can be obtained using these parameters. In one specific embodiment, for each pixel in the second reference image, projection parameters can be obtained using the projection matrix and rotation matrix; the projection transformation result can be obtained using the projection parameters and the sequential coordinates of each third pixel in the second reference image; the distortion correction result can be obtained using the projection transformation result and the distortion matrix; and the spatial transformation parameters can be obtained using the distortion correction result and the intrinsic parameter matrix.

[0029] The relative coordinates of each third pixel represent the relative coordinates between the corresponding first pixel in the target image and the corresponding second pixel in the source image. The relative coordinate set, composed of the relative coordinates of each third pixel, can be understood to include the relative coordinates of some third pixels in the second reference image, or it can include the relative coordinates of all third pixels in the second reference image.

[0030] In one embodiment, the relative coordinate set includes the relative coordinates between a plurality of first pixels in the target image and corresponding second pixels in the source image, wherein the number of the plurality of first pixels is less than the total number of pixels in the target image. That is, the relative coordinate set includes the relative coordinates of a portion of the third pixels in the second reference image. After obtaining the absolute coordinates of all the third pixels in the second reference image, the absolute coordinates of all the third pixels can be downsampled to obtain the absolute coordinates of a portion of the third pixels, and then the relative coordinates of the portion of the third pixels can be calculated. In another embodiment, after obtaining the relative coordinates of all the third pixels in the second reference image, the relative coordinates of all the third pixels can be downsampled by a preset multiple to obtain the relative coordinates of the portion of the third pixels.

[0031] S120: Send the relative coordinate set to the registration device so that the registration device can perform image registration based on the relative coordinates in the relative coordinate set.

[0032] In one embodiment, after acquiring the relative coordinate set, the registration device determines the absolute coordinates of each first pixel in the target image in the source image based on the relative coordinates in the relative coordinate set; based on the absolute coordinates of each first pixel and the pixel values ​​of the source image, the target image is obtained. The specific registration process of the registration device is described below, and will not be detailed here.

[0033] In this embodiment, the step of determining the relative coordinate set can be performed by the sending device. That is, the sending device obtains the relative coordinate set using the first reference image and the second reference image, and sends the relative coordinate set to the registration device. The sending device can be a CPU, and the registration device can be an FPGA. The processing power of the sending device is higher than that of the registration device. In one embodiment, the transmission bit width between the sending device and the registration device is a first data amount, such as 32 bits; the relative coordinate of each first pixel is a second data amount, such as 16 bits. The coordinate data sent by the sending device each time is formed by concatenating the relative coordinates of a preset number of first pixels, where the preset number is the quotient of the first data amount divided by the second data amount.

[0034] In the above method, after determining the relative coordinate set, the relative coordinate set is sent to the registration device, enabling the registration device to perform image registration based on the relative coordinates in the relative coordinate set. The relative coordinate set includes the relative coordinates of at least a portion of the pixels between the source image and the target image to be generated, where the target image represents the image obtained by registering the source image. Because the sending device sends a relative coordinate set, compared to sending an absolute coordinate set, the amount of data sent is reduced, thus reducing the time required to send registration parameters and improving image registration efficiency.

[0035] Please see Figure 2 , Figure 2This is a flowchart illustrating a second embodiment of the image registration method provided in this application. The method includes:

[0036] S210: Obtain the relative coordinate set sent by the sending device. The relative coordinate set includes the relative coordinates of at least some pixels between the source image and the target image to be generated. The target image represents the image obtained by registering the source image.

[0037] In one embodiment, the relative coordinate set is obtained by a sending device, which can be a CPU. Specifically, the sending device can obtain an absolute coordinate set based on a first reference image and a second reference image, and then use the absolute coordinate set to obtain a relative coordinate set. The first and second reference images are obtained from the same object under different acquisition conditions. Any pixel in the second reference image is taken as a third pixel, and the relative coordinates of each third pixel in the second reference image represent the relative coordinates between the corresponding first pixel in the target image and the corresponding second pixel in the source image. The relative coordinates of each third pixel are obtained by subtracting the absolute coordinates of each third pixel in the first reference image from their sequential coordinates in the second reference image. In other words, the relative coordinates between each first pixel in the target image and the corresponding second pixel in the source image are obtained by subtracting the absolute coordinates of each third pixel in the first reference image from their sequential coordinates in the second reference image. The relative coordinate set includes the relative coordinates of at least some pixels between the source image and the target image to be generated. That is, the relative coordinate set contains the relative coordinates of several third pixels. Among them, at least some pixels can be all pixels or some pixels. The sequential coordinates of each third pixel in the second reference image are obtained by device calibration. For example, the sequential coordinates of the third pixel in the first row and first column of the second reference image are labeled as (0,0), the sequential coordinates of the third pixel in the first row and second column are labeled as (0,1), and so on, to obtain the sequential coordinates of all third pixels in the second reference image.

[0038] In the above embodiments, the absolute coordinates of each third pixel in the second reference image in the first reference image can be determined using the spatial transformation parameters between the first and second reference images, thus obtaining a set of absolute coordinates. The relative coordinates of each third pixel are obtained by subtracting its absolute coordinates in the first reference image from its ordinal coordinates in the second reference image. The process of obtaining the spatial transformation parameters between the first and second reference images is described above and will not be repeated here.

[0039] In one specific implementation, the first reference image and the source image are acquired using the same acquisition conditions for the same object; that is, the first reference image and the source image are identical images. The target image represents an image acquired using the same acquisition conditions as the second reference image. Since the target image is obtained by registering the source image, the object in the target image should be consistent with the object in the source image. Therefore, after determining the third pixel in the second reference image, feature matching can be used to obtain the first pixel in the target image corresponding to the third pixel. For example, if the object in both the second reference image and the target image is a cat, and the determined third pixel is the pixel at the position of the cat's left eye, feature matching can be performed on the target image and the second reference image to find the position of the cat's left eye in the target image, thus obtaining the first pixel.

[0040] In another specific embodiment, the first reference image and the source image are obtained by acquiring different objects using the same acquisition conditions. That is, the first reference image is different from the source image, and therefore the objects in the second reference image and the target image are also different. There is a positional correspondence between the third pixel in the second reference image and the first pixel in the target image. For example, the position of the first third pixel in the second reference image corresponds to the position of the first first pixel in the target image. Therefore, the positional correspondence between the second reference image and the target image can be used to obtain the first pixel in the target image corresponding to the third pixel. Specifically, after determining the third pixel in the second reference image, the position of the corresponding third pixel in the target image is determined based on the positional correspondence, and then the first pixel is determined.

[0041] S220: Based on the relative coordinate set, determine the absolute coordinates of each first pixel in the target image in the source image.

[0042] The relative coordinate set includes the relative coordinates between several first pixels in the target image and corresponding second pixels in the source image. In one embodiment, the number of pixels in the several first pixels is less than the total number of pixels in the target image. Each first pixel is taken as a target pixel. From the relative coordinate set, at least one first associated pixel is found to correspond to the relative coordinates of the target pixel. The first associated pixel is a first pixel among the several first pixels that has a first positional relationship with the target pixel. For each first associated pixel, the absolute coordinates of the first associated pixel in the source image are obtained using the relative coordinates of the first associated pixel and the order coordinates of the first associated pixel in the target image. Interpolation is performed using the absolute coordinates of at least one first associated pixel to obtain the absolute coordinates of the target pixel. The at least one first associated pixel includes neighboring pixels among the several first pixels located in a preset direction of the target pixel. The preset direction includes at least one direction: left, right, up, and down.

[0043] In another embodiment, the number of pixels of a plurality of first pixels is equal to the total number of pixels of the target image. Each first pixel is taken as a target pixel. The relative coordinates of the corresponding second pixel are found from the relative coordinate set for the target pixel. Using the relative coordinates of the corresponding second pixel and the sequential coordinates of the second pixel, the absolute coordinates of the second pixel in the source image are obtained. At this time, the absolute coordinates of the second pixel in the source image are the absolute coordinates of the target pixel.

[0044] S230: Obtain the target image based on the absolute coordinates of each first pixel and the pixel values ​​of the source image.

[0045] In one embodiment, for each first pixel, at least one second associated pixel in the source image that has a second positional relationship with the absolute coordinates of the first pixel is found; interpolation is performed using the pixel values ​​of the at least one second associated pixel in the source image to obtain the pixel value of the first pixel in the target image. Specifically, after obtaining the absolute coordinates of each first pixel, four second associated pixels are found around the absolute coordinates of each first pixel, and interpolation is performed using the pixel values ​​of the four second associated pixels to obtain the pixel value of each first pixel in the target image. For example, if the absolute coordinates of the first pixel are (2.78, 35.21), then pixels with coordinates (2, 35), (2, 36), (3, 35), and (3, 36) can be selected as second associated pixels.

[0046] The method in this embodiment is executed by a registration device, which can be an FPGA, and the sending device can be a CPU.

[0047] The above method obtains the relative coordinate set sent by the sending device. The relative coordinate set includes the relative coordinates of at least some pixels between the source image and the target image to be generated. The target image represents the image obtained by registering the source image. Based on the relative coordinate set, the absolute coordinates of each first pixel in the target image in the source image are determined. Based on the absolute coordinates of each first pixel and the pixel value of the source image, the target image is obtained. Since the sending device sends a relative coordinate set, the amount of data sent is reduced compared to sending absolute coordinates, thus reducing the time for sending registration parameters. Furthermore, the registration device performs image registration based on the relative coordinates sent by the sending device, thereby reducing the time required for the entire image registration process and improving the image registration efficiency.

[0048] Please see Figure 3-6 , Figure 3 This is a flowchart illustrating the third embodiment of the image registration method provided in this application. Figure 4This is a simplified schematic diagram illustrating one implementation method for obtaining a relative coordinate set. Figure 5 This is a schematic diagram of a storage method for relative coordinate sets in memory. Figure 6 This is a schematic diagram illustrating the process of interpolating the absolute coordinates of a target pixel using the absolute coordinates of at least one first associated pixel. The method includes:

[0049] S310: Obtain the relative coordinate set sent by the sending device. The relative coordinate set includes the relative coordinates of at least some pixels between the source image and the target image to be generated. The target image represents the image obtained by registering the source image.

[0050] In one embodiment, the relative coordinate set includes the relative coordinates between a plurality of first pixels in the target image and corresponding second pixels in the source image. The number of pixels in the plurality of first pixels is less than the total number of pixels in the target image. The plurality of first pixels are obtained by downsampling the target image by a preset multiple. The preset multiple is K, where K is an integer and can be set as needed, for example, K is 5, 16, 32, etc. Figure 4 As shown, the source image has a resolution of 11*11. The source image is downsampled by a factor of 5, and one pixel is selected from every five pixels to obtain a sampled image with a resolution of 3*3. Selecting one pixel from every five pixels means selecting the first pixel among the five pixels, which is used as a pixel in the sampled image (i.e., one of the first pixels in the target image). It is understood that in other embodiments, other pixels among the five pixels can also be selected as pixels in the sampled image.

[0051] In one specific implementation, the resolution of the target image is 4096*2160. The target image is downsampled by 32 times, and the resulting relative coordinate size is (4096 / 32)*(2160 / 32 rounded up) = 8704. That is, the relative coordinate set contains 8704 sets of relative coordinates. Each set of relative coordinates contains an X-axis relative coordinate and a Y-axis relative coordinate. At this time, the relative coordinate set needs to be sent to the registration device 8704 times.

[0052] In another specific embodiment, the transmission bit width between the sending device and the registration device is 32 bits, and the second data amount of the relative coordinates of each first pixel is 16 bits. Then, two adjacent sets of relative coordinates can be concatenated and sent to the registration device. At this time, the number of sending times can be reduced to 4352.

[0053] S320: Store the relative coordinate set in at least one memory according to a preset storage strategy.

[0054] In one embodiment, all relative coordinates can be stored in a memory according to the order in which the relative coordinates in the relative coordinate set are sent. For example, when the relative coordinates in the relative coordinate set are sent 8704 times as described above, all relative coordinates can be stored in a memory.

[0055] like Figure 5 As shown, in another embodiment, a plurality of first pixels can be divided into first odd-numbered pixels and first even-numbered pixels. The relative coordinates corresponding to the first odd-numbered pixels and the relative coordinates corresponding to the first even-numbered pixels are then separated. For example, after concatenating two adjacent sets of relative coordinates, when sending them to the registration device, one set of relative coordinates represents the relative coordinates corresponding to the first odd-numbered pixels, and the other set represents the relative coordinates corresponding to the first even-numbered pixels. The relative coordinates corresponding to the first odd-numbered pixels are stored in the first memory sequentially according to the sending order. For example... Figure 4 In the image, ds_mapy[1] and ds_mapx[1] are used. ds_mapy[1] is the Y-axis relative coordinate of the first odd-numbered pixel in the downsampled image, and ds_mapx[1] is the X-axis relative coordinate of the first odd-numbered pixel in the downsampled image. The relative coordinates corresponding to the first even-numbered pixel are stored in the second memory, for example... Figure 4 In the sampled image, ds_mapy[2] and ds_mapx[2] are used. ds_mapy[1] is the Y-axis relative coordinate of the second even-numbered pixel in the sampled image after downsampling, and ds_mapx[1] is the X-axis relative coordinate of the second even-numbered pixel in the sampled image after downsampling.

[0056] S330: Take each first pixel of the target image as the target pixel.

[0057] S340: Determine the storage address of at least one first associated pixel in memory based on the sequential coordinates of the target pixel.

[0058] In one embodiment, the row and column numbers of a first associated pixel in the downsampled target image can be obtained by dividing each axis coordinate in the sequential coordinate system by a preset multiple and rounding down. The storage address of the first associated pixel is then obtained using its row and column numbers. Finally, the storage addresses of other first associated pixels are obtained using their storage addresses and their positional relationships with other first associated pixels. Here, the preset multiple represents the downsampling factor applied to the source image. The number of first associated points can be four, which are four points in a preset direction corresponding to the target pixel's position in the source image. The preset direction is left, right, up, and down.

[0059] In one specific embodiment, the sequential coordinates of the target pixel are (5, 34), the preset multiplier is 32, the resolution of the target image is 4096*2160, and the number of first pixels in the horizontal direction in the downsampled image should be 128. Dividing the left side of each axis of the sequential coordinates of the target pixel by 32 (5 / 32 rounded down = 0) and 34 divided by 32 rounded down = 1, we obtain a first associated pixel with a row number of 1 and a column number of 0 in the downsampled target image. Then, the row number 1*128 + column number 0 of the first associated pixel is used to obtain the storage address 128 of the first associated pixel. Other first associated pixels can be the right side, the bottom side, and the lower right side of the first associated pixel. The storage addresses of the other associated pixels are, in order, the storage address of the first associated pixel + 1 = 129, the storage address of the first associated pixel + 128 = 256, and the storage address of the first associated pixel + 128 + 1 = 257.

[0060] S350: Using the storage address, read the relative coordinates of at least one first associated pixel from the memory.

[0061] In one embodiment, the relative coordinates corresponding to several first pixels are stored separately in two memories. The first memory stores the relative coordinates corresponding to the first odd-numbered pixels, and the second memory stores the relative coordinates corresponding to the first even-numbered pixels. In the above embodiment, the read address of the first associated pixel is 128, and the read addresses of the other first associated pixels are 129, 256, and 257. For the first associated pixel with read address 128, its corresponding absolute coordinates are the 64th data in the second memory (the 64th data is ds_mapx

[128] , ds_mapy

[128] ); for the first associated pixel with read address 129, its corresponding absolute coordinates are the 65th data in the first memory (the 65th data is ds_mapx

[129] , ds_mapy

[129] ); for the first associated pixel with read address 256, its corresponding absolute coordinates are the 128th data in the second memory (the 128th data should be ds_mapx

[256] , ds_mapy

[256] ); for the first associated pixel with read address 257, its corresponding absolute coordinates are the 129th data in the first memory (the 129th data should be ds_mapx

[257] , ds_mapy

[257] ). Among them, ds_mapx

[128] and ds_mapy

[128] can be (+0.8, -1.2); ds_mapx

[129] and ds_mapy

[129] can be (+0.7, -1.2); ds_mapx

[256] and ds_mapy

[256] can be (+0.8, -1.3); ds_mapx

[257] and ds_mapy

[257] can be (+0.7, -1.3).

[0062] S360: Using the relative coordinates of at least one first associated pixel, obtain the absolute coordinates of the target pixel in the source image.

[0063] In one embodiment, the sequential coordinates of each first associated pixel in the target image are obtained. The sequential coordinates and relative coordinates of each first associated pixel in the target image are summed to obtain the absolute coordinates of each first associated pixel in the source image. Interpolation is then performed using the absolute coordinates of at least one first associated pixel to obtain the absolute coordinates of the target pixel. The sequential coordinates of the first associated pixels in the target image are obtained by device calibration. In a specific embodiment, after obtaining the absolute coordinates of each first associated pixel in the source image, bilinear interpolation can be used to obtain the absolute coordinates of the target pixel in the source image. Specifically, the absolute coordinates of the target pixel in the source image can be calculated using the interpolation formula f(x,y)=f(x1,y1)(1-u)(1-v)+f(x2,y1)u(1-v)+f(x1,y2)(1-u)v+f(x2,y2)u. Where u is the contribution of the first associated pixel in the Y direction to the absolute coordinates of the target pixel in the source image, and v is the contribution of the first associated pixel in the X direction to the absolute coordinates of the target pixel in the source image. The closer the first associated pixel is to the absolute coordinates of the target pixel in the source image, the smaller the values ​​of u and v are.

[0064] like Figure 6 As shown in the above embodiment, the sequential coordinates of the four first associated pixels are Q12(0,32), Q22(32,32), Q11(0,64), and Q21(32,64). The calculated absolute coordinates of the four first associated pixels in the source image are (0.8,30.8), (32.7,30.8), (0.8,62.7), and (32.7,62.7). If the sequential coordinates of the target pixel are (5,34), then u in the above formula is 34-32=2, and v is 5-0=5. Substituting the absolute coordinates of the four first associated pixels and the values ​​of u and v into the above interpolation formula, the absolute coordinates of the target pixel in the source image are obtained as (2.79375,35.784375), i.e. Figure 6 The location of point P in the middle.

[0065] S370: Obtain the target image based on the absolute coordinates of each target pixel and the pixel values ​​of the source image.

[0066] In one embodiment, for each target pixel, at least one second associated pixel is found in the source image that has a second positional relationship with the absolute coordinates of the target pixel. The second associated pixel includes neighboring pixels in the source image located in a preset direction relative to the corresponding pixel of the target pixel. The preset direction includes at least one direction: left, right, up, or down. Interpolation is performed using the pixel values ​​of at least one second associated pixel in the source image to obtain the pixel value of the first pixel in the target image.

[0067] This implementation is performed by a registration device. After receiving the relative coordinate set from the sending device, the registration device stores the relative coordinate set in at least one memory according to a preset storage strategy. After obtaining the sequential coordinates of the target pixels in the target image, the device retrieves the relative coordinates of at least one first associated pixel from the memory based on the sequential coordinates of the target pixels. Using the relative coordinates of each first associated pixel and its sequential coordinates in the target image, the absolute coordinates of each first associated pixel in the source image are obtained. Interpolation is performed using the absolute coordinates of each first associated pixel to obtain the absolute coordinates of the target pixel. Based on the absolute coordinates of the target pixel, four second associated pixels surrounding the absolute coordinates of the first pixel are found in the source image. Interpolation is performed using the pixel values ​​of the four second associated pixels to obtain the pixel value of the target pixel in the target image. The above processing is performed on all target pixels contained in the target image to obtain the pixel values ​​of the target pixels in the target image.

[0068] Please see Figure 7 , Figure 7 This is a schematic diagram of the framework structure of one embodiment of the electronic device provided in this application.

[0069] The electronic device is designated as the first electronic device 70. The first electronic device 70 includes a first memory 71 and a sending device 72, which are coupled to each other. The first memory 71 stores program instructions, and the sending device 72 executes the program instructions stored in the first memory 71 to implement the steps of the first embodiment of the image registration method described above. In a specific implementation scenario, the first electronic device 70 may include, but is not limited to, a microcomputer or a server. Furthermore, the first electronic device 70 may also include mobile devices such as laptops and tablets, without limitation.

[0070] In one embodiment, the sending device 72 may also be referred to as a CPU (Central Processing Unit). In other embodiments, the sending device 72 may be an integrated circuit chip with signal processing capabilities; the sending device 72 may also be a general-purpose processor, a digital signal processor (DSP), or any conventional processor.

[0071] Please see Figure 8 , Figure 8 This is a schematic diagram of another embodiment of the electronic device provided in this application.

[0072] This electronic device is designated as the second electronic device 80. The second electronic device 80 includes a second memory 81 and a registration device 82. The second memory 81 stores program instructions. The registration device 82 is used to execute the steps in the second and third embodiments of the image registration method described above. In a specific implementation scenario, the second electronic device 80 may include, but is not limited to, a microcomputer or a server. In addition, the second electronic device 80 may also include mobile devices such as laptops and tablets, without limitation.

[0073] In one embodiment, the registration device 82 may be a field-programmable gate array (FPGA). In other embodiments, the registration device 82 may also be an application-specific integrated circuit (ASIC) or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0074] Please see Figure 9 , Figure 9 This is a schematic diagram of a framework of one embodiment of the computer-readable storage medium provided in this application.

[0075] The computer-readable storage medium 90 stores program instructions 91, which, when executed by a processor, are used to implement the steps in any of the above method embodiments.

[0076] The computer-readable storage medium 90 can specifically be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or a medium that can store computer programs. Alternatively, it can be a server that stores the computer program, which can send the stored computer program to other devices for execution or can also run the stored computer program itself.

[0077] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0078] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0079] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0080] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0081] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0083] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An image registration method characterized by, The method comprises: determining a relative coordinate set; the relative coordinate set comprises relative coordinates of at least part of pixel points between a source image and a target image to be generated, and the target image represents an image obtained by registering the source image; downloading the relative coordinate set to a registration device, so that the registration device performs image registration based on the relative coordinates in the relative coordinate set; wherein the determination of the relative coordinate set comprises: determining absolute coordinates of each third pixel point in the second reference image in the first reference image by using spatial transformation parameters between the first reference image and the second reference image; wherein the first reference image and the second reference image are obtained by capturing the same object under different conditions; the first reference image and the source image are different images and are captured under the same condition, or the first reference image and the source image are the same image; and the target image represents an image obtained by capturing under the same condition as the second reference image; determining relative coordinates of at least one third pixel point based on the absolute coordinates of the at least one third pixel point in the first reference image and the sequential coordinates of the at least one third pixel point in the second reference image, wherein the relative coordinates of each third pixel point represent the relative coordinates between first pixel points at corresponding positions in the target image and second pixel points corresponding to the source image.

2. The method of claim 1, wherein, The relative coordinate set comprises relative coordinates between a plurality of first pixel points of the target image and second pixel points corresponding to the source image, and the number of the plurality of first pixel points is less than the total number of pixel points of the target image.

3. The method of claim 1, wherein, The determination of the absolute coordinates of each third pixel point in the second reference image in the first reference image by using the spatial transformation parameters between the first reference image and the second reference image comprises: obtaining device-related parameters of a device for capturing the first reference image and the second reference image; obtaining the spatial transformation parameters based on the device-related parameters; performing spatial transformation on the second reference image by using the spatial transformation parameters to obtain the absolute coordinates of each third pixel point in the second reference image in the first reference image; and / or, the determination of the relative coordinates of at least one third pixel point based on the absolute coordinates of the at least one third pixel point in the first reference image and the sequential coordinates of the at least one third pixel point in the second reference image comprises: for each third pixel point, obtaining the relative coordinates of the third pixel point by using the deviation between the absolute coordinates and the sequential coordinates of the third pixel point; down-sampling the relative coordinates of each third pixel point in the second reference image by a predetermined multiple to obtain the relative coordinates of a plurality of third pixel points to form the relative coordinate set.

4. The method of claim 1, wherein, The method is performed by a downloading device; The processing capability of the issuing device is higher than that of the registration device; and / or the transmission bit width between the issuing device and the registration device is a first data amount, the relative coordinates of each pixel point are a second data amount, the coordinate data issued by the issuing device each time is formed by splicing the relative coordinates of a preset number of first pixel points, and the preset number is the quotient of the first data amount divided by the second data amount.

5. An image registration method characterized by, The method comprises: obtaining a set of relative coordinates issued by an issuing device, the set of relative coordinates comprising relative coordinates of at least part of pixel points between a source image and a target image to be generated, the target image representing an image obtained by registering the source image; determining absolute coordinates of each first pixel point of the target image in the source image based on the set of relative coordinates; obtaining the target image based on the absolute coordinates of each first pixel point and pixel values of the source image; The set of relative coordinates comprises relative coordinates between a plurality of first pixel points of the target image and corresponding second pixel points of the source image, the relative coordinates of each first pixel point are obtained by subtracting sequential coordinates of each third pixel point in a second reference image from absolute coordinates of each third pixel point in a first reference image at corresponding positions, the third pixel points represent pixel points in the second reference image, and the absolute coordinates of each third pixel point are determined by using spatial transformation parameters between the first reference image and the second reference image; the first reference image and the second reference image are obtained by capturing the same object under different conditions; the first reference image and the source image are different images and are captured under the same condition, or the first reference image and the source image are the same image; and the target image represents an image captured under the same condition as the second reference image.

6. The method of claim 5, wherein, The set of relative coordinates comprises relative coordinates between a plurality of first pixel points of the target image and corresponding second pixel points of the source image, and the number of pixel points of the plurality of first pixel points is less than the total number of pixel points of the target image; The method comprises: respectively taking each first pixel point of the target image as a target pixel point; finding, for the target pixel point, the relative coordinates corresponding to at least one first associated pixel point from the set of relative coordinates, the first associated pixel point being a first pixel point in the plurality of first pixel points that has a first positional relationship with the target pixel point; obtaining absolute coordinates of the target pixel point in the source image by using the relative coordinates of the at least one first associated pixel point.

7. The method of claim 6, wherein, The at least one first associated pixel point comprises adjacent pixel points in the plurality of first pixel points in a preset direction of the target pixel point, and the preset direction comprises at least one of left, right, up and down.

8. The method of claim 6, wherein, Before determining the absolute coordinates corresponding to each first pixel point of the target image in the source image based on the relative coordinate set, the method further comprises: storing the relative coordinate set in at least one memory according to a preset storage strategy; the method further comprises: determining the storage address of the at least one first associated pixel point in the memory according to the sequential coordinates of the target pixel point; reading the relative coordinates of the at least one first associated pixel point from the memory by using the storage address.

9. The method of claim 8, wherein, the first pixel points are obtained by down-sampling the target image according to a preset multiple; the method further comprises: dividing each axis coordinate in the sequential coordinates by the preset multiple and taking the integer part to obtain the row sequence number and the column sequence number of the first associated pixel point in the target image subjected to the down-sampling; obtaining the storage address of the first associated pixel point by using the row sequence number and the column sequence number of the first associated pixel point; and obtaining the storage addresses of other first associated pixel points by using the storage address of the first associated pixel point and the positional relationship between the first associated pixel point and other first associated pixel points.

10. The method of claim 6, wherein, the method further comprises: for each first associated pixel point, obtaining the absolute coordinates corresponding to the first associated pixel point in the source image based on the relative coordinates of the first associated pixel point and the sequential coordinates of the first associated pixel point in the target image; performing interpolation processing on the absolute coordinates of the at least one first associated pixel point to obtain the absolute coordinates of the target pixel point.

11. The method of claim 5, wherein, the method further comprises: for each first pixel point, finding at least one second associated pixel point in the source image which has a second positional relationship with the absolute coordinates of the first pixel point; performing interpolation processing on the pixel values of the at least one second associated pixel point in the source image to obtain the pixel value of the first pixel point in the target image.

12. An electronic device, comprising: The electronic device comprises a first memory and a delivery device, the first memory stores program instructions, and the delivery device is configured to execute the program instructions to implement the method in any one of claims 1 to 4.

13. An electronic device, comprising: The electronic device comprises a second memory and a registration device; the second memory stores program instructions, and the registration device is configured to execute the program instructions to implement the method in any one of claims 5 to 11.

14. A computer-readable storage medium, characterized in that, The program instructions stored in the memory can be run by the processor, and the program instructions are used to implement the method in any one of claims 1 to 11. The program instructions stored in the memory can be run by the processor, and the program instructions are used to implement the method in any one of claims 1 to 11.

Images