Image processing method, image processing device, and computer-readable storage medium

Abstract

Disclosed are an image processing method, an image processing device, and a storage medium. The method includes: extracting overlapping regions from adjacent N-dimensional image pairs in multiple N-dimensional images of various parts of an object; segmenting the overlapping regions of the adjacent N-dimensional image pairs in N directions to obtain two sets of N-dimensional segmented images in each direction; projecting the two sets of N-dimensional segmented images in corresponding directions to obtain two sets of two-dimensional segmented images; performing pairwise matching between the two sets of two-dimensional segmented images to obtain an offset with the highest confidence in the corresponding direction; if the offset is greater than a predetermined threshold, repeating the above steps based on the offset until the offset is less than the predetermined threshold; repeating the above steps for other adjacent N-dimensional image pairs in the multiple N-dimensional images; recalculating the size of the multiple N-dimensional images after splicing based on the offsets of all N-dimensional image pairs, and fusing the overlapping regions between the N-dimensional image pairs.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to image stitching technology. Background Art

[0002] Obtaining high-resolution microscopic images of large samples has a wide range of applications in the biomedical field. When the sample size is relatively large, there are two problems. The first problem is that the microscope's field of view in the XY direction cannot capture the entire sample. The second problem is that because the sample is relatively thick, the laser cannot penetrate the object in the Z direction. The solution to the first problem is to translate the sample stage of the light sheet microscope in the XYZ directions, thereby performing multi-point scanning or tile scanning at multiple locations. The solution to the second problem is to rotate the sample stage to perform multi-angle scanning. Finally, the multi-point or tile scan and multi-angle scan images are stitched and fused to form a sample image with a larger field of view.

[0003] However, due to the complexity of the light-sheet microscope's optical imaging system and the sample itself, images collected at different locations can have discrepancies in the overlapping areas, resulting in unsatisfactory stitching. Furthermore, there is a lack of automatic stitching technology for rotated images. Summary of the Invention

[0004] A brief overview of the present disclosure is provided below to provide a basic understanding of certain aspects of the present disclosure. It should be understood that this overview is not an exhaustive overview of the present disclosure. It is not intended to identify key or important portions of the present disclosure, nor is it intended to limit the scope of the present disclosure. Its purpose is simply to present certain concepts in a simplified form as a prelude to a more detailed description that will be discussed later.

[0005] According to one aspect of the present invention, an image processing method is provided, comprising: extracting overlapping regions for N-dimensional image pairs located adjacent to each other on a plurality of N-dimensional images of parts of a photographed object, where N is an integer greater than or equal to three; segmenting the overlapping regions of the adjacent N-dimensional image pairs in N directions to obtain two sets of N-dimensional segmented images for each direction; projecting the two sets of N-dimensional segmented images in their corresponding directions to obtain two sets of two-dimensional segmented images; performing pairwise matching of the two-dimensional segmented images between the two sets of two-dimensional segmented images to obtain an offset with the highest confidence between the two sets of two-dimensional segmented images in their corresponding directions; repeating the above steps based on the offset if the offset is greater than a predetermined threshold until the offset is less than the predetermined threshold; iteratively performing the above steps for other adjacent N-dimensional image pairs in the plurality of N-dimensional images; recalculating the image size after stitching the plurality of N-dimensional images together based on the obtained offsets of all N-dimensional image pairs to perform image stitching, and fusing the overlapping regions between all N-dimensional image pairs.

[0006] Preferably, the image processing method according to the present invention further comprises adjusting the multiple N-dimensional images to the same viewing angle before intercepting the overlapping area.

[0007] Preferably, a fusion method based on Gaussian filtering is used to fuse the overlapping areas.

[0008] Preferably, dividing the overlapping area of ​​the adjacent N-dimensional image pair into blocks in N directions respectively includes: determining the number of blocks of the overlapping area in each direction according to the ratio of the length of the overlapping area in each direction to the difference between the block thickness and the overlap.

[0009] Preferably, the projection is based on a maximum intensity projection method.

[0010] Preferably, performing pairwise matching of two-dimensional block images between the two groups of two-dimensional block images to obtain offsets with the highest confidence in the corresponding directions of the two groups of two-dimensional block images includes: performing pairwise matching of two-dimensional block images in one of the two groups of two-dimensional block images with two-dimensional block images in the other group to obtain offsets in other N-1 directions other than the corresponding directions of the two groups of two-dimensional block images and confidence levels under the offsets; and using the offset with the highest confidence as the offset along the corresponding direction.

[0011] Preferably, the matching is based on a normalized cross-correlation algorithm.

[0012] Preferably, recalculating the image size after stitching the multiple N-dimensional images together includes: calculating the maximum absolute position of each N-dimensional image in N directions based on the corresponding offsets obtained; and determining the image size after stitching the multiple N-dimensional images together based on all the maximum absolute positions obtained.

[0013] Preferably, the predetermined threshold is less than or equal to 5 pixels.

[0014] According to another aspect of the present invention, there is provided an image processing device, comprising: a cropping device configured to crop overlapping areas of N-dimensional image pairs that are adjacent in position on a photographed object from a plurality of N-dimensional images of parts of the object, wherein N is an integer greater than or equal to three; a blocking device configured to block the overlapping areas of the adjacent N-dimensional image pairs in N directions, respectively, to obtain two groups of N-dimensional blocked images for each direction; a projection device configured to project the two groups of N-dimensional blocked images in their corresponding directions to obtain two groups of two-dimensional blocked images; a matching device configured to perform pairwise matching of the two-dimensional blocked images between the two groups of two-dimensional blocked images, to obtain an offset with the highest confidence between the two groups of two-dimensional blocked images in their corresponding directions; and a determination device configured to determine whether the offset is greater than a predetermined threshold, and if the offset is greater than the predetermined threshold, to re-crop the overlapping area and re-determine the offset with the highest confidence based on the offset, until the offset is less than the predetermined threshold. The image processing device is configured to: determine the offsets of other adjacent N-dimensional image pairs in the multiple N-dimensional images, and recalculate the image size after splicing the multiple N-dimensional images together based on the obtained offsets of all N-dimensional image pairs to perform image stitching, and fuse the overlapping areas between all N-dimensional image pairs.

[0015] According to other aspects of the present invention, corresponding computer program code, computer-readable storage medium and computer program product are also provided.

[0016] The image processing method and device of the present invention can reduce computational complexity, improve stitching effects, and also support automatic stitching of rotated images.

[0017] These and other advantages of the present invention will become more apparent from the following detailed description of preferred embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to further illustrate the above and other advantages and features of the present disclosure, the following is a further detailed description of specific embodiments of the present disclosure in conjunction with the accompanying drawings. The drawings, together with the detailed description below, are included in this specification and form a part of this specification. Elements with the same function and structure are represented by the same reference numerals. It should be understood that these drawings only depict typical examples of the present disclosure and should not be regarded as limiting the scope of the present disclosure. In the drawings:

[0019] Figure 1 is a flowchart of an image processing method according to one embodiment of the present invention;

[0020] Figure 2 is a flowchart of recalculating the size of a stitched image according to one embodiment of the present invention;

[0021] Figure 3A The figure schematically shows the MIP projection of the three-dimensional image divided into blocks on the Z axis;

[0022] Figure 3B Schematically illustrating pairwise matching between two sets of two-dimensional images;

[0023] Figure 4 Schematically shows the offset of the template image relative to the image to be matched;

[0024] Figure 5 The comparison of the splicing effect between the method according to the present invention and the prior art method is shown;

[0025] Figure 6 is a block diagram of an image processing apparatus according to one embodiment of the present invention;

[0026] Figure 7 is a block diagram of an exemplary structure of a general-purpose personal computer in which methods and / or apparatus according to embodiments of the present invention may be implemented. DETAILED DESCRIPTION

[0027] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings. For the sake of clarity and conciseness, not all features of an actual implementation are described in this specification. However, it should be understood that in the process of developing any such actual implementation, many implementation-specific decisions must be made in order to achieve the developer's specific goals, such as meeting those constraints related to the system and business, and these constraints may vary from implementation to implementation. In addition, it should be understood that although the development work may be very complex and time-consuming, it is a routine task for those skilled in the art who benefit from the content of this disclosure.

[0028] It is also necessary to explain here that, in order to avoid obscuring the present disclosure due to unnecessary details, the accompanying drawings only show the device structure and / or processing steps that are closely related to the solution according to the present disclosure, while other details that are not closely related to the present disclosure are omitted.

[0029] Currently, the stitching module, a plug-in in ImageJ, is commonly used for image stitching in biomedical image processing. This plug-in aligns images based on PCM (phase correlation method), calculates offsets, and then fuses the images. The iStitch module, a commonly used image stitching plug-in in Vaa3D, aligns images based on PCM and NCC (normalized cross correlation), calculates offsets, and then fuses the images. These methods are less effective for registering images with differences in overlapping areas, resulting in suboptimal overall stitching results. Furthermore, they do not support automatic stitching of rotated images.

[0030] In view of this, the present invention proposes a fine-grained image stitching method for multi-dimensional images acquired by translation and rotation scanning of a light sheet microscope. Figure 1 An image processing method 100 according to one embodiment of the present invention is described.

[0031] First, in step 101, overlapping regions are extracted from pairs of N-dimensional images of portions of a photographed object that are adjacent to each other on the object, where N is an integer greater than or equal to three. Specifically, in this embodiment, the N-dimensional images are, for example, three-dimensional images, and the pair of three-dimensional images that are adjacent to each other on the object are images that can be stitched together at the overlapping region to form an image containing a portion of the object.

[0032] It should be understood that the three-dimensional image is only an example and is not intended to limit the present invention. The method according to this embodiment is also applicable to four-dimensional or multi-dimensional images.

[0033] Specifically, in this embodiment, the overlapping area of ​​the 3D image pair is calculated according to the translation size and image size of the pair, and then the overlapping areas of the two 3D images are respectively intercepted to obtain two overlapping area 3D images.

[0034] Preferably, the image processing method 100 according to this embodiment further includes a step 101' of adjusting the multiple N-dimensional images to the same viewing angle before intercepting the overlapping region. Specifically, in this embodiment, if it is determined that the multiple three-dimensional images have been rotated, they are rotated back to zero degrees around the rotation axis so that all N three-dimensional images return to the same viewing angle.

[0035] It should be understood that how to determine whether a 3D image is rotated and how to rotate the 3D image back to zero degrees is known in the prior art and will therefore not be described in detail herein.

[0036] Next, in step 102, the overlapping regions of the adjacent N-dimensional image pairs are divided into blocks in N directions, thereby obtaining two sets of N-dimensional block images for each direction. Specifically, in this embodiment, the two overlapping 3D images obtained in step 101 are divided into blocks in the X, Y, and Z directions, thereby obtaining two sets of 3D images for each direction.

[0037] Preferably, the function The overlapping area three-dimensional image is divided into blocks in each direction, where c represents the thickness of each block in the corresponding direction, p represents the length of two adjacent blocks overlapping with each other in the corresponding direction, and m represents the overall length of the overlapping area three-dimensional image in the corresponding direction.

[0038] Preferably, the value of c may be, for example, one tenth of the length of the three-dimensional image in the X, Y, and Z directions, respectively, and the value of p may be 20% of c.

[0039] It should be understood that the above block division method is only an example and is not intended to limit the present invention. Other suitable block division methods can also be used.

[0040] It should also be understood that the values ​​of c and p are not limited to the above-mentioned method, but can also be appropriately set according to the accuracy requirements of image stitching.

[0041] Next, in step 103, the two sets of N-dimensional block images in each direction are projected in their corresponding directions to obtain two sets of two-dimensional block images. Specifically, in this embodiment, the two sets of three-dimensional images obtained in step 102 are projected, for example, using MIP (maximum intensity projection) to obtain two sets of two-dimensional images in each direction.

[0042] It should be understood that MIP projection is only an example and is not intended to limit the present invention. Other suitable projection methods may also be used.

[0043] Figure 3A The figure schematically shows the segmentation and MIP projection of the overlapping area three-dimensional image in the Z-axis direction. Figure 3A As shown, after MIP projection, we get A 2D image with a slice thickness of 1.

[0044] It should be noted that the block division step 102 and the projection step 103 reduce the amount of data to times, which dramatically reduces memory overhead and computational workload, and makes it possible to splice 1000GB of data on a common workstation with, for example, 128GB of memory.

[0045] Next, in step 104, pairwise matching of the two-dimensional block images is performed between the two groups of two-dimensional block images in each direction to obtain the offsets with the highest confidence in the two groups of two-dimensional block images in their corresponding directions.

[0046] Specifically, if Figure 3B As shown, one of the two sets of two-dimensional images in each direction obtained in step 103 is used as a template group, and the other is used as a group of images to be matched. Two-dimensional images. Each two-dimensional image in the template group is matched with each two-dimensional image in the image group to be matched, for example, based on the NCC algorithm, so as to obtain the offset of the two groups of two-dimensional images in one direction in the other two directions and the confidence under the offset. There are a total of Group matching results. The offset of the pair of two-dimensional images with the highest confidence in the group is used as a pair of offsets for the section along that direction. Thus, there are three pairs of offsets for the X, Y and Z directions.

[0047] Figure 4 The figure schematically shows the offset of the template image relative to the image to be matched, wherein the left front cube in the outermost cube is the image to be matched, and the right rear cube is the template image. Figure 4 In the example shown, the offset in the X direction is greater than zero, the offset in the Y direction is less than zero, and the offset in the Z direction is greater than zero.

[0048] It should be understood that how to calculate the offset and confidence based on the NCC algorithm is known in the prior art and will therefore not be described in detail here.

[0049] It should also be understood that the NCC algorithm is only an example and is not intended to limit the present invention. Other suitable algorithms may also be used to calculate the offset and confidence.

[0050] Next, in step 105, it is determined whether the three pairs of offsets obtained are greater than a predetermined threshold. If so, steps 101 to 104 are repeated based on the calculated offsets until the final offset is less than or equal to the predetermined threshold. In this embodiment, the predetermined threshold is, for example, less than or equal to 5 pixels.

[0051] It should be understood that the present invention is not limited thereto, and the predetermined threshold may be set as needed.

[0052] It should also be understood that re-performing steps 101 to 104 according to the calculated offset means re-cutting the overlapping area based on the calculated offset.

[0053] If it is determined in step 105 that the offset is less than or equal to the predetermined threshold, the process proceeds to step 106. In step 106, steps 101 to 105 are iteratively performed for other adjacent N-dimensional image pairs in the plurality of N-dimensional images. Specifically, in this embodiment, steps 101 to 105 are performed for all other adjacent 3D images on the object to determine the offsets of all adjacent 3D images.

[0054] Finally, in step 107 , the image size after stitching together the multiple N-dimensional images is recalculated based on the obtained offsets of all N-dimensional image pairs to perform image stitching, and the overlapping areas between all N-dimensional image pairs are fused.

[0055] The following combination Figure 2 One embodiment of recalculating the size of a stitched image is described.

[0056] First, in step 1071, the maximum absolute position of each N-dimensional image in the N directions is calculated based on the corresponding offsets obtained. Specifically, in this embodiment, the maximum absolute positions of all adjacent three-dimensional images in the X, Y, and Z directions are calculated based on the offsets of these three-dimensional images in the X, Y, and Z directions obtained in step 106. For example, based on the order of adjacent images, the position of the first image is used as the absolute zero position, and all consecutively adjacent images are traversed to calculate the absolute positions of these consecutively adjacent images based on the relative offsets.

[0057] Next, in step 1072, the size of the image resulting from stitching together the multiple N-dimensional images is determined based on all the obtained maximum absolute positions. Specifically, in this embodiment, the size of the three-dimensional image showing the complete object after stitching together the three-dimensional images is determined based on the sum of the maximum absolute positions of all the three-dimensional images in the X, Y, and Z directions.

[0058] Finally, in step 1072, the overlapping regions between all N-dimensional image pairs are fused. Specifically, in this embodiment, a fusion method based on Gaussian filtering can be used to fuse the overlapping regions. It should be understood that the present invention is not limited to this, and any suitable fusion method can be used, such as, but not limited to, linear fusion, minimum fusion, maximum fusion, mean fusion, etc. In the case of image fusion involving rotation, a fusion method based on Gaussian filtering is preferably used.

[0059] The method according to the present invention reduces computational complexity, improves stitching quality, and supports automatic stitching of rotated images. Table 1 below illustrates the experimental comparison of the method according to the present invention with the Imagej method and the Vaa3d method.

[0060]

[0061] Table 1

[0062] In Table 1, Imagej* represents methods in the stitching module of the Imagej plugin, and Vaa3d* represents methods in the iStitch module of the Vaa3d plugin. Each image data item is represented by X*Y*Z*C*P, which represents the image's width*height*depth*color channels*positions, and each pixel is 16 bits. In Table 1, "fli1:2.92GB(1024*1024*150*1*10)" indicates that the data is 2.92GB, has a width of 1024 pixels, a height of 1024 pixels, a depth of 150 layers, 1 color channel, and 10 positions.

[0063] Figure 5 The figure shows the comparison of the stitching effect between the method according to the present invention, the Imagej method and the Vaa3d method. Figure 5 In the figure, (a-1) to (a-4) are the MIPs in the Z direction of the images collected by the light sheet microscope, (b) is the MIP in the Z direction of the resulting image stitched by Imagej*, (c) is the MIP in the Z direction of the resulting image stitched by Vaa3d*, and (d) is the MIP in the Z direction of the resulting image stitched by the method of the present invention.

[0064] From Table 1 and Figure 5 It can be seen that the method of the present invention achieves better splicing effect than the prior art method.

[0065] The methods discussed above can be implemented entirely by a computer-executable program, or partially or completely by hardware and / or firmware. When implemented in hardware and / or firmware, or when a computer-executable program is loaded into a hardware device capable of running the program, the image processing device described below is implemented. Below, an overview of these devices is given without repeating some of the details discussed above. However, it should be noted that although these devices can perform the methods described above, the methods do not necessarily employ or are not necessarily performed by the components of the described devices.

[0066] Figure 6An image processing device 600 according to one embodiment is shown, which includes a capture device 601, a blocking device 602, a projection device 603, a matching device 604, a determination device 605, and a stitching device 606. The capture device 601 is used to capture overlapping areas of N-dimensional image pairs that are adjacent to each other on the object in multiple N-dimensional images of various parts of the photographed object, where N is an integer greater than or equal to three. The blocking device 602 is used to block the overlapping areas of the adjacent N-dimensional image pairs in N directions to obtain two groups of N-dimensional blocked images for each direction. The projection device 603 is used to project the two groups of N-dimensional blocked images in their corresponding directions to obtain two groups of two-dimensional blocked images. The matching device 604 is used to perform pairwise matching of the two-dimensional blocked images between the two groups of two-dimensional blocked images to obtain the offset with the highest confidence level of the two groups of two-dimensional blocked images in their corresponding directions. The determining device 605 is configured to determine whether the offset is greater than a predetermined threshold value, and if so, to re-cut the overlapping region based on the offset and re-determine the offset with the highest confidence until the offset is less than the predetermined threshold value. The stitching device 606 is configured to recalculate the image size of the stitched-together N-dimensional images based on the offsets of all N-dimensional image pairs obtained by the determining device 605, thereby performing image stitching, and to fuse the overlapping regions between all N-dimensional image pairs.

[0067] Preferably, the image processing device 600 according to this embodiment further includes an adjustment device 601 ′, which is used to adjust the multiple N-dimensional images to the same viewing angle before intercepting the overlapping area.

[0068] Figure 6 The image processing device 600 shown corresponds to Figure 1 Therefore, the details of each device in the image processing device 600 have been described in detail. Figure 1 The details of the image processing method 100 are given in detail and will not be repeated here.

[0069] Each component module and unit in the above device can be configured by software, firmware, hardware or a combination thereof. The specific means or methods that can be used for configuration are well known to those skilled in the art and will not be described in detail here. In the case of implementation by software or firmware, the data is transferred from a storage medium or a network to a computer with a dedicated hardware structure (e.g. Figure 7 The general-purpose computer 700 shown in FIG. 1 is installed with programs constituting the software. When various programs are installed, the computer can execute various functions, etc.

[0070] Figure 7 FIG. 1 is a block diagram of an exemplary structure of a general-purpose personal computer in which the methods and / or apparatus according to embodiments of the present invention may be implemented. Figure 7As shown, a central processing unit (CPU) 701 executes various processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage section 708 to a random access memory (RAM) 703. In the RAM 703, data required when the CPU 701 executes various processes, etc., is also stored as needed. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output interface 705 is also connected to the bus 704.

[0071] The following components are connected to the input / output interface 705: an input section 706 (including a keyboard, a mouse, etc.), an output section 707 (including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.), a storage section 708 (including a hard disk, etc.), and a communication section 709 (including a network interface card such as a LAN card, a modem, etc.). The communication section 709 performs communication processing via a network such as the Internet. A drive 710 may also be connected to the input / output interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is installed in the drive 710 as needed, so that a computer program read therefrom is installed in the storage section 708 as needed.

[0072] In the case of realizing the above-described series of processing by software, a program constituting the software is installed from a network such as the Internet or a storage medium such as the removable medium 711 .

[0073] It should be understood by those skilled in the art that such storage media is not limited to Figure 7 The removable medium 711 shown has a program stored therein and is distributed separately from the device to provide the program to the user. Examples of the removable medium 711 include magnetic disks (including floppy disks (registered trademark)), optical disks (including compact disk read-only memories (CD-ROMs) and digital versatile disks (DVDs)), magneto-optical disks (including minidiscs (MDs) (registered trademark)), and semiconductor memories. Alternatively, the storage medium may be ROM 702, a hard disk included in storage section 708, or the like, in which the program is stored and distributed to the user along with the device containing it.

[0074] The present invention also provides corresponding computer program codes and a computer program product storing machine-readable instruction codes. When the instruction codes are read and executed by a machine, the method according to the embodiment of the present invention can be executed.

[0075] Accordingly, storage media configured to carry the program product storing the machine-readable instruction code are also included in the disclosure of the present invention, including but not limited to floppy disks, optical disks, magneto-optical disks, memory cards, memory sticks, and the like.

[0076] Through the above description, the embodiments of the present disclosure provide the following technical solutions, but are not limited thereto.

[0077] Note 1. An image processing method comprising:

[0078] extracting overlapping regions from pairs of N-dimensional images of portions of a photographed object that are adjacent to each other on the object, among a plurality of N-dimensional images of the respective portions of the photographed object, where N is an integer greater than or equal to three;

[0079] Blocking the overlapping areas of the adjacent N-dimensional image pairs in N directions respectively, so as to obtain two groups of N-dimensional block images for each direction;

[0080] Projecting the two groups of N-dimensional block images in corresponding directions to obtain two groups of two-dimensional block images;

[0081] performing pairwise matching of the two groups of two-dimensional block images to obtain offsets of the two groups of two-dimensional block images in corresponding directions with the highest confidence;

[0082] If the offset is greater than the predetermined threshold, repeat the above steps according to the offset until the offset is less than the predetermined threshold;

[0083] Iteratively performing the above steps for other adjacent N-dimensional image pairs in the plurality of N-dimensional images; and

[0084] The image size after the multiple N-dimensional images are stitched together is recalculated based on the obtained offsets of all N-dimensional image pairs to perform image stitching, and overlapping areas between all N-dimensional image pairs are fused.

[0085] Note 2. The image processing method according to Note 1 further includes adjusting the multiple N-dimensional images to the same viewing angle before intercepting the overlapping area.

[0086] Note 3. The image processing method according to Note 2, wherein the overlapping areas are fused using a fusion method based on Gaussian filtering.

[0087] Supplement 4. The image processing method according to any one of Supplements 1 to 3, wherein the step of dividing the overlapping area of ​​the adjacent N-dimensional image pair into blocks in N directions comprises:

[0088] The number of blocks of the overlapping area in each direction is determined according to the ratio of the length of the overlapping area in each direction to the difference between the block thickness and the overlap.

[0089] Note 5. The image processing method according to any one of Notes 1 to 3, wherein the projection is based on a maximum intensity projection method.

[0090] Supplementary Note 6. The image processing method according to any one of Supplementary Notes 1 to 3, wherein performing pairwise matching between the two sets of two-dimensional block images to obtain the offsets of the two sets of two-dimensional block images in their corresponding directions with the highest confidence comprises:

[0091] Matching two-dimensional block images in one of the two groups of two-dimensional block images with two-dimensional block images in the other group in pairs to obtain offsets in other N-1 directions other than the corresponding directions of the two groups of two-dimensional block images and confidence levels under the offsets; and

[0092] The offset with the highest confidence is taken as the offset along the corresponding direction.

[0093] Note 7. The image processing method according to Note 6, wherein the matching is based on a normalized cross-correlation algorithm.

[0094] Supplementary note 8. The image processing method according to Supplementary note 6, wherein recalculating the size of the image after splicing the multiple N-dimensional images together comprises:

[0095] Calculating the maximum absolute position of each N-dimensional image in N directions based on the corresponding offsets obtained; and

[0096] The size of an image after the multiple N-dimensional images are spliced ​​together is determined according to all the obtained maximum absolute positions.

[0097] Note 9. The image processing method according to any one of Notes 1 to 3, wherein the predetermined threshold is less than or equal to 5 pixels.

[0098] Note 10. An image processing device comprising:

[0099] a cropping device configured to crop an overlapping region for a pair of N-dimensional images of portions of a photographed object that are adjacent to each other on the object, among a plurality of N-dimensional images of the portions of the photographed object, wherein N is an integer greater than or equal to three;

[0100] a blocking device configured to block the overlapping area of ​​the adjacent N-dimensional image pair in N directions respectively, so as to obtain two groups of N-dimensional blocked images for each direction;

[0101] a projection device configured to project the two groups of N-dimensional block images in corresponding directions to obtain two groups of two-dimensional block images;

[0102] a matching device configured to perform pairwise matching of the two-dimensional block images between the two groups of two-dimensional block images to obtain the offsets of the two groups of two-dimensional block images in their corresponding directions with the highest confidence; and

[0103] a determining device configured to determine whether the offset is greater than a predetermined threshold, and if the offset is greater than the predetermined threshold, re-cut the overlapping area according to the offset and re-determine the offset with the highest confidence until the offset is less than the predetermined threshold; and

[0104] The stitching device is configured to recalculate the image size after stitching the multiple N-dimensional images together based on the offsets of all N-dimensional image pairs obtained by the determining device to perform image stitching, and to fuse the overlapping areas between all N-dimensional image pairs.

[0105] Supplementary note 11. The image processing apparatus according to Supplementary note 10 further comprises an angle adjustment device configured to adjust the plurality of N-dimensional images to the same field of view angle before intercepting the overlapping area.

[0106] Note 12. The image processing device according to Note 11, wherein the overlapping areas are fused using a fusion method based on Gaussian filtering.

[0107] Supplement 13. The image processing apparatus according to any one of Supplements 10 to 12, wherein the blocking device is further configured to:

[0108] The number of blocks of the overlapping area in each direction is determined according to the ratio of the length of the overlapping area in each direction to the difference between the block thickness and the overlap.

[0109] Supplement 14. The image processing apparatus according to any one of Supplements 10 to 12, wherein the projection is based on a maximum intensity projection method.

[0110] Supplement 15. The image processing apparatus according to any one of Supplements 10 to 12, wherein the matching device is further configured to:

[0111] Matching two-dimensional block images in one of the two groups of two-dimensional block images with two-dimensional block images in the other group in pairs to obtain offsets in other N-1 directions other than the corresponding directions of the two groups of two-dimensional block images and confidence levels under the offsets; and

[0112] The offset with the highest confidence is taken as the offset along the corresponding direction.

[0113] Supplement 16. The image processing apparatus according to Supplement 15, wherein the matching is based on a normalized cross-correlation algorithm.

[0114] Supplementary note 17. The image processing device according to Supplementary note 15, wherein recalculating the size of the image after stitching together the multiple N-dimensional images comprises:

[0115] Calculating the maximum absolute position of each N-dimensional image in N directions based on the corresponding offsets obtained; and

[0116] The size of an image after the multiple N-dimensional images are spliced ​​together is determined according to all the obtained maximum absolute positions.

[0117] Supplement 18. The image processing device according to any one of Supplements 10 to 12, wherein the predetermined threshold is less than or equal to 5 pixels.

[0118] Note 19. A computer-readable storage medium storing a program that can be executed by a processor to perform the following operations:

[0119] extracting overlapping regions from pairs of N-dimensional images of portions of a photographed object that are adjacent to each other on the object, among a plurality of N-dimensional images of the respective portions of the photographed object, where N is an integer greater than or equal to three;

[0120] Blocking the overlapping areas of the adjacent N-dimensional image pairs in N directions respectively, so as to obtain two groups of N-dimensional block images for each direction;

[0121] Projecting the two groups of N-dimensional block images in corresponding directions to obtain two groups of two-dimensional block images;

[0122] performing pairwise matching of the two groups of two-dimensional block images to obtain offsets of the two groups of two-dimensional block images in corresponding directions with the highest confidence;

[0123] If the offset is greater than the predetermined threshold, repeat the above steps according to the offset until the offset is less than the predetermined threshold;

[0124] Iteratively performing the above steps for other adjacent N-dimensional image pairs in the plurality of N-dimensional images; and

[0125] The image size after the multiple N-dimensional images are stitched together is recalculated based on the obtained offsets of all N-dimensional image pairs to perform image stitching, and overlapping areas between all N-dimensional image pairs are fused.

[0126] Finally, it should be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. Furthermore, in the absence of further limitations, an element defined by the phrase "comprises a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0127] Although the embodiments of the present invention have been described in detail above with reference to the accompanying drawings, it should be understood that the embodiments described above are merely illustrative of the present invention and are not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations may be made to the above embodiments without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention is limited solely by the appended claims and their equivalents.

Claims

1. An image processing method, comprising: Adjust multiple N-dimensional images to the same viewing angle; extracting overlapping regions from pairs of N-dimensional images of portions of a photographed object that are adjacent to each other on the object, among a plurality of N-dimensional images of the respective portions of the photographed object, where N is an integer greater than or equal to three; The overlapping regions of the adjacent N-dimensional image pairs are divided into blocks in N directions, and the number of blocks of the overlapping region in each direction is determined based on the ratio of the length of the overlapping region in each direction to the difference between the block thickness and the overlap, so as to obtain two groups of N-dimensional block images in each direction; and the two groups of N-dimensional block images are projected in their corresponding directions based on the maximum projection density method to obtain two groups of two-dimensional block images. Matching two-dimensional block images in one of the two groups of two-dimensional block images with two-dimensional block images in the other group in pairs to obtain offsets in N-1 directions other than the corresponding directions of the two groups of two-dimensional block images and confidence levels of the offsets, and taking the offset with the highest confidence level as the offset along the corresponding direction; If the offset is greater than the predetermined threshold, repeat the above steps according to the offset until the offset is less than the predetermined threshold; Iteratively performing the above steps on other adjacent N-dimensional image pairs in the plurality of N-dimensional images; and Based on the offsets of all the obtained N-dimensional image pairs, the maximum absolute position of each N-dimensional image in N directions is calculated, the image size after the multiple N-dimensional images are stitched together is determined according to all the obtained maximum absolute positions to perform image stitching, and the overlapping areas between all the N-dimensional image pairs are fused.

2. The image processing method according to claim 1, wherein: The overlapping areas are fused using a fusion method based on Gaussian filtering.

3. The image processing method according to claim 1 or 2, wherein: The matching is based on a normalized cross-correlation algorithm.

4. The image processing method according to claim 1 or 2, wherein: The predetermined threshold is less than or equal to 5 pixels.

5. An image processing device comprising: A rotating device adjusts multiple N-dimensional images to the same viewing angle; a cropping device configured to crop an overlapping region for a pair of N-dimensional images of portions of a photographed object that are adjacent to each other on the object, among a plurality of N-dimensional images of the portions of the photographed object, wherein N is an integer greater than or equal to three; a blocking device configured to block the overlapping area of ​​the adjacent N-dimensional image pair in N directions respectively, and determine the number of blocks of the overlapping area in each direction according to the ratio of the length of the overlapping area in each direction to the difference between the block thickness and the overlap, so as to obtain two sets of N-dimensional blocked images in each direction; A projection device configured to project the two groups of N-dimensional block images in corresponding directions based on a maximum projection density method to obtain two groups of two-dimensional block images; a matching device configured to perform pairwise matching of the two sets of two-dimensional block images to obtain offsets in N-1 directions other than the corresponding directions of the two sets of two-dimensional block images and confidence levels under the offsets, and to use the offset with the highest confidence level as the offset along the corresponding direction; a determining device configured to determine whether the offset is greater than a predetermined threshold, and if the offset is greater than the predetermined threshold, re-cut the overlapping area according to the offset and re-determine the offset with the highest confidence until the offset is less than the predetermined threshold; and A stitching device is configured to calculate the maximum absolute position of each N-dimensional image in N directions based on the offsets of all N-dimensional image pairs obtained by the determining device, determine the image size after stitching the multiple N-dimensional images together according to all the obtained maximum absolute positions to perform image stitching, and fuse the overlapping areas between all N-dimensional image pairs.

6. A computer-readable storage medium storing a program that can be executed by a processor to perform the following operations: Adjust multiple N-dimensional images to the same viewing angle; extracting overlapping regions from pairs of N-dimensional images of portions of a photographed object that are adjacent to each other on the object, among a plurality of N-dimensional images of the respective portions of the photographed object, where N is an integer greater than or equal to three; Blocking the overlapping regions of the adjacent N-dimensional image pairs in N directions respectively, and determining the number of blocks of the overlapping region in each direction based on the ratio of the length of the overlapping region in each direction to the difference between the block thickness and the overlap, so as to obtain two sets of N-dimensional block images for each direction; Projecting the two groups of N-dimensional block images in corresponding directions based on a maximum projection density method to obtain two groups of two-dimensional block images; performing pairwise matching of the two sets of two-dimensional block images to obtain offsets in N-1 directions other than the corresponding directions of the two sets of two-dimensional block images and confidence levels of the offsets, and taking the offset with the highest confidence level as the offset along the corresponding direction; If the offset is greater than the predetermined threshold, repeat the above steps according to the offset until the offset is less than the predetermined threshold; Iteratively performing the above steps on other adjacent N-dimensional image pairs in the plurality of N-dimensional images; and Based on the offsets of all the obtained N-dimensional image pairs, the maximum absolute position of each N-dimensional image in N directions is calculated, the image size after the multiple N-dimensional images are stitched together is determined according to all the obtained maximum absolute positions to perform image stitching, and the overlapping areas between all the N-dimensional image pairs are fused.

Images