Spatial plane image comparison method, device and computer readable storage medium

Abstract

The embodiment of the present invention provides a method, device and computer-readable storage medium for comparing spatial plane images. The comparison method includes: acquiring a first spatial plane image and a second spatial plane image; performing corner point detection on the first spatial plane image, and determining multiple first image blocks based on multiple corner points; performing template matching on each of the multiple first image blocks with the second spatial plane image to obtain a matched second image block on the second spatial plane image; aligning a specific first image block with the matched second image block according to the position difference and / or pixel difference between each first image block and the matched second image block to obtain an aligned first spatial plane image and a second spatial plane image; and comparing the pixel categories of pixels at corresponding positions of the aligned first spatial plane image and the second spatial plane image to obtain a comparison result of the first spatial plane image and the second spatial plane image.

Description

Technical Field

[0001] The present application relates to the field of spatial plane image processing, and in particular to a spatial plane image comparison method, device and computer-readable storage medium. Background Art

[0002] Currently, in the fields of virtual reality (VR), urban planning, real estate, etc., it is usually necessary to obtain a spatial plane image that reflects the two-dimensional layout of the entire three-dimensional space in order to achieve the purpose of fast navigation, planning and design, or apartment type screening. For example, in the process of virtual roaming (which is an important branch of virtual reality technology), the roamer can use the spatial plane image covering each three-dimensional scene to select different navigation points on the image to quickly jump to the next scene he wants to roam. For another example, in the real estate industry, it is usually necessary to generate a spatial plane image of the entire house (in this case, it can be called a floor plan or house floor plan) so that customers can screen apartment types.

[0003] In addition, the development of technologies such as deep learning has made it possible to automatically generate spatial plane images using multiple images obtained by shooting three-dimensional scenes. In this case, it is often necessary to evaluate the accuracy of the generated images in order to evaluate and adjust the algorithm. To this end, an accurate target image obtained manually (for example, drawn by software) can be compared with an image generated by the algorithm. However, the existing spatial plane image comparison methods measure coarse-grained standards such as the segmentation status, shape or area of ​​the sub-regions in the image, and the comparison results obtained are relatively rough and cannot accurately measure the accuracy of the spatial plane image.

[0004] It can be seen that it is necessary to construct a method and device that can obtain more accurate spatial plane image comparison results. Summary of the invention

[0005] To solve the above technical problems, according to one aspect of the present invention, a spatial plane image comparison method is provided, wherein the spatial plane image includes multiple non-overlapping sub-regions and each sub-region category is associated with a corresponding pixel category, and the method includes: acquiring a first spatial plane image and a second spatial plane image, wherein the first spatial plane image and the second spatial plane image have the same resolution; performing corner point detection on the first spatial plane image to obtain multiple corner points, and determining multiple first image blocks on the first spatial plane image based on the multiple corner points; performing template matching on each of the multiple first image blocks with the second spatial plane image to obtain a matched second image block on the second spatial plane image; aligning a specific first image block with the matched second image block according to the position difference and / or pixel difference between each first image block and the matched second image block to obtain the aligned first spatial plane image and the second spatial plane image; and comparing the pixel categories of pixels at corresponding positions of the aligned first spatial plane image and the second spatial plane image to obtain a comparison result of the first spatial plane image and the second spatial plane image.

[0006] According to another aspect of the present invention, there is provided a spatial plane image comparison device, wherein the spatial plane image includes a plurality of non-overlapping sub-regions and each sub-region category is associated with a corresponding pixel category, and the device includes: an acquisition unit, configured to acquire a first spatial plane image and a second spatial plane image, wherein the first spatial plane image and the second spatial plane image are associated with the same three-dimensional space and have the same resolution; a detection unit, configured to perform corner point detection on the first spatial plane image to obtain a plurality of corner points, and determine a plurality of first image blocks on the first spatial plane image based on the plurality of corner points; a matching unit, configured to perform template matching on each of the plurality of first image blocks with the second spatial plane image to obtain a matched second image block on the second spatial plane image; an alignment unit, configured to align a specific first image block with a matched second image block according to a position difference and / or pixel difference between each first image block and the matched second image block to obtain the aligned first spatial plane image and the second spatial plane image; and a comparison unit, configured to compare the pixel categories of pixels at corresponding positions of the aligned first spatial plane image and the second spatial plane image to obtain a comparison result of the first spatial plane image and the second spatial plane image.

[0007] According to another aspect of the present invention, there is provided a spatial plane image comparison device, wherein the spatial plane image includes a plurality of non-overlapping sub-regions and each sub-region category is associated with a corresponding pixel category, and the device includes: a processor; and a memory, wherein computer program instructions are stored in the memory, wherein when the computer program instructions are executed by the processor, the processor is caused to perform the following steps: acquiring a first spatial plane image and a second spatial plane image, wherein the first spatial plane image and the second spatial plane image are associated with the same three-dimensional space and have the same resolution; performing corner point detection on the first spatial plane image to obtain a plurality of corner points, and determining a plurality of first image blocks on the first spatial plane image based on the plurality of corner points; performing template matching on each of the plurality of first image blocks with the second spatial plane image to obtain a matched second image block on the second spatial plane image; aligning a specific first image block with the matched second image block according to the position difference and / or pixel difference between each first image block and the matched second image block to obtain the aligned first spatial plane image and the second spatial plane image; and comparing the pixel categories of pixels at corresponding positions of the aligned first spatial plane image and the second spatial plane image to obtain a comparison result of the first spatial plane image and the second spatial plane image.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the spatial plane image comparison method as described above.

[0009] According to the above-mentioned spatial plane image comparison method, device and computer-readable storage medium of the present invention, two spatial plane images can be aligned and each pixel on the image can be compared by using corner point detection and template matching, so as to obtain a spatial plane image comparison result at the pixel level. This spatial plane image comparison method, device and computer-readable storage medium can obtain a more accurate comparison result at the pixel level, and it is convenient to improve the image generation algorithm by using the comparison result to improve the generation quality of the spatial plane image. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other objects, features and advantages of the present invention will become more apparent by describing in detail the embodiments of the present invention in conjunction with the accompanying drawings.

[0011] Figure 1 A flow chart of a spatial plane image comparison method according to an embodiment of the present invention is shown;

[0012] Figure 2A schematic diagram showing acquisition of a first spatial plane image and a second spatial plane image according to an embodiment of the present invention is shown;

[0013] Figure 3 A schematic diagram showing corner point detection for a first spatial plane image according to an embodiment of the present invention is shown;

[0014] Figure 4 A schematic diagram showing accurate matching and inaccurate matching results obtained by performing template matching between a first image block and a second spatial plane image according to an embodiment of the present invention;

[0015] Figure 5 A schematic diagram showing an aligned first spatial plane image and a second spatial plane image shown in an overlapping manner according to an embodiment of the present invention is shown;

[0016] Figure 6 shows a block diagram of a spatial plane image comparison device 600 according to an embodiment of the present invention;

[0017] Figure 7 FIG. 7 shows a block diagram of a spatial plane image comparison device 700 according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will describe a spatial plane image comparison method, device, and computer-readable storage medium according to an embodiment of the present invention with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same elements from beginning to end. It should be understood that the embodiments described herein are merely illustrative and should not be interpreted as limiting the scope of the present invention.

[0019] In an embodiment of the present invention, a spatial plane image can be used to describe the two-dimensional layout of a three-dimensional space, and can include multiple non-overlapping sub-areas. For example, when the spatial plane image is a floor plan, it can show the two-dimensional layout of the position, shape, size, etc. of each room, door, wall, etc. inside the house, and the multiple non-overlapping sub-areas can represent different rooms, doors, walls, etc.

[0020] In addition, in an embodiment of the present invention, each sub-region category of the spatial plane image can be associated with a corresponding pixel category, so as to facilitate the distinction between different sub-regions on the image and pixel comparison. In one example, the pixel category can be defined according to the pixel color, so that each sub-region category is associated with a different pixel color. For example, when the spatial plane image is a floor plan, the pixels contained in the room sub-region with 4 vertices can be red, the pixels contained in the room sub-region with 6 vertices can be green, the pixels contained in the sub-region corresponding to the door can be blue, the pixels contained in the sub-region corresponding to the wall can be black, and so on. In another example, the pixel category can also be defined according to the pixel gray value, so that each sub-region category is associated with a different pixel gray value. In practical applications, more or fewer sub-region categories can be divided on the spatial plane image according to the specific situation, and there is no limitation here.

[0021] The following will refer to Figure 1 A spatial plane image comparison method according to an embodiment of the present invention is described. Figure 1 A flow chart of the spatial plane image comparison method 100 is shown.

[0022] like Figure 1 As shown, in step S101, a first spatial plane image and a second spatial plane image are acquired, wherein the first spatial plane image and the second spatial plane image have the same resolution.

[0023] Figure 2 A schematic diagram of obtaining a first spatial plane image and a second spatial plane image according to an embodiment of the present invention is shown. In this example, the first spatial plane image may be an accurate target image used as a comparison benchmark, and the second spatial plane image may be an image generated after a plurality of images obtained by shooting a three-dimensional scene are input into an algorithm. For example, the first spatial plane image may be obtained by direct drawing. In addition, the first spatial plane image may also be obtained by annotating an actual spatial plane image (e.g., a house floor plan, a building construction drawing, etc.), for example, removing redundant details on the actual spatial plane image to retain only the outer contours of each sub-region, and annotating the pixels of each sub-region on the image using the corresponding pixel category, etc. The second spatial plane image may be an image generated using a method such as a deep learning-based method (such as a HorizonNet method, a LayoutNet method, a RoomNet method, etc.) or an image segmentation method based on a CV model, and each sub-region category on the generated second spatial plane image may be associated with a corresponding pixel category.

[0024] In the following, the first spatial plane image as the target image and the second spatial plane image as the algorithm-generated image will be used as examples for specific description, but the present invention is not limited thereto. In practical applications, the first spatial plane image may be an algorithm-generated image and the second spatial plane image may be a target image, or the first spatial plane image and the second spatial plane image may be spatial plane images to be compared obtained by any means, which are not limited here.

[0025] In this step, when the resolutions of the initial first spatial plane image and the second spatial plane image are inconsistent, one of the first spatial plane image and the second spatial plane image can be first upsampled or downsampled to obtain the first spatial plane image and the second spatial plane image with the same resolution, thereby facilitating the subsequent pixel comparison process thereon.

[0026] In step S102, corner point detection is performed on the first spatial plane image to obtain a plurality of corner points, and a plurality of first image blocks on the first spatial plane image are determined based on the plurality of corner points.

[0027] Figure 3 A schematic diagram of performing corner point detection on a first spatial plane image according to an embodiment of the present invention is shown. In this step, the first spatial plane image can be processed using a method such as the Harris corner point detection method, the Shi-Tomasi corner point detection method, etc. to obtain multiple corner points detected on the first spatial plane image.

[0028] After obtaining the detected multiple corner points, multiple first image blocks on the first spatial plane image can be determined based on the multiple corner points, so that the multiple first image blocks can be used as multiple templates for template matching with the second spatial plane image. For example, the detected corner points can be used as the upper left corner, lower left corner, upper right corner or lower right corner vertex of the image block to define the first image block corresponding to each corner point. The size of the appropriate first image block can be selected according to the size of the entire first spatial plane image and the size of each sub-area therein, such as 8*8, 16*16 or 32*32 (in pixels), etc., which is not limited here.

[0029] In one example, determining the plurality of first image blocks on the first spatial plane image based on the plurality of corner points may include: determining the plurality of first image blocks by taking each of the plurality of corner points as a center point of an image block. Figure 3As shown, since the detected corner points are usually located near the junction of each sub-area, the first image block constructed by taking it as the center point of the image block can have more pixel differences, so that a more ideal template matching result can be obtained when it is used as a template for template matching.

[0030] In step S103, template matching is performed on each of the plurality of first image blocks and the second spatial plane image to obtain a matched second image block on the second spatial plane image.

[0031] In this step, each of the multiple first image blocks can be used as a template to slide on the second spatial plane image, and an image block of the same size as the first image block can be cut out from the second spatial plane image to perform a pixel comparison operation, so as to find the second image block that is most matched or closest to the first image block on the second spatial plane image.

[0032] Figure 4 A schematic diagram showing accurate matching and inaccurate matching results obtained by performing template matching between a first image block and a second spatial plane image according to an embodiment of the present invention. Figure 4 The first image block 401 and the matched second image block 402 with a less ideal matching result are shown above, while Figure 4 The first image block 403 and the matched second image block 404 with relatively accurate matching results are shown below. That is to say, not all combinations of the first image block and the matched second image block are exactly matched combinations. Therefore, it is necessary to further screen the combinations of the first image blocks and the matched second image blocks, so as to screen out the exactly matched image block combinations to align the first spatial plane image and the second spatial plane image.

[0033] In step S104, a specific first image block is aligned with the matched second image block according to the position difference and / or pixel difference between each first image block and the matched second image block to obtain the aligned first spatial plane image and the second spatial plane image.

[0034] In this step, an accurately matched image block combination can be screened out as a specific first image block and a matched second image block based on the position difference and / or pixel difference between each first image block and the matched second image block, thereby aligning the first spatial plane image and the second spatial plane image by aligning the specific first image block and the matched second image block.

[0035] In one example, the distance between a specific point on each first image block and a corresponding point on the matched second image block can be used as the position difference. For example, the position difference can include the distance between the center point of each first image block and the center point of the matched second image block. When the first image block is determined by taking the corner point as the center point of the image block, the position difference is equivalent to the distance between each corner point and the center point of the matched second image block. The pixel difference can be determined by comparing the pixel categories of the pixels at the corresponding positions of each first image block and the matched second image block. For example, in the case where the pixel category is defined according to the pixel color, the pixel difference can be the number of pixels with different pixel colors at the corresponding positions of each first image block and the matched second image block. In addition, a pixel comparison operation can be performed on each first image block and the matched second image block and an operation result such as the square sum of pixel difference values ​​can be obtained as the pixel difference, and in this case, the pixel comparison operation result obtained in the template matching process can be directly used to further reduce the amount of operation.

[0036] In one example, the first image block with the smallest position difference can be aligned with the matched second image block. In another example, the first image block with the smallest pixel difference can also be aligned with the matched second image block. In addition, in order to further screen out the most accurately matched image block combination to obtain a better alignment effect, both pixel differences and position differences can be considered comprehensively. For example, aligning a specific first image block with a matched second image block based on the position difference and / or pixel difference between each first image block and the matched second image block can include: determining the sum of the position difference and pixel difference between each first image block and the matched second image block, and aligning the first image block with the smallest sum of the position difference and pixel difference with the matched second image block.

[0037] An example of experimental results obtained by template matching according to an embodiment of the present invention is shown in Table 1. As shown in the following Table 1, the position difference and pixel difference corresponding to each combination of the first image block and the matched second image block are calculated, wherein combination 5 has the smallest position difference, while the pixel differences of combinations 2, 3, 4 and 7 are all 0, however, since combination 4 has the smallest sum of position difference and pixel difference, after comprehensively considering the pixel difference and position difference, combination 4 can be screened out as the most accurately matched image block combination, so that the specific first image block corresponding to combination 4 and the matched second image block are aligned to obtain the aligned first spatial plane image and the second spatial plane image.

[0038] Table 1

[0039]

[0040]

[0041] In step S105, pixel categories of pixels at corresponding positions of the aligned first spatial plane image and the second spatial plane image are compared to obtain a comparison result of the first spatial plane image and the second spatial plane image.

[0042] Since the aligned first spatial plane image and the second spatial plane image have been obtained through the above steps, the pixel-level spatial plane image comparison result can be obtained by comparing the pixels at corresponding positions on the two images. In one example, since the aligned first spatial plane image and the second spatial plane image may not completely overlap, the pixels in the overlapped part of the two aligned images can be selected for comparison, and the pixels in the non-overlapped part of the two images can be ignored.

[0043] In this step, the comparison of the pixel categories of the pixels at the corresponding positions of the first spatial plane image and the second spatial plane image after alignment to obtain the comparison result of the first spatial plane image and the second spatial plane image may include: comparing the pixel categories of the pixels at the corresponding positions of the first spatial plane image and the second spatial plane image after alignment pixel by pixel, and taking the number of pixels with the same or different pixel categories as the comparison result of the first spatial plane image and the second spatial plane image. For example, in the case where the pixel category is defined according to the pixel color, the comparison result may be the number of pixels with the same or different pixel colors at the corresponding positions of the first spatial plane image and the second spatial plane image.

[0044] In addition, in one example, the first spatial plane image and the second spatial plane image may not be completely covered by each sub-region, and there may be a blank area composed of transparent pixels (for example, Figure 1). In this case, when determining whether two pixels at corresponding positions of the first spatial plane image and the second spatial plane image have the same pixel category, it can be determined first whether the two pixels are both pixels of the transparent category: if the two pixels are both pixels of the transparent category, it can be determined that the two pixels are both located in the blank area and are considered to have the same pixel category; if one of the two pixels is a pixel of the transparent category and the other is not a pixel of the transparent category, it can be considered that the two pixels have different pixel categories; if the two pixels are not pixels of the transparent category, it can be determined that the two pixels are not located in the blank area, and the pixel categories (for example, pixel color or pixel grayscale value, etc.) associated with the sub-area category of the two pixels are further compared to finally determine whether the two pixels have the same pixel category.

[0045] Figure 5 FIG. 4 is a schematic diagram showing an aligned first spatial plane image and a second spatial plane image in an overlapping manner according to an embodiment of the present invention. Figure 5 The part shown in the lightest color in the upper right corner of indicates that the sub-region corresponding to the part only exists in the first spatial plane image, but is missing in the second spatial plane image. The reason for this missing may be that, for example, when using a camera to shoot a three-dimensional scene, some local areas in the scene may be missed, so that the generated second spatial plane image lacks the part corresponding to the local area and an erroneous blank area appears. In this case, the comparison result obtained by comparing the entire first spatial plane image with the second spatial plane image, which is quite different, may not be a true measure of the algorithm used to generate the second spatial plane image.

[0046] Therefore, according to one embodiment of the present invention, only the pixels in the sub-region where the specific first image block is located and the pixels in the sub-region where the second image block matched according to the specific first image block is located can be compared, and the obtained sub-region pixel comparison result can be used to measure the algorithm for generating the second spatial plane image. In the case where the first image block is determined by taking the corner point as the center point of the image block, the sub-region where the center point of the specific first image block is located can be the first sub-region, and the sub-region where the center point of the second image block matched according to the specific first image block is located can be the second sub-region, and the pixel categories of the pixels at the corresponding positions of the first sub-region and the second sub-region can be compared pixel by pixel to obtain the number of pixels with the same or different pixel categories as the comparison result of the first sub-region and the second sub-region, so that the comparison result is used to measure the algorithm for generating the second spatial plane image. When the specific first image block and the matched second image block cover multiple sub-regions, one of the multiple sub-regions or a combination of sub-regions can be arbitrarily selected for comparison, without limitation here.

[0047] In addition, according to another embodiment of the present invention, in addition to comparing the first sub-region and the second sub-region, other corresponding sub-regions for comparison can be found based on the first sub-region and the second sub-region using the sub-region connection relationship. For example, the acquisition of the first spatial plane image and the second spatial plane image includes acquiring the sub-region connection relationship of the first spatial plane image or the second spatial plane image, and the method further includes: based on the sub-region connection relationship and the first sub-region and the second sub-region, respectively determining the third sub-region on the first spatial plane image and the fourth sub-region on the second spatial plane image, and comparing the pixel categories of the pixels at the corresponding positions of the third sub-region and the fourth sub-region pixel by pixel, so as to obtain the number of pixels with the same or different pixel categories as the comparison result of the third sub-region and the fourth sub-region.

[0048] In this embodiment, the sub-region connection relationship may reflect the positional relationship between each interconnected sub-region. For example, the sub-region connection relationship may record in a list the other sub-regions to which each sub-region is connected in the four directions of up, down, left and right. Therefore, in one example, based on the sub-region connection relationship, a sub-region connected to the first sub-region in a specific direction may be taken as the third sub-region, and a sub-region connected to the second sub-region in the same direction may be taken as the fourth sub-region, thereby determining the corresponding third sub-region and fourth sub-region for comparison.

[0049] In a specific application scenario in which the image generation algorithm is improved by using the comparison result, the first spatial plane image and the second spatial plane image may be associated with the same three-dimensional space, wherein the first spatial plane image may be a training image obtained by pre-marking a plane image of the three-dimensional space, and the second spatial plane image may be an image output by inputting a panoramic image of the three-dimensional space into a neural network for generating spatial plane images. In this case, the comparison result may be used to construct a loss function for training the neural network, thereby training the neural network using the loss function to adjust the parameters of the neural network, and using the parameters when the loss function converges as the adjusted parameters.

[0050] In this application scenario, the loss function can be constructed based on the comparison result of the aligned entire first spatial plane image and the second spatial plane image. In addition, as mentioned above, since the comparison result based on the entire first spatial plane image and the second image spatial plane image may not be a good measure of the algorithm used to generate the second spatial plane image, the loss function can also be constructed based on the comparison result of one or more corresponding sub-regions. The type of loss function can be a cross entropy loss function or a Smooth L1 loss function, but other types of loss functions can also be used, which are not limited here.

[0051] According to the above-mentioned spatial plane image comparison method of the present invention, two spatial plane images can be aligned by using corner point detection and template matching and each pixel on the image can be compared, thereby obtaining a spatial plane image comparison result at the pixel level. This spatial plane image comparison method can obtain a more accurate comparison result at the pixel level, and it is convenient to use the comparison result to improve the image generation algorithm to improve the generation quality of the spatial plane image.

[0052] Below, refer to Figure 6 To describe the spatial plane image comparison device according to an embodiment of the present invention. Figure 6 FIG. 6 is a block diagram of a spatial plane image comparison device 600 according to an embodiment of the present invention. Figure 6 As shown, the spatial plane image comparison device 600 includes an acquisition unit 610, a detection unit 620, a matching unit 630, an alignment unit 640, a comparison unit 650, and a training unit 660. In addition to these units, the spatial plane image comparison device 600 may also include other components, however, since these components are irrelevant to the content of the embodiment of the present invention, their illustration and description are omitted here.

[0053] Figure 6The acquisition unit 610 acquires a first spatial plane image and a second spatial plane image, wherein the first spatial plane image and the second spatial plane image have the same resolution.

[0054] Figure 2 A schematic diagram of obtaining a first spatial plane image and a second spatial plane image according to an embodiment of the present invention is shown. In this example, the first spatial plane image obtained by the acquisition unit 610 may be an accurate target image used as a comparison benchmark, and the second spatial plane image obtained may be an image generated after a plurality of images obtained by shooting a three-dimensional scene are input into an algorithm. For example, the first spatial plane image may be obtained by direct drawing. In addition, the first spatial plane image may also be obtained by annotating an actual spatial plane image (e.g., a house floor plan, a building construction drawing, etc.), for example, removing redundant details on the actual spatial plane image to retain only the outer contours of each sub-region, and annotating the pixels of each sub-region on the image using the corresponding pixel category, etc. The second spatial plane image may be an image generated using a method such as a deep learning-based method (such as a HorizonNet method, a LayoutNet method, a RoomNet method, etc.) or an image segmentation method based on a CV model, and each sub-region category on the generated second spatial plane image may be associated with a corresponding pixel category.

[0055] In the following, the first spatial plane image as the target image and the second spatial plane image as the algorithm-generated image will be specifically described as an example, but the present invention is not limited to this. In practical applications, the first spatial plane image acquired by the acquisition unit 610 may be an algorithm-generated image and the second spatial plane image acquired may be a target image, or the first spatial plane image and the second spatial plane image acquired may be spatial plane images that need to be compared obtained by any means, which are not limited here.

[0056] Among them, when the resolutions of the initial first spatial plane image and the second spatial plane image acquired by the acquisition unit 610 are inconsistent, the acquisition unit 610 can first upsample or downsample one of the first spatial plane image and the second spatial plane image to obtain the first spatial plane image and the second spatial plane image with the same resolution, thereby facilitating the subsequent pixel comparison process thereon.

[0057] The detection unit 620 performs corner point detection on the first spatial plane image to obtain a plurality of corner points, and determines a plurality of first image blocks on the first spatial plane image based on the plurality of corner points.

[0058] Figure 3A schematic diagram of performing corner point detection on a first spatial plane image according to an embodiment of the present invention is shown. The detection unit 620 may process the first spatial plane image using a method such as the Harris corner point detection method, the Shi-Tomasi corner point detection method, etc., to obtain a plurality of corner points detected on the first spatial plane image.

[0059] After obtaining the detected multiple corner points, the detection unit 620 determines multiple first image blocks on the first spatial plane image based on the multiple corner points, so as to use the multiple first image blocks as multiple templates for template matching with the second spatial plane image. For example, the detection unit 620 can use the detected corner points as the upper left corner, lower left corner, upper right corner or lower right corner vertex of the image block to define the first image block corresponding to each corner point. The detection unit 620 can select a suitable size of the first image block according to the size of the entire first spatial plane image and the size of each sub-area therein, such as 8*8, 16*16 or 32*32 (in pixels), etc., which is not limited here.

[0060] In an example, the detecting unit 620 determining the plurality of first image blocks on the first spatial plane image based on the plurality of corner points may include: determining the plurality of first image blocks by taking each of the plurality of corner points as a center point of an image block. Figure 3 As shown, since the corner points detected by the detection unit 620 are usually located near the junction of each sub-area, the first image block constructed by taking it as the center point of the image block can have more pixel differences, so that a more ideal template matching result can be obtained when it is used as a template for template matching.

[0061] The matching unit 630 performs template matching on each of the plurality of first image blocks and the second spatial plane image to obtain a matched second image block on the second spatial plane image.

[0062] Among them, the matching unit 630 can slide each of the multiple first image blocks as a template on the second spatial plane image, and cut out an image block of the same size as the first image block from the second spatial plane image to perform a pixel comparison operation, so as to find the second image block that is most matched or closest to the first image block on the second spatial plane image.

[0063] Figure 4 A schematic diagram showing accurate matching and inaccurate matching results obtained by performing template matching between a first image block and a second spatial plane image according to an embodiment of the present invention. Figure 4 The first image block 401 and the matched second image block 402 with a less ideal matching result are shown above, while Figure 4 The first image block 403 and the matched second image block 404 with relatively accurate matching results are shown below. That is to say, not all combinations of the first image block and the matched second image block obtained by the matching unit 630 are exactly matched combinations. Therefore, it is necessary to further screen the combinations of the first image blocks and the matched second image blocks, so as to screen out the exactly matched image block combinations to align the first spatial plane image and the second spatial plane image.

[0064] The alignment unit 640 aligns a specific first image block with the matched second image block according to the position difference and / or pixel difference between each first image block and the matched second image block to obtain the aligned first spatial plane image and the second spatial plane image.

[0065] Among them, the alignment unit 640 can screen out an accurately matched image block combination as a specific first image block and a matched second image block based on the position difference and / or pixel difference between each first image block and the matched second image block, thereby aligning the first spatial plane image and the second spatial plane image by aligning the specific first image block and the matched second image block.

[0066] In one example, the alignment unit 640 may use the distance between a specific point on each first image block and a corresponding point on the matched second image block as the position difference. For example, the position difference may include the distance between the center point of each first image block and the center point of the matched second image block. When the first image block is determined by taking the corner point as the center point of the image block, the position difference is equivalent to the distance between each corner point and the center point of the matched second image block. The alignment unit 640 may determine the pixel difference by comparing the pixel categories of the pixels at the corresponding positions of each first image block and the matched second image block. For example, in the case where the pixel category is defined according to the pixel color, the pixel difference may be the number of pixels having different pixel colors at the corresponding positions of each first image block and the matched second image block. In addition, the alignment unit 640 may also perform a pixel comparison operation on each first image block and the matched second image block and obtain an operation result such as the square sum of pixel difference values ​​as the pixel difference, and in this case, the pixel comparison operation result obtained in the template matching process may be directly used to further reduce the amount of operation.

[0067] In one example, the alignment unit 640 may align the first image block with the smallest position difference with the matched second image block. In another example, the alignment unit 640 may also align the first image block with the smallest pixel difference with the matched second image block. In addition, in order to further screen out the most accurately matched image block combination to obtain a better alignment effect, the alignment unit 640 may comprehensively consider both the pixel difference and the position difference. For example, aligning a specific first image block with the matched second image block according to the position difference and / or pixel difference between each first image block and the matched second image block may include: determining the sum of the position difference and pixel difference between each first image block and the matched second image block, and aligning the first image block with the smallest sum of the position difference and pixel difference with the matched second image block.

[0068] The comparison unit 650 compares pixel categories of pixels at corresponding positions of the aligned first spatial plane image and the second spatial plane image to obtain a comparison result of the first spatial plane image and the second spatial plane image.

[0069] Since the alignment unit 640 has obtained the aligned first spatial plane image and the second spatial plane image, the pixels at corresponding positions on the two images can be compared by the comparison unit 650, thereby obtaining a pixel-level spatial plane image comparison result. In one example, since the aligned first spatial plane image and the second spatial plane image may not completely overlap, the comparison unit 650 can select pixels in the overlapped portion of the two aligned images for comparison, and ignore pixels in the non-overlapped portion of the two images.

[0070] In this step, the comparison unit 650 compares the pixel categories of the pixels at the corresponding positions of the first spatial plane image and the second spatial plane image after alignment to obtain the comparison result of the first spatial plane image and the second spatial plane image, which may include: comparing the pixel categories of the pixels at the corresponding positions of the first spatial plane image and the second spatial plane image after alignment pixel by pixel, and taking the number of pixels with the same or different pixel categories as the comparison result of the first spatial plane image and the second spatial plane image. For example, in the case where the pixel category is defined according to the pixel color, the comparison result may be the number of pixels with the same or different pixel colors at the corresponding positions of the first spatial plane image and the second spatial plane image.

[0071] In addition, in one example, the first spatial plane image and the second spatial plane image may not be completely covered by each sub-region, and there may be a blank area composed of transparent pixels (for example, Figure 1). In this case, when determining whether two pixels at corresponding positions of the first spatial plane image and the second spatial plane image have the same pixel category, the comparison unit 650 may first determine whether the two pixels are both pixels of the transparent category: if the two pixels are both pixels of the transparent category, the comparison unit 650 may determine that the two pixels are both located in the blank area and consider that they have the same pixel category; if one of the two pixels is a pixel of the transparent category and the other pixel is not a pixel of the transparent category, the comparison unit 650 may consider that the two pixels have different pixel categories; if the two pixels are not pixels of the transparent category, the comparison unit 650 may determine that the two pixels are not located in the blank area, and further compare the pixel categories (for example, pixel color or pixel gray value, etc.) associated with the sub-area category of the two pixels to finally determine whether the two pixels have the same pixel category.

[0072] Figure 5 FIG. 4 is a schematic diagram showing an aligned first spatial plane image and a second spatial plane image in an overlapping manner according to an embodiment of the present invention. Figure 5 The portion shown in the lightest color in the upper right corner of indicates that the sub-region corresponding to the portion exists only on the first spatial plane image, but is missing on the second spatial plane image. The reason for this missing portion may be that, for example, when a three-dimensional scene is photographed using a camera, some local areas in the scene may be missed, so that the generated second spatial plane image lacks the portion corresponding to the local area and an erroneous blank area appears. In this case, the comparison result with a large difference obtained by comparing the entire first spatial plane image with the second spatial plane image by the comparison unit 650 may not be a true measure of the algorithm used to generate the second spatial plane image.

[0073] Therefore, according to an embodiment of the present invention, the comparison unit 650 may only compare the pixels in the sub-region where the specific first image block is located and the sub-region where the second image block matched according to the specific first image block is located, and the obtained sub-region pixel comparison result is used to measure the algorithm for generating the second spatial plane image. In the case where the first image block is determined by taking the corner point as the center point of the image block, the sub-region where the center point of the specific first image block is located may be the first sub-region, and the sub-region where the center point of the second image block matched according to the specific first image block is located may be the second sub-region, and the comparison unit 650 may compare the pixel categories of the pixels at the corresponding positions of the first sub-region and the second sub-region pixel by pixel to obtain the number of pixels with the same or different pixel categories as the comparison result of the first sub-region and the second sub-region, so as to use the comparison result to measure the algorithm for generating the second spatial plane image. When the specific first image block and the matched second image block cover multiple sub-regions, the comparison unit 650 may arbitrarily select one of the multiple sub-regions or a combination of sub-regions for comparison, which is not limited here.

[0074] In addition, according to another embodiment of the present invention, in addition to comparing the first sub-region and the second sub-region, the comparison unit 650 can also find other corresponding sub-regions for comparison based on the first sub-region and the second sub-region using the sub-region connection relationship. For example, the acquisition unit 610 can also acquire the sub-region connection relationship of the first spatial plane image or the second spatial plane image, and the comparison unit 650 can also determine the third sub-region on the first spatial plane image and the fourth sub-region on the second spatial plane image based on the sub-region connection relationship and the first sub-region and the second sub-region, respectively, and compare the pixel categories of the pixels at the corresponding positions of the third sub-region and the fourth sub-region pixel by pixel to obtain the number of pixels with the same or different pixel categories as the comparison result of the third sub-region and the fourth sub-region.

[0075] In this embodiment, the sub-region connection relationship may reflect the positional relationship between each interconnected sub-region. For example, the sub-region connection relationship may record in a list the other sub-regions to which each sub-region is connected in the four directions of top, bottom, left and right. Therefore, in one example, the comparison unit 650 may, based on the sub-region connection relationship, take a sub-region connected to the first sub-region in a specific direction as the third sub-region, and take a sub-region connected to the second sub-region in the same direction as the fourth sub-region, thereby determining the corresponding third sub-region and fourth sub-region for comparison.

[0076] In a specific application scenario in which the image generation algorithm is improved by using the comparison result, the first spatial plane image and the second spatial plane image may be associated with the same three-dimensional space, wherein the first spatial plane image may be a training image obtained by pre-marking a plane image of the three-dimensional space, and the second spatial plane image may be an image output by inputting a panoramic image of the three-dimensional space into a neural network for generating spatial plane images. In this case, the comparison result may be used by the training unit 660 to construct a loss function for training the neural network, thereby training the neural network using the loss function to adjust the parameters of the neural network, and using the parameters when the loss function converges as the adjusted parameters.

[0077] In this application scenario, the training unit 660 can construct the loss function based on the comparison result of the aligned entire first spatial plane image and the second spatial plane image. In addition, as described above, since the comparison result based on the entire first spatial plane image and the second spatial plane image may not be a good measure of the algorithm used to generate the second spatial plane image, the training unit 660 can also construct the loss function based on the comparison result of one or more corresponding sub-regions. The type of loss function can be a cross entropy loss function or a Smooth L1 loss function, but other types of loss functions can also be used, which are not limited here.

[0078] According to the above-mentioned spatial plane image comparison device of the present invention, it is possible to align two spatial plane images by using corner point detection and template matching and compare each pixel on the image, thereby obtaining a spatial plane image comparison result at the pixel level. This spatial plane image comparison device can obtain a more accurate comparison result at the pixel level, and it is convenient to use the comparison result to improve the image generation algorithm to improve the generation quality of the spatial plane image.

[0079] Below, refer to Figure 7 The spatial plane image comparison device according to an embodiment of the present invention is described below. The spatial plane image may include a plurality of non-overlapping sub-regions and each sub-region category may be associated with a corresponding pixel category. Figure 7 FIG. 7 shows a block diagram of a spatial plane image comparison device 700 according to an embodiment of the present invention. Figure 7 As shown, the device 700 may be a computer or a server.

[0080] like Figure 7As shown, the spatial plane image comparison device 700 includes one or more processors 710 and a memory 720. Of course, in addition to this, the spatial plane image comparison device 700 may also include a multi-sensor imaging system and an output device (not shown), etc. These components can be interconnected through a bus system and / or other forms of connection mechanisms. It should be noted that Figure 7 The components and structure of the spatial plane image comparison device 700 shown are merely exemplary and not restrictive. The spatial plane image comparison device 700 may also have other components and structures as required.

[0081] The processor 710 can be a central processing unit (CPU) or other form of processing unit with data processing capability and / or instruction execution capability, and can use computer program instructions stored in the memory 720 to perform desired functions, which may include: acquiring a first spatial plane image and a second spatial plane image, wherein the first spatial plane image and the second spatial plane image are associated with the same three-dimensional space and have the same resolution; performing corner point detection on the first spatial plane image to obtain multiple corner points, and determining multiple first image blocks on the first spatial plane image based on the multiple corner points; performing template matching on each of the multiple first image blocks with the second spatial plane image to obtain a matched second image block on the second spatial plane image; aligning a specific first image block with the matched second image block according to the position difference and / or pixel difference between each first image block and the matched second image block to obtain the aligned first spatial plane image and the second spatial plane image; and comparing the pixel categories of pixels at corresponding positions of the aligned first spatial plane image and the second spatial plane image to obtain a comparison result of the first spatial plane image and the second spatial plane image.

[0082] The memory 720 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 710 may run the program instructions to implement the functions of the image processing device of the embodiment of the present invention described above and / or other desired functions, and / or may execute the image processing method according to the embodiment of the present invention. Various applications and various data may also be stored in the computer-readable storage medium.

[0083] The following describes a computer-readable storage medium according to an embodiment of the present invention, on which computer program instructions are stored, wherein the computer program instructions implement the spatial plane image comparison method as described above when executed by a processor.

[0084] Of course, the above-mentioned specific embodiments are merely examples rather than limitations, and those skilled in the art can, based on the concept of the present invention, merge and combine some steps and devices from the various embodiments described separately above to achieve the effects of the present invention. Such merged and combined embodiments are also included in the present invention, and such merges and combinations are not described one by one herein.

[0085] Note that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above invention are only for the purpose of illustration and facilitating understanding, not limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0086] The block diagrams of the devices, apparatuses, equipment, and systems involved in the present invention are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The word "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0087] The step flow charts and the above method descriptions in the present invention are only illustrative examples and are not intended to require or imply that the steps of each embodiment must be performed in the order given. As will be appreciated by those skilled in the art, the order of the steps in the above embodiments can be performed in any order. Words such as "thereafter", "then", "next", etc. are not intended to limit the order of the steps; these words are only used to guide the reader through the description of these methods. In addition, any reference to an element in the singular, such as using "one", "one", or "the" is not to be construed as limiting the element to the singular.

[0088] In addition, the steps and devices in the various embodiments of this document are not limited to being implemented in a certain embodiment. In fact, based on the concept of the present invention, relevant partial steps and partial devices in the various embodiments of this document can be combined to conceive new embodiments, and these new embodiments are also included in the scope of the present invention.

[0089] Each operation of the method described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware and / or software components and / or modules, including but not limited to circuits, application specific integrated circuits (ASICs) or processors.

[0090] The various illustrated logic blocks, modules and circuits described may be implemented or performed using a general purpose processor, digital signal processor (DSP), ASIC, field programmable gate array signal (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but as an alternative, the processor may be any commercially available processor, controller, microcontroller or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0091] The steps of the method or algorithm described in conjunction with the present invention can be directly embedded in hardware, in a software module executed by a processor, or in a combination of the two. A software module can exist in any form of tangible storage medium. Some examples of storage media that can be used include random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, etc. A storage medium can be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. In an alternative manner, the storage medium can be integral with the processor. A software module can be a single instruction or many instructions, and can be distributed on several different code segments, between different programs, and across multiple storage media.

[0092] The method invented herein includes one or more actions for implementing the described method. The method and / or action can be interchangeable with each other without departing from the scope of the claims. In other words, unless a specific order of actions is specified, the order and / or use of specific actions can be modified without departing from the scope of the claims.

[0093] The functions described can be implemented by hardware, software, firmware or any combination thereof. If implemented in software, the functions can be stored as one or more instructions on a tangible computer-readable medium. The storage medium can be any available tangible medium accessed by a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device or any other tangible medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer. As used herein, a disc includes a compact disc (CD), a laser disc, an optical disc, a digital versatile disc (DVD), a floppy disk and a blue disc.

[0094] Thus, a computer program product may perform the operations presented herein. For example, such a computer program product may be a computer-readable tangible medium having instructions tangibly stored (and / or encoded) thereon, which instructions may be executed by one or more processors to perform the operations described herein. The computer program product may include packaging materials.

[0095] Software or instructions may also be transmitted via a transmission medium. For example, the software may be transmitted from a website, server or other remote source using a transmission medium such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio or microwave.

[0096] In addition, the module and / or other appropriate means for performing the methods and techniques described herein can be downloaded and / or otherwise obtained by the user terminal and / or base station when appropriate. For example, such a device can be coupled to a server to facilitate the transmission of the means for performing the methods described herein. Alternatively, the various methods described herein can be provided via a storage component (e.g., RAM, ROM, a physical storage medium such as a CD or a floppy disk, etc.) so that the user terminal and / or base station can obtain the various methods when being coupled to the device or providing a storage component to the device. In addition, any other appropriate technology for providing the methods and techniques described herein to a device can be utilized.

[0097] Other examples and implementations are within the scope and spirit of the present invention and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hard wiring, or any combination of these. Features that implement the functions can also be physically located in various locations, including being distributed so that parts of the functions are implemented at different physical locations. Moreover, as used herein, including as used in the claims, "or" used in the enumeration of items beginning with "at least one" indicates a separate enumeration, so that, for example, the enumeration of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). In addition, the wording "exemplary" does not mean that the example described is preferred or better than other examples.

[0098] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present invention is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0099] The above description of the invented aspects is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present invention. Therefore, the present invention is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features of the present invention.

[0100] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present invention to the form invented herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for comparing a spatial plane image, wherein the spatial plane image comprises a plurality of non-overlapping sub-regions and each sub-region category is associated with a corresponding pixel category, the method comprising: Acquire a first spatial plane image and a second spatial plane image, wherein the first spatial plane image and the second spatial plane image have the same resolution; Performing corner point detection on the first spatial plane image to obtain a plurality of corner points, and determining a plurality of first image blocks on the first spatial plane image based on each of the plurality of corner points as a vertex or a center point of a first image block and in combination with a size of the first image block; Performing template matching on each of the plurality of first image blocks and the second spatial plane image to obtain a matched second image block on the second spatial plane image; Aligning a specific first image block with the matched second image block according to a position difference and / or a pixel difference between each first image block and the matched second image block, so as to obtain the aligned first spatial plane image and the second spatial plane image, wherein the position difference represents a distance between a specific point on each first image block and a corresponding point on the matched second image block, and the pixel difference is obtained by comparing pixel categories of pixels at corresponding positions of each first image block and the matched second image block, or by performing a pixel comparison operation on each first image block and the matched second image block; as well as Comparing pixel categories of pixels at corresponding positions of the aligned first spatial plane image and the second spatial plane image to obtain a comparison result of the first spatial plane image and the second spatial plane image; Wherein, aligning a specific first image block with a matched second image block according to a position difference and / or a pixel difference between each first image block and a matched second image block comprises: determining a sum of a position difference and / or a pixel difference between each first image block and a matched second image block, and aligning a first image block having a smallest sum of a position difference and / or a pixel difference with the matched second image block.

2. The method of claim 1, wherein: Determine the plurality of first image blocks by taking each of the plurality of corner points as a center point of the first image block, wherein the position difference comprises a distance between the center point of each first image block and a center point of the matched second image block; The aligning of a specific first image block with the matched second image block according to the position difference and / or pixel difference between each first image block and the matched second image block comprises: determining the sum of the position difference and the pixel difference between each first image block and the matched second image block, and aligning the first image block having the smallest sum of the position difference and the pixel difference with the matched second image block.

3. The method according to claim 1 or 2, wherein: The comparing the pixel categories of pixels at corresponding positions of the aligned first spatial plane image and the second spatial plane image to obtain a comparison result of the first spatial plane image and the second spatial plane image includes: The pixel categories of pixels at corresponding positions of the aligned first spatial plane image and the second spatial plane image are compared pixel by pixel, and the number of pixels with the same or different pixel categories is used as the comparison result of the first spatial plane image and the second spatial plane image.

4. The method according to claim 1 or 2, wherein: The plurality of first image blocks are determined by taking each of the plurality of corner points as a center point of the first image block, the subregion where the center point of the specific first image block is located is the first subregion, and the subregion where the center point of the second image block matched according to the specific first image block is located is the second subregion, and the method further includes: The pixel categories of the pixels at corresponding positions of the first sub-region and the second sub-region are compared pixel by pixel to obtain the number of pixels having the same or different pixel categories as the comparison result of the first sub-region and the second sub-region.

5. The method of claim 4, wherein: The acquiring of the first spatial plane image and the second spatial plane image includes acquiring a sub-region connection relationship of the first spatial plane image or the second spatial plane image, and the method further includes: Based on the sub-region connection relationship and the first sub-region and the second sub-region, a third sub-region on the first spatial plane image and a fourth sub-region on the second spatial plane image are respectively determined, and the pixel categories of the pixels at corresponding positions of the third sub-region and the fourth sub-region are compared pixel by pixel to obtain the number of pixels with the same or different pixel categories as the comparison result of the third sub-region and the fourth sub-region.

6. The method according to claim 1 or 2, wherein: The first spatial plane image and the second spatial plane image are associated with the same three-dimensional space, the first spatial plane image is a training image obtained by pre-labeling a plane image of the three-dimensional space, and the second spatial plane image is an image output by inputting a panoramic image of the three-dimensional space into a neural network for generating spatial plane images. The comparison result is used to construct a loss function for training the neural network, and the method further includes training the neural network using the loss function to adjust the parameters of the neural network.

7. A spatial plane image comparison device, wherein the spatial plane image includes a plurality of non-overlapping sub-regions and each sub-region category is associated with a corresponding pixel category, the device comprising: an acquisition unit configured to acquire a first spatial plane image and a second spatial plane image, wherein the first spatial plane image and the second spatial plane image are associated with the same three-dimensional space and have the same resolution; a detection unit configured to perform corner point detection on the first spatial plane image to obtain a plurality of corner points, and determine a plurality of first image blocks on the first spatial plane image based on each of the plurality of corner points as a vertex or a center point of a first image block and in combination with a size of the first image block; a matching unit configured to perform template matching on each of the plurality of first image blocks and the second spatial plane image to obtain a matched second image block on the second spatial plane image; an alignment unit, configured to align a specific first image block with a matched second image block according to a position difference and / or a pixel difference between each first image block and a matched second image block, so as to obtain the aligned first spatial plane image and the second spatial plane image, wherein the position difference represents a distance between a specific point on each first image block and a corresponding point on the matched second image block, and the pixel difference is obtained by comparing pixel categories of pixels at corresponding positions of each first image block and the matched second image block, or by performing a pixel comparison operation on each first image block and the matched second image block; as well as a comparing unit configured to compare pixel categories of pixels at corresponding positions of the aligned first spatial plane image and the second spatial plane image to obtain a comparison result of the first spatial plane image and the second spatial plane image; The alignment unit determines the sum of position differences and / or pixel differences between each first image block and the matched second image block, and aligns the first image block having the smallest sum of position differences and / or pixel differences with the matched second image block.

8. A spatial plane image comparison device, wherein the spatial plane image comprises a plurality of non-overlapping sub-regions and each sub-region category is associated with a corresponding pixel category, the device comprising: processor; and a memory having computer program instructions stored therein, Wherein, when the computer program instructions are executed by the processor, the processor is caused to perform the following steps: Acquire a first spatial plane image and a second spatial plane image, wherein the first spatial plane image and the second spatial plane image are associated with the same three-dimensional space and have the same resolution; Performing corner point detection on the first spatial plane image to obtain a plurality of corner points, and determining a plurality of first image blocks on the first spatial plane image based on each of the plurality of corner points as a vertex or a center point of a first image block and in combination with a size of the first image block; Performing template matching on each of the plurality of first image blocks and the second spatial plane image to obtain a matched second image block on the second spatial plane image; Aligning a specific first image block with the matched second image block according to a position difference and / or a pixel difference between each first image block and the matched second image block to obtain the aligned first spatial plane image and the second spatial plane image, wherein the position difference represents a distance between a specific point on each first image block and a corresponding point on the matched second image block, and the pixel difference is obtained by comparing pixel categories of pixels at corresponding positions of each first image block and the matched second image block, or by performing a pixel comparison operation on each first image block and the matched second image block; and Comparing pixel categories of pixels at corresponding positions of the aligned first spatial plane image and the second spatial plane image to obtain a comparison result of the first spatial plane image and the second spatial plane image; The step of aligning a specific first image block with a matched second image block according to a position difference and / or a pixel difference between each first image block and a matched second image block comprises: The position difference and / or the sum of pixel differences between each first image block and the matched second image block is determined, and the first image block having the smallest position difference and / or the sum of pixel differences is aligned with the matched second image block.

9. A computer-readable storage medium having computer program instructions stored thereon, wherein: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images