An image registration method, system, apparatus and medium suitable for photographing documents

Abstract

The application discloses a kind of image registration method, system, device and medium suitable for photographing document, wherein method includes: obtaining photographing document image, the non-rigid pre-alignment is carried out to photographing document image, and pre-alignment document image is obtained;Clean document image is obtained, and the layered alignment is carried out to clean document image and pre-alignment document image, and multi-scale feature map and offset field are obtained;The fine-grained cycle refinement is carried out to the multi-scale feature map and offset field, and the offset field of the same scale size with the pre-alignment document image is obtained.The application can process the photographing document image with different interference factors, including paper twist, paper wrinkle, shooting illumination change or shadow change condition.The application can be widely applied in pattern recognition and artificial intelligence technical field.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pattern recognition and artificial intelligence, and particularly relates to an image registration method, system and device suitable for photographed documents and a medium. BACKGROUND

[0002] The purpose of the image registration task is to match and align two or more images acquired at different times, by different imaging devices, or under different conditions. This technology has important applications in the fields of medical images and remote sensing, and can achieve automatic medical diagnosis and remote sensing image stitching. In recent years, image registration has been widely studied in natural images, which can be divided into optical flow correspondence, geometric correspondence and semantic correspondence according to the differences between paired images. However, there is less research on document images. Document image registration is more related to geometric correspondence, which can achieve automatic information extraction of documents and thus realize office automation. Although people have made in-depth research on natural image registration and have made great progress, these methods are designed based on the characteristics of natural images. Compared with natural images, document images have a large number of repetitive patterns, significant global misalignment introduced by unconstrained capture environment, the need for high accuracy and the ability to handle large changes, and the basic elements (i.e. characters and words) have a more fine-grained structure than natural images. The above difficulties make the existing methods ineffective for document images. SUMMARY

[0003] To at least partially solve one of the technical problems existing in the prior art, the purpose of the present application is to provide an image registration method, system, device and medium suitable for photographed documents.

[0004] The technical solution adopted by the present application is as follows:

[0005] An image registration method suitable for photographed documents, comprising the following steps:

[0006] obtaining a photographed document image, performing non-rigid pre-alignment on the photographed document image, and obtaining a pre-aligned document image;

[0007] obtaining a clean document image, performing hierarchical alignment on the clean document image and the pre-aligned document image, and obtaining a multi-scale feature map and an offset field;

[0008] performing fine-grained cyclic refinement on the multi-scale feature map and the offset field to obtain an offset field with the same scale size as the pre-aligned document image.

[0009] Further, the photographed document image is a document image obtained by a shooting device. Due to the influence of the shooting device and the shooting environment, there are usually paper distortion, wrinkles, light shadow changes and other interference factors in the photographed document image, which are all normal situations of the photographed document image.

[0010] Further, the non-rigid pre-alignment of the photographed document image to obtain a pre-aligned document image comprises:

[0011] Edge extraction is performed on the photographed document image to distinguish the foreground document region and the environment boundary region in the photographed document image, and a mask image of the foreground document region is obtained.

[0012] Control point detection is performed on the mask image to obtain four corner points and a series of equidistant points on four edges.

[0013] The four corner points and the series of equidistant points on the four edges are mapped to reference points of a quadrilateral to obtain a mapping point pair.

[0014] Thin plate spline (TPS) non-rigid transformation is performed on the mapping point pair to obtain a pre-aligned document image.

[0015] Further, the clean document image is an image obtained by converting a digital format or an image obtained by scanning a scanning device. The image is usually regular and clean and does not have interference such as distortion, wrinkle, shadow, etc.

[0016] Further, the layered alignment of the clean document image and the pre-aligned document image to obtain a multi-scale feature map and an offset field comprises:

[0017] The clean document image is input into a preset backbone network to obtain a multi-scale feature map

[0018] The pre-aligned document image is input into a preset backbone network to obtain a multi-scale feature map

[0019] The first three layers of features in the multi-scale feature maps X t and X s are iteratively aligned to obtain an offset field.

[0020] Further, the iterative alignment of the first three layers of features in the multi-scale feature maps X t and X s to obtain an offset field comprises:

[0021] The l-1th layer of multi-scale features is up-sampled and added to the lth layer of multi-scale feature maps to obtain the lth layer of multi-scale feature maps

[0022] A correlation layer is used to calculate the matching loss between the multi-scale feature maps X and X ;

[0023] The offset field between the lth layer multi-scale feature maps is calculated by matching loss and a CNN decoder.

[0024] Further, the multi-scale feature maps and the offset field are refined in a fine-grained cycle, and an offset field with the same scale size as the pre-aligned document image is obtained.

[0025] The third layer offset field is subjected to a 2-fold upsampling operation to obtain an offset field enlarged by 2 times;

[0026] The third layer offset field is subjected to an 8-fold upsampling operation to obtain an offset field enlarged by 8 times;

[0027] The fourth layer multi-scale feature maps, the offset field enlarged by 2 times and the offset field enlarged by 8 times are refined in a cycle using a ConvGRU unit, and an offset field with the same scale size as the pre-aligned document image is obtained.

[0028] Another technical solution adopted by the application is:

[0029] An image registration system suitable for a photographed document, comprising:

[0030] A non-rigid pre-alignment module is configured to obtain a photographed document image, and perform non-rigid pre-alignment on the photographed document image to obtain a pre-aligned document image.

[0031] A hierarchical alignment module is configured to obtain a clean document image, and perform hierarchical alignment on the clean document image and the pre-aligned document image to obtain multi-scale feature maps and an offset field.

[0032] A fine-grained cycle refinement module is configured to refine the multi-scale feature maps and the offset field in a fine-grained cycle to obtain an offset field with the same scale size as the pre-aligned document image.

[0033] Another technical solution adopted by the application is:

[0034] An image registration device suitable for a photographed document, comprising:

[0035] At least one processor;

[0036] At least one memory configured to store at least one program;

[0037] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0038] Another technical solution adopted by the application is:

[0039] A computer-readable storage medium having stored therein a program executable by a processor for performing the method as described above when executed by the processor.

[0040] The present application has the advantage that it can handle photographed document images with different interference factors, including paper distortion, paper creases, changes in shooting light or shadow. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following introduces the drawings of the related technical solutions in the embodiments of the present application or the prior art. It should be understood that the drawings in the following introduction are only for the convenience of clearly and simply describing part of the embodiments of the technical solutions of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0042] Figure 1 is a general flowchart of an image registration method suitable for photographed documents in an embodiment of the present application;

[0043] Figure 2 is a structure flowchart of a fine-grained cycle refinement module in an embodiment of the present application;

[0044] Figure 3 is the effect of document image rectification using the offset field obtained by the method of the present application in an embodiment of the present application;

[0045] Figure 4 is a step flowchart of an image registration method suitable for photographed documents in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application. For the step numbers in the following embodiments, they are only set for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0047] In the description of the present application, it should be understood that the orientation description, such as the orientation or position relationship indicated by up, down, front, back, left, right, etc. is based on the orientation or position relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0048] In the description of the present application, the meaning of one or more is one or more, the meaning of multiple is more than two, greater than, less than, more than, etc. are understood as not including the number, above, below, within, etc. are understood as including the number. If it is described as first, second, it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or implicitly indicating the sequence of indicated technical features.

[0049] In the description of the present application, unless otherwise explicitly limited, the words such as setting, installing, connecting, etc. should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution.

[0050] As shown in Figure 4 The embodiment provides an image registration method suitable for a photographed document. The method can perform image registration alignment between a photographed document image and a clean document image, obtain an offset field, and thus can establish a pixel-to-pixel mapping relationship between the same document in a photographed image and a clean image scene. The method has great significance for document layout analysis and document image correction. The method specifically includes the following steps.

[0051] S1, a photographed document image is obtained, and a non-rigid pre-alignment is performed on the photographed document image to obtain a pre-aligned document image.

[0052] In the embodiment, the photographed document image can be a document image obtained by a shooting device (for example, a smart terminal, a video camera, etc.). Due to the influence of the shooting device and the shooting environment, there are usually paper distortion, wrinkles, light and shadow changes and other interference factors in the photographed document image, which are all normal situations of the photographed document image.

[0053] Specifically, as shown in Figure 1 The left non-rigid pre-alignment module, step S1 includes steps S11-S14:

[0054] S11, a deep learning-based semantic segmentation method is used to classify the pixel points of the photographed document image, and then the edges of the photographed document are extracted, the foreground document region and the environment boundary region in the photographed document image are distinguished, and the mask graph of the foreground document region is obtained;

[0055] S12, a polygon fitting algorithm is used to extract four corner points and a minimum circumscribed rectangle of the document on the mask graph of the foreground document region, and then a vertical bisector is set on the minimum circumscribed rectangle according to a preset equal division ratio, so that the bisector intersects with the mask graph boundary to obtain a series of equidistant points on the mask Figure Four border;

[0056] S13. Map the four corner points and a series of equidistant points on the four sides of the quadrilateral to the four foci and a series of reference points on the four sides of the quadrilateral to obtain the mapped point pairs;

[0057] S14. Perform a non-rigid transformation based on TPS on the mapping point pair to obtain a pre-aligned document image.

[0058] S2. Obtain a clean document image, perform layered alignment on the clean document image and the pre-aligned document image to obtain a multi-scale feature map and offset field.

[0059] In this embodiment, the clean document image can be an image converted from digital formats such as Word and PDF, or an image obtained by scanning with a scanning device. The image is usually relatively neat and clean and does not have interference such as distortion, wrinkles, or shadows.

[0060] Specifically, such as Figure 1 As shown in the intermediate layer alignment module, step S2 includes steps S21-S23:

[0061] S21. Use a convolutional neural network-based backbone network to extract features from clean document images. Multi-scale feature maps of clean document images can be obtained from network layers of different depths.

[0062] S22. Use a convolutional neural network-based backbone network to extract features from the pre-aligned document image. Multi-scale feature maps of the pre-aligned document image can be obtained from network layers of different depths.

[0063] S23, such as Figure 1 As shown, steps S21 and S22 can obtain four layers of feature maps at different scales for the clean document image and the pre-aligned document image, respectively. For the multi-scale feature map X... t and X s The first three layers of features are iteratively aligned. First, the (l-1)th layer multi-scale features are upsampled by a factor of two and added to the l-th layer multi-scale feature map to obtain the l-th layer multi-scale feature map. Then, a multi-scale feature map is computed through a global / local correlation layer. and The matching loss between them is calculated; finally, the offset field between the multi-scale features of the l-th layer is calculated by the matching loss and the CNN decoder.

[0064] S3. Perform fine-grained cyclic refinement on the multi-scale feature map and offset field to obtain an offset field of the same scale as the pre-aligned document image.

[0065] The step S23 iteratively aligns the first three layers of the multi-scale feature map, and an offset field of the third layer of the multi-scale feature map is obtained by iteration. The width and height of the offset field are 1 / 8 of the width and height of the pre-aligned document image. The step S3 performs fine-grained recurrent refinement on the multi-scale feature obtained in the step S2 and the offset field, so as to obtain an output offset field with the same width and height as the pre-aligned document image.

[0066] Specifically, as Figure 1 the fine-grained recurrent refinement module in the step S3 and Figure 2 The structure of the module is shown in a flowchart, and the step S3 includes steps S31-S33.

[0067] S31, performing a 2-fold upsampling operation on the offset field of the third layer of the multi-scale feature map obtained in the step S2. The upsampling operation is bilinear interpolation, and a 2-fold enlarged offset field is obtained.

[0068] S32, performing an 8-fold upsampling operation on the offset field of the third layer of the multi-scale feature map obtained in the step S2. The upsampling operation is bilinear interpolation, and an 8-fold enlarged offset field is obtained.

[0069] S33, using a ConvGRU unit to perform recurrent refinement on the fourth layer of the multi-scale feature map and the 2-fold enlarged offset field and the 8-fold enlarged offset field, and a total of n iterations are performed to obtain an offset field with the same size as the pre-aligned document image.

[0070] Therefore, by the steps S1-S3, image registration can be performed in the case of a given pair of clean document images and photographed document images, and an offset field between the two is obtained. The offset field can establish a pixel-to-pixel mapping relationship between the pair of clean document images and the photographed document images. By using the obtained offset field, document image correction can be performed. The document image correction effect is shown in Figure 3 As shown in the figure, the method provided in the embodiment can achieve good correction effect in the case of distortion and wrinkle interference in the photographed document image.

[0071] In some optional embodiments, the multi-scale feature map and the offset field of the pre-aligned document image and the clean document image are obtained by using a deep neural network in the steps S2 and S3, and the network parameters are pre-trained and optimized by using synthetic data and self-supervised trained and optimized by using real data. The network parameter optimization specifically includes:

[0072] (1) Data acquisition: PDF files are collected on the Internet and converted into clean document images, and corresponding synthetic photographed document images and corresponding offset fields are obtained by using data synthesis technology. 10,000 training data samples and 2,000 verification data samples are included.

[0073] (2) Network training:

[0074] (2-1) Constructing a deep neural network: using a ResNet-18 model as the backbone network, and using a ConvGRU unit as the basic unit of the fine-grained recurrent refinement module.

[0075] (2-2) Training method: training uses a gradient descent algorithm, which calculates the gradient from the last layer, layer by layer, updates all parameters, and achieves the purpose of training the network. The loss function during training is L1 loss.

[0076] (2-3) Setting of training parameters:

[0077] Number of iterations: 100 epochs

[0078] Optimizer: Adam

[0079] Learning rate: 0.0005 (learning rate update strategy: every 30 iterations, the learning rate is reduced to 0.3 of the original)

[0080] Weight decay: 0.0005

[0081] (2-4) The backbone network is initialized using the pre-trained weights on the ImageNet dataset, and the remaining modules are initialized using random initialization parameters, and then the deep neural network is trained.

[0082] The embodiment also provides an image registration system suitable for a photographed document, comprising:

[0083] A non-rigid pre-alignment module is configured to obtain a photographed document image, perform non-rigid pre-alignment on the photographed document image, and obtain a pre-aligned document image.

[0084] A hierarchical alignment module is configured to obtain a clean document image, perform hierarchical alignment on the clean document image and the pre-aligned document image, and obtain a multi-scale feature map and an offset field.

[0085] A fine-grained recurrent refinement module is configured to perform fine-grained recurrent refinement on the multi-scale feature map and the offset field, and obtain an offset field with the same scale size as the pre-aligned document image.

[0086] The image registration system suitable for the photographed document provided in the embodiment can execute the image registration method suitable for the photographed document provided in the method embodiment, and can execute any combination of the steps of the method embodiment, and has the functions and advantages of the method.

[0087] The embodiment also provides an image registration device suitable for a photographed document, comprising:

[0088] at least one processor;

[0089] at least one memory for storing at least one program;

[0090] when the at least one program is executed by the at least one processor, the at least one processor implements Figure 4 the method.

[0091] The image registration device for photographing documents provided by the embodiment can execute the image registration method for photographing documents provided by the method embodiment of the application, can execute the implementation steps of any combination of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0092] The embodiment of the application further discloses a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 4 the method.

[0093] The embodiment further provides a storage medium storing instructions or programs that can execute the document image geometric correction method provided by the method embodiment of the application. When the instructions or programs are executed, implementation steps of any combination of the method embodiment can be executed, and the corresponding functions and beneficial effects of the method are possessed.

[0094] In some alternative embodiments, the functions / operations mentioned in the block diagram can not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, two blocks shown in succession can actually be executed substantially simultaneously or the blocks can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flowcharts of the application are provided by way of example, with the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of larger operations are independently executed.

[0095] Furthermore, although the present application is described in the context of functional modules, it is to be understood that one or more of the described functions and / or features can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It will also be appreciated that detailed discussion of the actual implementation of each module is not necessary to an understanding of the application. Rather, the actual implementation of the modules, in combination with their attributes, functions, and internal relationships, are to be understood within the context of the devices disclosed herein. Thus, those skilled in the art with access to patents, scientific journals, and other public sources known by those skilled in the art will be able, using ordinary skill, to practice the application as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are merely illustrative and that the scope of the present application is to be determined by the entire scope of the claims, along with all equivalents of the claims and their equivalents.

[0096] If the functions are implemented in software, the functions can be stored in or implemented as one or more computer program products, which can be incorporated into a computer- readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a "computer-readable medium" can be any means that can store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0097] The logic and / or steps represented in the flowcharts and / or otherwise described herein, for example, can be embodied in non-transitory computer-readable media, which can be executed by an instruction execution system, apparatus, or device such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a "computer-readable medium" can be any means that can store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0098] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

[0099] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented in software or firmware which is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following techniques, which are well known in the art, can be used alone or in combination to implement the application: a discrete logic circuit having logic gates for implementing logic functions upon data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), and / or the like.

[0100] In the above-description of various embodiments of the present application, no distinction is made between the use of the terms "about" "substantially" "approximately" "generally" "premi¬sely" or "essentially" to describe the application. It should be understood that every embodiment can employ any one of these terms, and the use of terms, possibly even within a single embodiment, can be interchanged with each other without departing from the spirit of the present application.

[0101] While the embodiments of the present application have been shown and described with reference to various embodiments / versions thereof, it will be understood that various additions, modifications and substitutions can be made by those skilled in the art to adapt the method and system of the present application to particular situations without departing from the spirit and scope of the present application as defined by the claims in appreciation of this disclosure.

[0102] The above description is that of the preferred embodiments of the present application. Various modifications and changes can be made thereto without departing from the spirit and scope of the application, which is defined by the appended claims. Accordingly, the disclosure of the present application is intended to be illustrative, but not limiting, of the scope of the application, which is set forth in the following claims.

Claims

1. An image registration method suitable for photographed documents, characterized in that, Includes the following steps: Acquire a photographed document image, and perform non-rigid pre-alignment on the photographed document image to obtain a pre-aligned document image; A clean document image is obtained, and the clean document image and the pre-aligned document image are layered and aligned to obtain multi-scale feature maps and offset fields. The multi-scale feature map and offset field are refined in a fine-grained cyclic manner to obtain an offset field of the same scale as the pre-aligned document image; The step of performing layered alignment on the clean document image and the pre-aligned document image to obtain multi-scale feature maps and offset fields includes: The clean document image is fed into a preset backbone network to obtain multi-scale feature maps. ; The pre-aligned document image is fed into a preset backbone network to obtain multi-scale feature maps. ; For the multi-scale feature map and The first three layers of features are iteratively aligned to obtain the offset field; The multi-scale feature map and The first three layers of features are iteratively aligned to obtain the offset field, including: For the first l-1 Upsampling operation is performed on multi-scale features of layer 1, and then compared with the first layer. l By adding the feature maps of the multi-scale layers, the first layer is obtained. l Multi-scale feature map ; The multi-scale feature map is calculated through a correlation layer. and Matching loss between them; The first step is obtained by calculating the matching loss and the CNN decoder. l Offset fields between multi-scale features of layers.

2. The image registration method for photographed documents according to claim 1, characterized in that, The photographed document image is a document image obtained by taking a picture with a camera.

3. The image registration method for photographed documents according to claim 1, characterized in that, The step of performing non-rigid pre-alignment on the photographed document image to obtain a pre-aligned document image includes: Edge extraction is performed on the photographed document image to distinguish between the foreground document region and the environment boundary region in the photographed document image, and a mask image of the foreground document region is obtained. Control point detection is performed on the mask image to obtain four corner points and multiple equidistant points on the four sides; Map the four corner points and multiple equidistant points on the four sides to the reference point of the quadrilateral to obtain the mapping point pairs; A non-rigid transformation with thin-plate spline correction is performed on the mapping point pair to obtain a pre-aligned document image.

4. The image registration method for photographed documents according to claim 1, characterized in that, The clean document image is an image obtained by converting a digital format or by scanning with a scanning device.

5. The image registration method for photographed documents according to claim 1, characterized in that, The step of performing fine-grained cyclic refinement of the multi-scale feature map and offset field to obtain an offset field of the same scale as the pre-aligned document image includes: The third layer of the offset field is upsampled by 2 times to obtain an offset field magnified by 2 times; The third layer offset field is upsampled by 8 times to obtain an offset field magnified by 8 times; The ConvGRU unit is used to iteratively refine the fourth-layer multi-scale feature map, the offset field magnified by 2x, and the offset field magnified by 8x to obtain an offset field of the same scale as the pre-aligned document image.

6. An image registration system suitable for photographed documents, characterized in that, include: The non-rigid pre-alignment module is used to acquire a photographed document image, perform non-rigid pre-alignment on the photographed document image, and obtain a pre-aligned document image. The layer alignment module is used to acquire a clean document image, perform layer alignment on the clean document image and the pre-aligned document image, and obtain multi-scale feature maps and offset fields. The fine-grained cyclic refinement module is used to perform fine-grained cyclic refinement on the multi-scale feature map and offset field to obtain an offset field of the same scale as the pre-aligned document image. The step of performing layered alignment on the clean document image and the pre-aligned document image to obtain multi-scale feature maps and offset fields includes: The clean document image is fed into a preset backbone network to obtain multi-scale feature maps. ; The pre-aligned document image is fed into a preset backbone network to obtain multi-scale feature maps. ; For the multi-scale feature map and The first three layers of features are iteratively aligned to obtain the offset field; The multi-scale feature map and The first three layers of features are iteratively aligned to obtain the offset field, including: For the first l-1 Upsampling operation is performed on multi-scale features of layer 1, and then compared with the first layer. l By adding the feature maps of the multi-scale layers, the first layer is obtained. l Multi-scale feature map ; The multi-scale feature map is calculated through a correlation layer. and Matching loss between them; The first step is obtained by calculating the matching loss and the CNN decoder. l Offset fields between multi-scale features of layers.

7. An image registration device suitable for photographed documents, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method of any one of claims 1-5.

8. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform the method as described in any one of claims 1-5.

Images