Apparatus and method for automated precision alignment of multi-source optical imagery

Abstract

The multi-source image automatic alignment device of the present invention is characterized by comprising: a memory; and a processor that extracts feature data from each of a plurality of images, aligns the plurality of images based on the size and orientation of the feature data, and extracts corresponding points from the aligned plurality of images to set as ground reference points.

Description

Technology Field

[0001] The present invention relates to a multi-source image automatic alignment device and method. Background Technology

[0002] A Ground Control Point (GCP) refers to a fixed physical point on the ground where precise surveyed coordinates are known. These ground control points may be targets artificially installed on the ground using patterns such as checkerboard, or designated as building corners, manhole centers, or road intersections.

[0003] Through this, accurate coordinate data can be provided for images captured by satellites or drones, and ground control points are used to detect elements that change over time or to map images taken at different locations.

[0004] Therefore, there is a need for a method to efficiently set ground control points on an image.

[0005] The background technology of the present invention is disclosed in Korean Registered Patent Publication No. 10-2013625 (August 19, 2019). The problem to be solved

[0006] The present invention was devised to improve upon the aforementioned problems, and an objective according to one aspect of the present invention is to provide a multi-source image automatic alignment device and method capable of performing precise image alignment even for multi-source images where ground control points are insufficient or data regarding the region is lacking.

[0007] In addition, an object according to one aspect of the present invention is to provide a multi-source image automatic alignment device and method capable of achieving precise alignment results even for a plurality of images with poor initial alignment.

[0008] The problems that the present invention aims to solve are not limited to the problem(s) mentioned above, and other unmentioned problems will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0009] A multi-source image automatic alignment device according to one aspect of the present invention comprises: a memory; and a processor that extracts feature data from each of a plurality of images, aligns the plurality of images based on the size and orientation of the feature data, and extracts corresponding points from the aligned plurality of images to set as ground reference points.

[0010] In the present invention, the plurality of images is characterized by including images in which at least one of the shooting equipment, the time of shooting, and the shooting location is taken under different conditions.

[0011] In the present invention, the processor inputs each of the plurality of images into a pre-prepared feature extraction model, estimates a plurality of feature points corresponding to each of the plurality of images and descriptors for each of the plurality of feature points from the feature extraction model, and generates each of the estimated plurality of feature points and descriptors as feature data.

[0012] In the present invention, the processor is characterized by aligning the plurality of images based on the size and orientation of feature data extracted from each of the plurality of images, estimating an arbitrary verification region from each of the plurality of images, and correcting residual errors for the aligned plurality of images based on the estimated verification region.

[0013] In the present invention, the processor compares the size and orientation of the feature data, aligns a plurality of images based on the comparison result, compares the plurality of feature data estimated from each of the aligned plurality of images, calculates the error between the plurality of images based on the comparison result, and aligns the plurality of images according to the alignment result when the error between the plurality of images is smallest.

[0014] In the present invention, the processor inputs each of the plurality of matched images into a pre-prepared classification model, estimates the verification region corresponding to a predetermined object from the classification model, and performs verification of the matched result by comparing the verification regions estimated from each of the plurality of images.

[0015] In the present invention, the processor is characterized by comparing the phase of the verification region, calculating the residual error based on the comparison result, and moving subpixels for the plurality of images to correct the residual error.

[0016] In the present invention, the processor is characterized by dividing each of the matched plurality of images into a predetermined tile size to generate a plurality of overlapping tiles, identifying corresponding points from each of the plurality of overlapping tiles that correspond to one another using a pre-prepared feature matching model, and setting the identified corresponding points as ground reference points.

[0017] In the present invention, the size and overlap area of ​​each of the overlapping tiles are determined by taking into account the memory usage of the processor.

[0018] In the present invention, the processor is characterized by matching coordinate data to the corresponding point identified as corresponding to each of the plurality of overlapping tiles and setting it as the ground reference point.

[0019] A multi-source image automatic alignment method according to one aspect of the present invention comprises: a step in which a processor extracts feature data from each of a plurality of images; a step in which the processor aligns the plurality of images based on the size and orientation of the feature data; and a step in which the processor extracts corresponding points from the aligned plurality of images and sets them as ground reference points.

[0020] In the present invention, the plurality of images is characterized by including images in which at least one of the shooting equipment, the time of shooting, and the shooting location is taken under different conditions.

[0021] In the present invention, the step of extracting feature data comprises: inputting each of the plurality of images into a pre-prepared feature extraction model; estimating a plurality of feature points corresponding to each of the plurality of images and descriptors for each of the plurality of feature points from the feature extraction model; and generating each of the estimated plurality of feature points and descriptors as feature data.

[0022] In the present invention, the matching step comprises: a step of matching the plurality of images based on the size and direction of feature data extracted from each of the plurality of images; a step of estimating an arbitrary verification region from each of the plurality of images; and a step of correcting residual errors for the matched plurality of images based on the estimated verification region.

[0023] In the present invention, the step of aligning the plurality of images comprises: a step of comparing the size and orientation of the feature data and aligning the plurality of images based on the comparison result; a step of comparing the plurality of feature data estimated from each of the aligned plurality of images and calculating the error between the plurality of images based on the comparison result; and a step of aligning the plurality of images according to the alignment result when the error between the plurality of images is smallest.

[0024] In the present invention, the step of performing the verification comprises: inputting each of the matched plurality of images into a pre-prepared classification model; estimating the verification region corresponding to a predetermined object from the classification model; and performing verification of the match result by comparing the verification regions estimated from each of the plurality of images.

[0025] In the present invention, the step of correcting the residual error comprises: a step of comparing the phase of the verification region; a step of calculating the residual error based on the comparison result; and a step of moving subpixels for the plurality of images to correct the residual error.

[0026] In the present invention, the step of setting the ground reference point comprises: a step of generating a plurality of overlapping tiles by dividing each of the matched plurality of images into a predetermined tile size; a step of identifying corresponding points from each of the plurality of overlapping tiles using a pre-prepared feature matching model; and a step of setting the identified corresponding points as the ground reference point.

[0027] In the present invention, the size and overlap area of ​​each of the overlapping tiles are determined by taking into account the memory usage of the processor.

[0028] In the present invention, the step of setting the ground reference point is characterized by matching coordinate data to the corresponding point identified as corresponding to each of the plurality of overlapping tiles to set the ground reference point. Effects of the invention

[0029] According to an embodiment of the present invention, by extracting feature points from a multi-source image to perform image matching and setting ground control points for the multi-source image based on the extracted feature points, there is an effect of being able to perform precise image matching even for multi-source images where ground control points are insufficient or data regarding the region is lacking.

[0030] In addition, according to an embodiment of the present invention, by performing a global alignment process that corrects large-scale errors and a local alignment process that corrects precise errors in stages for multiple source images, it is possible to achieve precise alignment results even for multiple images with poor initial alignment.

[0031] Meanwhile, the effects of the present invention are not limited to those mentioned above, and various effects may be included within the scope obvious to a person skilled in the art from the contents described below. Brief explanation of the drawing

[0032] FIG. 1 is a block diagram showing a multi-source image automatic alignment device according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating a multi-source image automatic alignment method according to an embodiment of the present invention. Figure 3 is a flowchart illustrating an example of extracting feature data. Figure 4 is a flowchart illustrating an example of aligning images. FIG. 5 is a flowchart illustrating an example of aligning images based on feature data. FIG. 6 is a flowchart illustrating an example of setting ground reference points. Specific details for implementing the invention

[0033] Preferred embodiments of the present invention will be described in detail below with reference to the attached drawings. Prior to this, terms and words used in this specification and claims should not be interpreted as being limited to their ordinary or dictionary meanings. Instead, based on the principle that the inventor may appropriately define the concepts of terms to best describe his invention, they should be interpreted in a meaning and concept consistent with the technical spirit of the present invention. Therefore, it should be understood that the embodiments described in this specification and the configurations illustrated in the drawings are merely some of the most preferred embodiments of the present invention and do not represent all of the technical spirit of the present invention; thus, various equivalents and modifications that can replace them may exist at the time of filing this application. Furthermore, as used in this specification, "comprise" or "include" and / or "comprising" or "including" specify the presence of the mentioned features, numbers, steps, actions, parts, elements, and / or groups thereof, and do not exclude the presence or addition of one or more other features, numbers, actions, parts, elements, and / or groups. In addition, when describing embodiments of the present invention, "may" and "may be" may include "one or more embodiments of the present invention."

[0034] Additionally, to aid in understanding the invention, the attached drawings are not drawn to actual scale, and the dimensions of some components may be exaggerated. Furthermore, the same reference numerals may be assigned to identical components in different embodiments.

[0035] The statement that two subjects of comparison are 'identical' means that they are 'substantially identical.' Therefore, substantial identity may include deviations considered low in the industry, for example, deviations within 5%. Additionally, the statement that a parameter is uniform in a given area may mean that it is uniform from an average perspective.

[0036] Although terms such as "first," "second," etc., are used to describe various components, it goes without saying that these components are not limited by these terms. These terms are used merely to distinguish one component from another, and unless specifically stated otherwise, the first component may also be the second component.

[0037] Throughout the specification, unless specifically stated otherwise, each component may be singular or plural.

[0038] The fact that any configuration is placed on the "upper (or lower)" of a component or on the "upper (or lower)" of a component may mean not only that any configuration is placed in contact with the upper (or lower) surface of said component, but also that another configuration may be interposed between said component and any configuration placed on (or below) said component.

[0039] Furthermore, where it is stated that one component is "connected," "coupled," or "connected" to another component, it should be understood that while said components may be directly connected or connected to each other, another component may be "interposed" between each component, or that each component may be "connected," "coupled," or "connected" through another component. Additionally, when it is stated that a part is electrically coupled with another part, this includes not only cases where they are directly connected but also cases where they are connected with another component in between.

[0040] Throughout the specification, "A and / or B" means A, B, or A and B unless specifically stated otherwise. That is, "and / or" includes any combination or any combination of the enumerated items. "C to D" means C or more and D or less, unless specifically stated otherwise.

[0041] The implementations described herein may be implemented, for example, as methods or processes, devices, software programs, data streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the discussed features may also be implemented in other forms (e.g., devices or programs). Devices may be implemented in appropriate hardware, software, and firmware, etc. Methods may be implemented in devices such as processors, which generally refer to processing devices including, for example, computers, microprocessors, integrated circuits, or programmable logic devices.

[0043] FIG. 1 is a block diagram showing a multi-source image automatic alignment device according to an embodiment of the present invention.

[0044] Referring to FIG. 1, a multi-source image automatic alignment device (100) according to an embodiment of the present invention may include a memory (110) and a processor (120). In this regard, the multi-source image automatic alignment device (100) may further include various components in addition to the components shown in FIG. 1 according to the embodiment.

[0045] The memory (110) may store at least one instruction executed in the process of controlling the overall operation of the multi-source image automatic alignment device (100). At this time, the at least one instruction may include an instruction to perform an operation or step, etc., according to an embodiment of the present invention.

[0046] That is, multiple images can be stored in the memory (110). Here, the multiple images may be images taken of adjacent or identical areas in different environments.

[0047] That is, multiple images may include images taken of an arbitrary area using different shooting equipment, or images taken of an arbitrary area at different times. Additionally, multiple images may include images taken of an arbitrary area at different points.

[0048] Therefore, multiple images may have different light source positions or intensities, different image resolutions, or different image orientations.

[0049] Meanwhile, the image may include coordinate data corresponding to pixels. Here, the coordinate data may include at least one of latitude, longitude, and altitude. According to an embodiment, the image may include coordinate data corresponding to all pixels or coordinate data corresponding to at least one pixel.

[0050] Accordingly, the processor (120) can access data stored in memory (110) and perform an operation to control the multi-source image automatic alignment device (100).

[0051] To this end, the memory (110) may include at least one storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), and EEPROM (Electrically Erasable Programmable Read-Only Memory).

[0052] The processor (120) can be connected to memory (110) and can control at least one other component (e.g., hardware or software component) connected to the processor (120) by executing instructions stored in memory (110), and can perform various data processing or operations.

[0053] Additionally, the processor (120) may be configured to perform each function separately at the hardware, software, or logic level. In this case, dedicated hardware may be used to perform each function. To this end, the processor (120) may be implemented as or include at least one of an ASIC (Application Specific Integrated Circuit), DSP (Digital Signal Processor), PLD (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), CPU (Central Processing Unit), microcontrollers, and / or microprocessors.

[0054] The processor (120) may be implemented as a Central Processing Unit (CPU) or a System on Chip (SoC), and may control multiple hardware or software components connected to the processor (120) by running an operating system or application, and may perform various data processing and operations. The processor (120) may be configured to execute at least one instruction stored in memory (110) and store the execution result data in memory (110).

[0055] Accordingly, the processor (120) can control the overall operation of the multi-source image automatic alignment device (100) according to the present invention. That is, the processor (120) can extract feature data from each of the plurality of images, align the plurality of images based on the size and direction of the feature data, and extract corresponding points from the aligned plurality of images to set as ground reference points.

[0056] To this end, the processor (120) inputs each of a plurality of images into a pre-prepared feature extraction model, estimates a plurality of feature points corresponding to each of the plurality of images and descriptors for each of the plurality of feature points from the feature extraction model, and can generate each of the estimated plurality of feature points and descriptors as feature data.

[0057] At this time, the feature extraction model may be configured to include multiple convolution blocks to estimate a feature map representing the feature probability of each pixel from an image, to identify feature points based on the feature probability from the feature map, and to generate a descriptor representing the pattern between the pixel corresponding to the feature point and other pixels on the feature map.

[0058] This feature extraction model may be pre-trained based on arbitrary training data, and, depending on the embodiment, the feature extraction model may be implemented based on XFeat technology.

[0059] The processor (120) can compare the size and orientation of feature data extracted from each of the multiple images and align the multiple images based on the comparison result. At this time, the processor (120) can perform alignment by correcting another target image (e.g., resizing and rotation) based on one of the reference images among the multiple images.

[0060] To this end, the processor (120) can identify one or more of a plurality of feature points from each of a plurality of images and align the plurality of images using the identified one or more feature points.

[0061] At this time, the processor (120) can align multiple images by considering the size and direction of one or more specific feature points mentioned earlier. For example, the processor (120) can align multiple images such that the location of the feature point and the vector of the descriptor corresponding to the feature point match.

[0062] Accordingly, the processor (120) can compare multiple feature points estimated from each of the multiple aligned images and calculate the error between the multiple images based on the comparison result.

[0063] In this regard, the processor (120) may repeat the process of aligning multiple images by specifying one or more feature points and calculating the error between the aligned multiple images. Accordingly, the processor (120) may align multiple images according to the alignment result when the error between the multiple images is smallest.

[0064] That is, the processor (120) can generate multiple lines connecting multiple feature points on a reference image and multiple feature points on a target image, respectively, and calculate the error between multiple images by considering the length, angle, and length difference of each of the multiple lines.

[0065] At this time, the processor (120) can remove a line as an outlier if at least one of the lengths and angles of the lines deviates from a predetermined standard (e.g., length, pattern, etc.), and remove a line as an outlier if the deviation in length between the lines deviates from a predetermined standard.

[0066] In this case, the processor (120) can recognize the removed outliers as errors, and when the number of outliers is the smallest, it can align multiple images using specific multiple feature point pairs in multiple images.

[0067] In one embodiment, the processor (120) may specify one or more of a plurality of feature points based on the RANSAC (RANdom SAmple Consensus) algorithm and may match a plurality of images using the specified one or more feature points.

[0068] Meanwhile, the processor (120) may input each of the multiple images that have been matched prior to a pre-prepared classification model, estimate a verification area corresponding to a predetermined object from the classification model, and perform verification of the matching result by comparing the verification areas estimated from each of the multiple images.

[0069] At this time, the classification model may be configured to detect a predetermined object from an image. To this end, the classification model may be trained using arbitrary training data and ground truth data labeled on the training data, and in one embodiment, the classification model may be implemented based on YOLO (You Only Look Once).

[0070] In this regard, a predetermined object may refer to any object that appears relatively high in contrast in an image, and may include, for example, structures such as buildings, roads, and intersections.

[0071] Additionally, the processor (120) can compare the phase of the verification region estimated from the previously matched multiple images, calculate a residual error based on the comparison result, and move subpixels for the multiple images to correct the residual error.

[0072] That is, the processor (120) can perform a Fourier transform to verify the phase (or power spectrum) of the verification region. Accordingly, the processor (120) can compare the phases between multiple images and, based on the comparison result, move at least one subpixel among the multiple images.

[0073] At this time, if the distance of a subpixel moved according to the comparison result is greater than a predetermined threshold (e.g., 2 pixels), the processor (120) may determine that the alignment of multiple images failed or attempt the process of extracting feature data from multiple images again.

[0074] Through this, the processor (120) can derive precise alignment results for multiple images.

[0075] The processor (120) divides each of the matched images into a predetermined tile size to generate multiple overlapping tiles, identifies corresponding points from each of the multiple overlapping tiles that correspond to one another using a pre-prepared feature matching model, and can set the identified corresponding points as ground reference points.

[0076] At this time, the processor (120) may divide each of the multiple images such that at least a portion of the overlap tiles divided from each of the multiple images overlap each other. Here, the size of the overlap tile and the overlap area of ​​each may be determined by taking into account the memory usage of the processor (120). For example, the size of the overlap tile may be set to 500 x 500.

[0077] In addition, the feature matching model may be a foundation model pre-trained based on large-scale training data, implemented to identify corresponding points from overlapping tiles even under various conditions such as resolution, season, weather, and lighting.

[0078] In this regard, the processor (120) can extract semantic correspondences through a feature matching model. For example, the processor (120) can extract building corners, manhole covers, points on one side of an intersection, etc. from overlapping tiles through a feature matching model.

[0079] Accordingly, the processor (120) can identify corresponding points corresponding to each other from each of the multiple overlapping tiles through a feature matching model.

[0080] At this time, the processor (120) calculates the positional error between corresponding points identified as corresponding to each other for each of the multiple overlapping tiles, and if the calculated error exceeds a predetermined threshold, the corresponding point may be removed. At this time, the processor (120) may perform alignment on the multiple overlapping tiles or multiple images based on the calculated error.

[0081] Additionally, the processor (120) may remove a corresponding point if, for each of the multiple overlapping tiles, a corresponding point identified as corresponding to one another is identified outside a predetermined area range within the overlapping tile.

[0082] Accordingly, the processor (120) can set a ground reference point by matching coordinate data to a corresponding point identified as corresponding to each of the multiple overlapping tiles.

[0083] Here, a ground control point (GCP) can refer to a point designated to align different images.

[0084] Accordingly, the processor (120) can establish ground reference points by matching coordinate data corresponding to latitude and longitude to corresponding points identified as corresponding to each of the plurality of overlapping tiles. At this time, the processor (120) can match pre-measured coordinate data to feature points for any one of the plurality of images.

[0085] Meanwhile, when multiple ground reference points are set for each of the multiple images, the processor (120) can group two or more of the multiple ground reference points based on the density between the multiple ground reference points.

[0086] For example, the processor (120) can identify two or more ground reference points among a plurality of ground reference points that are shorter than a predetermined threshold distance, and group the identified two or more ground reference points.

[0087] At this time, the processor (120) can perform grouping of ground reference points by identifying a single ground reference point among two or more ground reference points and removing one or more other ground reference points excluding the single ground reference point.

[0088] In this regard, the processor (120) may extract a feature point (or, corresponding point) corresponding to a ground reference point from any image, and correct the coordinate data for the image based on the difference between the coordinate data of the extracted feature point (or, corresponding point) and the coordinate data matched to the ground reference point.

[0090] FIG. 2 is a flowchart illustrating a method for automatically aligning multiple source images according to an embodiment of the present invention. FIG. 3 is a flowchart illustrating an embodiment for extracting feature data. FIG. 4 is a flowchart illustrating an embodiment for aligning images. FIG. 5 is a flowchart illustrating an embodiment for aligning images based on feature data. FIG. 6 is a flowchart illustrating an embodiment for setting ground reference points.

[0091] Hereinafter, a method for automatically aligning multiple source images according to an embodiment of the present invention will be described with reference to the configuration of the multi-source image automatic alignment device (100) described above. Some of the steps described below may be performed in a different order than the order described below or omitted.

[0092] Referring to FIG. 2, the processor (120) can extract feature data from each of a plurality of images (S100).

[0093] To this end, with reference to FIG. 3, the processor (120) inputs each of a plurality of images into a pre-prepared feature extraction model (S110), estimates a plurality of feature points corresponding to each of the plurality of images and descriptors for each of the plurality of feature points from the feature extraction model (S120), and can generate each of the estimated plurality of feature points and descriptors as feature data (S130).

[0094] The processor (120) can align multiple images based on the size and orientation of feature data (S200).

[0095] To this end, with reference to FIG. 4, the processor (120) aligns multiple images based on the size and orientation of feature data extracted from each of the multiple images (S210), estimates an arbitrary verification region from each of the multiple images (S220), and corrects residual errors for the previously aligned multiple images based on the estimated verification region (S230).

[0096] That is, as shown in FIG. 5, the processor (120) can compare the size and orientation of feature data extracted from each of the multiple images and align the multiple images based on the comparison result.

[0097] To this end, the processor (120) can identify one or more of a plurality of feature points from each of a plurality of images and align the plurality of images using the identified one or more feature points.

[0098] At this time, the processor (120) can compare multiple feature points estimated from each of the multiple aligned images (S211), and calculate the error between the multiple images based on the comparison result (S212).

[0099] In this regard, the processor (120) may repeat the process of aligning multiple images by specifying one or more feature points and calculating the error between the aligned multiple images. Accordingly, the processor (120) may align multiple images according to the alignment result when the error between the multiple images is smallest (S213).

[0100] Meanwhile, the processor (120) may input each of the multiple images that have been matched prior to a pre-prepared classification model, estimate a verification area corresponding to a predetermined object from the classification model, and perform verification of the matching result by comparing the verification areas estimated from each of the multiple images.

[0101] Additionally, the processor (120) can compare the phase of the verification region estimated from the previously matched multiple images, calculate a residual error based on the comparison result, and move subpixels for the multiple images to correct the residual error.

[0102] The processor (120) can extract corresponding points from a plurality of aligned images and set them as ground reference points (S300).

[0103] To this end, referring to FIG. 6, the processor (120) divides each of the matched images into a predetermined tile size to generate a plurality of overlapping tiles (S310), identifies corresponding points corresponding to each other from each of the plurality of overlapping tiles using a pre-prepared feature matching model (S320), and can set the identified corresponding points as ground reference points (S330).

[0104] At this time, the processor (120) calculates the position error between corresponding points identified as corresponding to each other for each of the multiple overlapping tiles, and if the calculated error exceeds a predetermined threshold, the corresponding point may be removed.

[0105] Additionally, the processor (120) may remove a corresponding point if, for each of the multiple overlapping tiles, a corresponding point identified as corresponding to one another is identified outside a predetermined area range within the overlapping tile.

[0106] Accordingly, the processor (120) can set a ground reference point by matching coordinate data to a corresponding point identified as corresponding to each of the multiple overlapping tiles.

[0107] Meanwhile, when multiple ground reference points are set for each of the multiple images, the processor (120) can group two or more of the multiple ground reference points based on the density between the multiple ground reference points.

[0109] As described above, according to the multi-source image automatic alignment device and method of the embodiment of the present invention, by extracting feature points from a multi-source image to perform image alignment and setting ground reference points for the multi-source image based on the extracted feature points, precise image alignment can be performed even for multi-source images where ground reference points are insufficient or data regarding the region is lacking.

[0110] In addition, according to the multi-source image automatic alignment device and method of the embodiment of the present invention, by performing a global alignment process that corrects large-scale errors and a local alignment process that corrects precise errors for multi-source images in stages, precise alignment results can be achieved even for multiple images with poor initial alignment status.

[0111] Although the present invention has been described with reference to the embodiments illustrated in the drawings, this is merely illustrative and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the present invention should be determined by the claims below. Explanation of the symbols

[0112] 100: Multi-source image automatic alignment device 110: Memory 120: Processor

Claims

Claim 1 A multi-source image automatic alignment device comprising: a memory; and a processor that extracts feature data from each of a plurality of images, aligns the plurality of images based on the size and direction of the feature data, and extracts corresponding points from the aligned plurality of images and sets them as ground reference points; wherein the processor generates a plurality of lines connecting a plurality of feature points on a reference image and a plurality of feature points on a target image, removes a line as an outlier if at least one of the length and angle of each of the plurality of lines deviates from a predetermined standard, removes a line as an outlier if the deviation in length between the plurality of lines deviates from a predetermined standard, and aligns the plurality of images using the pair of feature points when the number of outliers is the smallest. Claim 2 A multi-source image automatic alignment device according to claim 1, wherein the plurality of images includes images in which at least one of the shooting equipment, the time of shooting, and the shooting point is different. Claim 3 A multi-source image automatic alignment device according to claim 1, wherein the processor inputs each of the plurality of images into a pre-prepared feature extraction model, estimates a plurality of feature points corresponding to each of the plurality of images and descriptors for each of the plurality of feature points from the feature extraction model, and generates each of the estimated plurality of feature points and descriptors as feature data. Claim 4 A multi-source image automatic alignment device according to claim 1, wherein the processor aligns the plurality of images based on the size and direction of feature data extracted from each of the plurality of images, estimates an arbitrary verification region from each of the plurality of images, and corrects residual errors for the aligned plurality of images based on the estimated verification region. Claim 5 A multi-source image automatic alignment device according to claim 4, wherein the processor compares the size and orientation of the feature data, aligns a plurality of images based on the comparison result, compares the plurality of feature data estimated from each of the aligned plurality of images, calculates the error between the plurality of images based on the comparison result, and aligns the plurality of images according to the alignment result when the error between the plurality of images is smallest. Claim 6 A multi-source image automatic alignment device according to claim 4, wherein the processor inputs each of the plurality of aligned images into a pre-prepared classification model, estimates the verification region corresponding to a predetermined object from the classification model, and performs verification of the alignment result by comparing the verification regions estimated from each of the plurality of images. Claim 7 In claim 4, the processor compares the phase of the verification region, calculates the residual error based on the comparison result, and moves subpixels for the plurality of images to correct the residual error, a multi-source image automatic alignment device. Claim 8 A multi-source image automatic alignment device according to claim 1, wherein the processor divides each of the matched plurality of images into a predetermined tile size to generate a plurality of overlapping tiles, identifies corresponding points corresponding to each other from each of the plurality of overlapping tiles using a pre-prepared feature matching model, and sets the identified corresponding points as ground reference points. Claim 9 A multi-source image automatic alignment device according to claim 8, wherein the size and overlap area of ​​each of the overlap tiles are determined in consideration of the memory usage of the processor. Claim 10 In claim 8, the processor sets the corresponding points identified as corresponding to each of the plurality of overlapping tiles as ground reference points by matching coordinate data to the corresponding points, thereby forming a multi-source image automatic alignment device. Claim 11 A multi-source image automatic alignment method comprising: a step in which a processor extracts feature data from each of a plurality of images; a step in which the processor aligns the plurality of images based on the size and direction of the feature data; and a step in which the processor extracts corresponding points from the aligned plurality of images and sets them as ground reference points; wherein, in the aligning step, the processor generates a plurality of lines connecting a plurality of feature points on a reference image and a plurality of feature points on a target image, respectively, removes a line as an outlier if at least one of the length and angle of each of the plurality of lines deviates from a predetermined standard, removes a line as an outlier if the deviation in length between the plurality of lines deviates from a predetermined standard, and aligns the plurality of images using the pair of feature points when the number of outliers is the smallest. Claim 12 A method for automatically aligning multiple source images, wherein, in claim 11, the plurality of images include images in which at least one of the shooting equipment, the time of shooting, and the shooting location is different. Claim 13 A method for automatically aligning multiple source images according to claim 11, wherein the step of extracting feature data comprises: inputting each of the plurality of images into a pre-prepared feature extraction model; estimating a plurality of feature points corresponding to each of the plurality of images and descriptors for each of the plurality of feature points from the feature extraction model; and generating each of the estimated plurality of feature points and descriptors as feature data. Claim 14 A method for automatically aligning multiple source images according to claim 11, wherein the matching step comprises: matching the plurality of images based on the size and direction of feature data extracted from each of the plurality of images; estimating an arbitrary verification region from each of the plurality of images; and correcting residual errors for the matched plurality of images based on the estimated verification region. Claim 15 A method for automatically aligning multiple source images according to claim 14, wherein the step of aligning the plurality of images comprises: a step of comparing the size and orientation of the feature data and aligning the plurality of images based on the comparison result; a step of comparing the plurality of feature data estimated from each of the aligned plurality of images and calculating the error between the plurality of images based on the comparison result; and a step of aligning the plurality of images according to the alignment result when the error between the plurality of images is smallest. Claim 16 A method for automatically aligning multiple source images according to claim 14, wherein the step of performing the verification comprises: inputting each of the plurality of aligned images into a pre-prepared classification model; estimating the verification area corresponding to a predetermined object from the classification model; and performing verification of the alignment result by comparing the verification areas estimated from each of the plurality of images. Claim 17 A multi-source image automatic alignment method according to claim 14, wherein the step of correcting the residual error comprises: a step of comparing the phase of the verification region; a step of calculating the residual error based on the comparison result; and a step of moving subpixels for the plurality of images to correct the residual error. Claim 18 A method for automatically aligning multiple source images according to claim 11, wherein the step of setting the ground reference point comprises: dividing each of the matched plurality of images into a predetermined tile size to generate a plurality of overlapping tiles; identifying corresponding points corresponding to each other from each of the plurality of overlapping tiles using a pre-prepared feature matching model; and setting the identified corresponding points as the ground reference point. Claim 19 A multi-source image automatic alignment method according to claim 18, wherein the size and overlap area of ​​each of the overlapping tiles are determined in consideration of the memory usage of the processor. Claim 20 In claim 18, the step of setting the ground reference point is to set the ground reference point by matching coordinate data to the corresponding point identified as corresponding to each of the plurality of overlapping tiles. A multi-source image automatic alignment method.

Images