Information processing device, information processing method, and information processing program

Abstract

To obtain orthomosaic images with fewer errors while minimizing the increase in computation time. [Solution] The information processing device 100 performs an inverse orthorectification on a group of basic feature points, which are multiple feature points of an orthoimage corresponding to the aerial image, using basic transformation coefficients, which are orthorectification coefficients of the aerial image, to calculate an inverse transformation feature point group corresponding to the basic feature point group. The device updates the basic transformation coefficients based on a basic positional shift amount, which is the positional shift amount of the basic feature point group relative to a group of reference feature points, which are multiple feature points of a reference image, to obtain an improved transformation coefficient. The device performs an orthorectification on the inverse transformation feature point group using the improved transformation coefficient to calculate an improved feature point group corresponding to the basic feature point group. The device calculates the positional shift amount of the improved feature point group relative to the reference feature point group as the improved positional shift amount obtained by improving the basic feature point group.

Description

Technical Field

[0001] This disclosure relates to a technique for obtaining an orthoimage with few errors.

Background Art

[0002] There are errors in the information indicating the shooting range of satellite images. Therefore, in order to overlay satellite images with past images or maps, alignment based on the ground features shown in the satellite images is necessary. As a means of automatically performing this alignment, there is a method of ortho-converting a satellite image, extracting image feature points from the orthoimage, and performing alignment by feature point matching.

[0003] For ortho-conversion, the image before alignment and DEM data are used. However, since the image before alignment is used, an error occurs between the position of the building on the image and the position of the elevation data held by the DEM data, and ortho-conversion cannot be accurately performed, and the position of the image feature points includes errors, so accurate alignment cannot be performed. Ortho-conversion is also called ortho-correction. DEM is an abbreviation for Digital Elevation Model.

[0004] The technique of Patent Document 1 repeatedly performs ortho-conversion, feature point extraction, and calculation of the amount of misalignment using the result of alignment, thereby realizing high-precision alignment. As a result, it becomes possible to extract feature points with little distortion and to perform high-precision alignment that suppresses the influence of the misalignment between the image and the DEM. However, this technique has a problem that the calculation time increases according to the number of times of repeatedly performing ortho-conversion, feature point extraction, and calculation of the amount of misalignment.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] This disclosure aims to enable the acquisition of orthomosaic images with fewer errors while suppressing the increase in computation time. [Means for solving the problem]

[0007] The information processing device disclosed herein is The improvement processing unit performs an inverse orthorectification on a group of basic feature points, which are multiple feature points of an orthoimage corresponding to the aerial image, using basic transformation coefficients, which are orthorectification coefficients of the aerial image, to calculate an inverse transformation feature point group corresponding to the basic feature point group. Based on the basic positional displacement amount, which is the positional displacement amount of the basic feature point group with respect to a group of reference feature points, which are multiple feature points of a reference image, the basic transformation coefficients are updated to obtain improved transformation coefficients. An orthorectification is performed on the inverse transformation feature point group using the improved transformation coefficients to calculate an improved feature point group corresponding to the basic feature point group. The positional displacement amount of the improved feature point group with respect to the reference feature point group is calculated as the improved positional displacement amount obtained by improving the basic feature point group. It is equipped with. [Effects of the Invention]

[0008] According to this disclosure, it is possible to obtain orthomosaic images with fewer errors while suppressing the increase in computation time. [Brief explanation of the drawing]

[0009] [Figure 1] A diagram showing the configuration of the information processing device 100 in Embodiment 1. [Figure 2] Functional configuration diagram of the information processing device 100 in Embodiment 1. [Figure 3] Functional configuration diagram of the information processing device 100 in Embodiment 1. [Figure 4] A flowchart of the information processing method in Embodiment 1. [Figure 5] A flowchart of the information processing method in Embodiment 1. [Figure 6] A diagram showing how a subject appears in an aerial image. [Figure 7] A diagram showing how a subject appears in an orthoimage corresponding to an aerial image. [Figure 8] A diagram showing the state of alignment between an orthoimage and a reference image. [Figure 9] A diagram showing how a subject appears in an orthoimage corresponding to an aerial image. [Figure 10] A diagram showing the state of alignment between an orthoimage and a reference image. [Figure 11] A diagram showing an overview of the prior art. [Figure 12] A diagram showing an overview of Embodiment 1.

Mode for Carrying Out the Invention

[0010] In the embodiments and the drawings, the same elements or corresponding elements are denoted by the same reference numerals. The description of elements denoted by the same reference numerals as those already described is omitted or simplified as appropriate. The arrows in the figures mainly indicate the flow of data or the flow of processing.

[0011] Embodiment 1. The information processing apparatus 100 will be described based on FIGS. 1 to 5.

[0012] ***Description of Configuration*** Based on FIG. 1, the configuration of the information processing apparatus 100 will be described. The information processing apparatus 100 is a computer including hardware such as a processor 101, a memory 102, an auxiliary storage device 103, a communication device 104, and an input / output interface 105. These hardware components are connected to each other via signal lines.

[0013] The processor 101 is an IC that performs arithmetic processing and controls other hardware. For example, the processor 101 is a CPU, a DSP, a GPU, or a combination thereof. IC is an abbreviation for Integrated Circuit. The CPU is an abbreviation for Central Processing Unit. The DSP is an abbreviation for Digital Signal Processor. The GPU is an abbreviation for Graphics Processing Unit.

[0014] The memory 102 is a volatile or non-volatile storage device. The memory 102 is also called the main memory device or main memory. For example, the memory 102 is RAM. The data stored in the memory 102 is saved to the auxiliary storage device 103 as needed. RAM is an abbreviation for Random Access Memory.

[0015] The auxiliary storage device 103 is a non-volatile storage device. For example, the auxiliary storage device 103 is a ROM, HDD, flash memory, or a combination thereof. The data stored in the auxiliary storage device 103 is loaded into the memory 102 as needed. ROM is an abbreviation for Read Only Memory. HDD is an abbreviation for Hard Disk Drive.

[0016] The communication device 104 is a receiver and a transmitter. For example, the communication device 104 is a communication chip or NIC. The communication of the information processing device 100 is performed using the communication device 104. NIC is an abbreviation for Network Interface Card.

[0017] The input / output interface 105 is a port to which an input device and an output device are connected. For example, the input / output interface 105 is a USB terminal, the input device is a keyboard and a mouse, and the output device is a display. The input / output of the information processing device 100 is performed via the input / output interface 105. USB is an abbreviation for Universal Serial Bus.

[0018] The information processing device 100 comprises elements such as a basic processing unit 110, an improvement judgment unit 120, an improvement processing unit 130, and an image correction unit 140. These elements are implemented by software.

[0019] The auxiliary storage device 103 stores information processing programs that enable the computer to function as a basic processing unit 110, an improvement judgment unit 120, an improvement processing unit 130, and an image correction unit 140. The information processing programs are loaded into memory 102 and executed by the processor 101. The auxiliary storage device 103 also stores the operating system. At least a portion of the OS is loaded into memory 102 and executed by the processor 101. Processor 101 executes information processing programs while running the operating system. OS is an abbreviation for Operating System.

[0020] The data for the information processing program (input data, output data, etc.) is stored in the storage unit 190. Memory 102 functions as a storage unit 190. However, storage devices such as auxiliary storage device 103, registers in the processor 101, and cache memory in the processor 101 may function as a storage unit 190 instead of memory 102, or together with memory 102.

[0021] Information processing programs can be recorded (stored) in a computer-readable format on non-volatile recording media such as optical discs or flash memory.

[0022] The configuration of the basic processing unit 110 will be explained based on Figure 2. The basic processing unit 110 includes elements such as a data acquisition unit 111, an orthorectification unit 112, a feature point extraction unit 113, a matching unit 114, and a positional displacement amount calculation unit 115. The image storage unit 201, the shooting location information storage unit 202, the ground altitude information storage unit 203, the ground location information storage unit 204, and the acquired data storage unit 191 will be described later.

[0023] The configuration of the improvement processing unit 130 will be explained based on Figure 3. The improvement processing unit 130 includes elements such as an inverse orthorectification unit 131, a coefficient update unit 132, an orthorectification unit 133, a matching unit 134, and a positional displacement amount calculation unit 135.

[0024] ***Explanation of operation*** The operating procedure of the information processing device 100 corresponds to the information processing method. Furthermore, the operating procedure of the information processing device 100 corresponds to the processing procedure of the information processing program.

[0025] The information processing method will be explained based on Figures 4 and 5. In step S111, the data acquisition unit 111 acquires various types of data and stores the acquired data in the acquired data storage unit 191.

[0026] Specifically, the following data will be obtained. The image storage unit 201 stores aerial images and reference images. Aerial images are associated with orthorectification coefficients. An example of orthorectification coefficients is Rational. This is Polynomial Coefficients (RPC). The orthorectification coefficients of aerial images are called "basic transformation coefficients." The data acquisition unit 111 acquires aerial images, reference images, and basic transformation coefficients from the image storage unit 201. Aerial images are images obtained by photographing a target area from the air. Aerial images show the target area as seen from the point of photography. The target area is the area being photographed. For example, aerial images are obtained by photographing a target area from a flying object that is suspended above that area. An example of such a flying object is a satellite. The reference image shows the target area as viewed from directly above.

[0027] The shooting location information storage unit 202 stores the shooting location information. The data acquisition unit 111 acquires shooting location information from the shooting location information storage unit 202. The shooting location information indicates the position of the flying object (shooting location) in the target area at the time of shooting. The position is shown as a three-dimensional coordinate value.

[0028] Ground altitude information storage unit 203 stores ground altitude information. The data acquisition unit 111 acquires ground altitude information from the ground altitude information storage unit 203. Ground elevation information indicates the altitude of each point in a target area. Ground elevation information is also called topographic data. An example of ground elevation information is a DEM (Digital Element Model).

[0029] Ground position information is stored in the ground position information storage unit 204. The data acquisition unit 111 acquires ground position information from the ground position information storage unit 204. Ground position information indicates the location of each point in the target area. Ground position information is also called GCP information. GCP is an abbreviation for Ground Control Point.

[0030] In step S112, the orthorectifier unit 112 performs orthorectification on the aerial image using basic transformation coefficients.

[0031] Specifically, the orthorectifier unit 112 orthorectifies the aerial image using the basic conversion coefficient, the shooting location information, the ground altitude information, and the ground position information.

[0032] This creates an orthomosaic image corresponding to the aerial image. The orthomosaic image corresponding to the aerial image represents the target area as it was viewed from directly above.

[0033] In the following steps, "orthoimage" refers to the orthoimage created in step S112.

[0034] In step S113, the feature point extraction unit 113 performs feature point extraction on both the orthoimage and the reference image. In feature point extraction, multiple feature points are extracted, and the feature quantity for each feature point is calculated.

[0035] For example, the feature point extraction unit 113 performs SIFT on both the orthophoto and the reference image. SIFT is an abbreviation for Scale Invariant Feature Transform.

[0036] This process extracts multiple feature points from the orthomosaic image. These multiple feature points extracted from the orthomosaic image are called the "basic feature point set." Furthermore, multiple feature points are extracted from the reference image. These multiple feature points extracted from the reference image are called the "reference feature point group."

[0037] In step S114, the matching unit 114 matches the basic feature point cloud with the reference feature point cloud.

[0038] In matching, basic feature points and reference feature points with similar features are associated with each other. Matching is performed using existing methods.

[0039] In step S115, the positional displacement calculation unit 115 calculates the amount of positional displacement (positional displacement) of the basic feature point cloud relative to the position of the reference feature point cloud based on the matching results.

[0040] The displacement is calculated based on the difference in positions between the two matched feature point clouds.

[0041] The amount of positional displacement calculated in step S115 is called the "basic positional displacement."

[0042] In step S116, the improvement determination unit 120 determines whether or not improvement of the basic feature point cloud is necessary based on the basic positional displacement amount.

[0043] The need for improvement of the basic feature point cloud is determined as follows: The improvement judgment unit 120 compares the basic positional deviation amount with a positional deviation threshold. The positional deviation threshold is predetermined. If the basic positional deviation is greater than the positional deviation threshold, the improvement determination unit 120 determines that improvement of the basic feature point cloud is necessary. If the basic positional deviation is less than or equal to the positional deviation threshold, the improvement determination unit 120 determines that improvement of the basic feature point cloud is unnecessary.

[0044] If it is determined that improvement of the basic feature point cloud is necessary, the process proceeds to step S121. If it is determined that no improvement is needed to the basic feature point cloud, the process proceeds to step S117.

[0045] In step S117, the image correction unit 140 performs alignment correction on the orthomosaic image using the basic positional displacement amount.

[0046] Alignment correction is performed as follows: First, the image correction unit 140 creates an affine matrix based on the amount of positional displacement. The affine matrix is ​​also called the alignment coefficient. Then, the image correction unit 140 performs an affine transformation on the orthorectified image using an affine matrix. This applies at least one of the following to the orthorectal image: scaling, translation, or rotation.

[0047] The image correction unit 140 then outputs an orthomosaic image after alignment correction. For example, the orthomosaic image after alignment correction is displayed on the screen.

[0048] In step S121, the inverse orthorectifier unit 131 performs an inverse orthorectifier on the target feature point cloud using the target transformation coefficients.

[0049] In the first step S121, the target transformation coefficients are the basic transformation coefficients, and the target feature point cloud is the basic feature point cloud.

[0050] In the second and subsequent steps S121, the target transformation coefficient is the improved transformation coefficient obtained in the previous step S122 (the latest improved transformation coefficient at this point). Also, the target feature point cloud is the improved feature point cloud calculated in the previous step S123 (the latest improved feature point cloud at this point).

[0051] This allows the inverse transform feature point cloud corresponding to the target feature point cloud to be calculated. The inverse transform feature point group is the target feature point group after the inverse orthorectification.

[0052] The inverse transform feature point group calculated in step S121 will be referred to as the "current inverse transform feature point group." The inverse transform feature point group calculated in the second and subsequent steps S121 will be referred to as the "new inverse transform feature point group."

[0053] In step S122, the coefficient update unit 132 updates the target conversion coefficient based on the target position displacement amount.

[0054] In the first step S122, the target position displacement is the basic position displacement, and the target transformation coefficient is the basic transformation coefficient.

[0055] In the second and subsequent steps S122, the target positional displacement is the improved positional displacement calculated in the previous step S125 (the latest improved positional displacement at this point). The target conversion coefficient is the improved conversion coefficient obtained in the previous step S122 (the latest improved conversion coefficient at this point).

[0056] The target transformation coefficients are updated as follows: First, the coefficient update unit 132 creates an affine matrix (or homography matrix) based on the target position displacement. Then, the coefficient update unit 132 performs an affine transformation (or homography transformation) on the target transformation coefficients using an affine matrix (or homography matrix), and calculates new transformation coefficients by linear multiple regression based on the correspondence of coordinates before and after the affine transformation (or homography transformation).

[0057] This yields an improved conversion coefficient. The improved conversion coefficient is the target conversion coefficient after the update.

[0058] The improvement conversion coefficient obtained in step S122 will be referred to as the "current improvement conversion coefficient." The improvement conversion coefficient obtained in the second and subsequent steps S122 is referred to as the "new improvement conversion coefficient."

[0059] In step S123, the orthorectifier unit 133 performs an orthorectifier on the inverse transformed feature point cloud using the improved transformation coefficients.

[0060] This allows for the calculation of an improved feature point cloud corresponding to the target feature point cloud. The improved feature point cloud is the inverse transform feature point cloud after the orthorectification.

[0061] The set of improved feature points calculated in step S123 will be referred to as the "current set of improved feature points". The improved feature point set calculated in the second and subsequent steps S123 will be referred to as the "new improved feature point set."

[0062] In step S124, the matching unit 134 matches the improved feature point group with the reference feature point group.

[0063] In step S125, the positional displacement calculation unit 135 calculates the amount of positional displacement (improved positional displacement) of the improved feature point cloud relative to the position of the reference feature point cloud based on the matching results.

[0064] The amount of positional deviation improved in step S125 will be referred to as the "amount of positional deviation improved in this step." The amount of positional misalignment improved in the first step S125 is referred to as the "initial improved positional misalignment," and the amount of positional misalignment improved in the second and subsequent steps S125 is referred to as the "new improved positional misalignment."

[0065] In step S126, the improvement determination unit 120 determines whether or not improvement is necessary for the current improvement feature point group (the latest improvement feature point group at this point) based on the current improvement positional deviation amount (the latest improvement positional deviation amount at this point).

[0066] The necessity of improving the feature point cloud in this improvement is determined as follows: The improvement judgment unit 120 compares the amount of positional deviation improved in this case with a positional deviation threshold. If the amount of positional misalignment corrected in this case is greater than the positional misalignment threshold, and the number of executions of step S126 is less than the execution threshold, the improvement determination unit 120 determines that improvement of the feature point cloud corrected in this case is necessary. If the amount of positional misalignment corrected in this case is less than or equal to the positional misalignment threshold, or if the number of executions of step S126 is greater than or equal to the execution threshold, the improvement determination unit 120 determines that improvement of the feature point cloud corrected in this case is unnecessary. The number of times step S126 is executed corresponds to the number of times the orthorectification coefficients are updated (step S122). The threshold number of executions is predetermined.

[0067] If it is determined that improvement to the feature point cloud is necessary, the process proceeds to step S121. If it is determined that no further improvement is needed for the feature point cloud in this improvement, the process proceeds to step S127.

[0068] In step S127, the coefficient update unit 132 updates the latest improvement conversion coefficient (the improvement conversion coefficient obtained in the last step S122) based on the latest improvement position deviation amount (the improvement position deviation amount calculated in the last step S125). The update method is the same as the method in step S122. The improvement conversion coefficient obtained by this update is called the updated improvement conversion coefficient. The image correction unit 140 then performs orthorectification on the aerial image using the updated improvement conversion coefficient. This creates an orthophoto image after alignment correction.

[0069] The image correction unit 140 then outputs an orthomosaic image after alignment correction.

[0070] ***Effects of Embodiment 1*** The information processing device 100 performs orthorectification in the first instance, similar to conventional technology. However, from the second instance onward (nth instance), the information processing device 100 performs inverse orthorectification only on feature points using the (n-1)th orthorectification coefficients. In this inverse orthorectification, only the coordinate values ​​of the feature points are converted back to their original coordinate values ​​on the aerial image. The information processing device 100 also updates the (n-1)th orthorectification coefficients to the nth orthorectification coefficients using the (n-1)th positional displacement amount. Then, the information processing device 100 performs orthorectification only on feature points using the nth orthorectification coefficients and proceeds with the matching and subsequent processes. This makes it possible to achieve high-speed and computationally efficient feature point extraction with minimal distortion, as well as high-precision alignment that minimizes the effects of image / DEM misalignment.

[0071] ***Supplementary Information on the Embodiment*** Figure 6 shows how the subject appears in aerial images when (A) the target area is photographed from an oblique angle above and (B) the target area is photographed from directly above. The subject is a mountain. This mountain has one feature at its summit and two features on its slopes. The dashed lines represent contour lines. (A) When the target area is photographed from an oblique angle above, and (B) when the target area is photographed from directly above, the appearance of the mountains differs in the aerial images, and the positions of the three features are different in each case.

[0072] Figure 7 shows how the subject appears in the orthorectified aerial image (orthographic image). When an aerial image is orthorectified, an orthophoto is created. An orthomosaic image represents a subject as seen from directly above.

[0073] Figure 8 shows the alignment of the orthomosaic image and the reference image. In Figure 8, the difference (positional shift) between the position of features in the orthomosaic image and the position of features in the reference image is minimal. If the misalignment is slight, it is possible to accurately align and correct the orthorectal image by scaling, translation, and rotation.

[0074] In Figure 9, the positional shift of the aerial image relative to the terrain data is large. In such cases, orthorectification cannot be performed correctly, resulting in the creation of orthophotos with large errors. In Figure 10, the orthomosaic image has a large error, resulting in a large difference (positional shift) between the position of features in the orthomosaic image and the position of features in the reference image. If the misalignment is large, it is difficult to accurately align and correct the orthorectal image by scaling, translation, and rotation.

[0075] Based on Figure 11, we will explain the overview of the conventional technology. In the first step, an orthorectification is performed using orthorectification coefficients from the aerial image to create an orthophoto. Then, alignment is performed, with full awareness of the large error margin of the orthophoto. In subsequent attempts, an orthorectification is performed using orthorectification coefficients that reflect the results of the alignment, creating a new orthoimage. Then, alignment is performed with the orthoimage's error improved. In other words, from the second time onward, the orthorectification is performed on the entire orthophoto.

[0076] An overview of Embodiment 1 will be described based on Figure 12. In the first step, an orthorectification is performed using orthorectification coefficients from the aerial image to create an orthophoto. Then, alignment is performed, with full awareness of the large error margin of the orthophoto. In subsequent attempts, the feature points in the aerial image are inversely orthorectified using the old orthorectification coefficients, and the feature points after inverse orthorectification are orthorectified again using new orthorectification coefficients that reflect the alignment results. Then, alignment is performed with the feature point error improved. In other words, from the second time onward, the orthorectification is not performed on the entire orthoimage, but only on the feature points, with both inverse orthorectification and orthorectification being applied.

[0077] Embodiment 1 is an example of a preferred form and is not intended to limit the technical scope of this disclosure. Embodiment 1 may be implemented in part or in combination with other forms. The procedure described using flowcharts, etc., may be modified as appropriate. While an affine matrix was used as an example of the alignment coefficient, other linear or nonlinear models may also be used. Linear models include offset matrices, affine matrices, or homography matrices. Nonlinear models include polynomials of degree 3 or less, or polynomials of degree 4 or higher.

[0078] The information processing device 100 may be composed of multiple devices. Furthermore, the functions of the information processing device 100 may be implemented by multiple devices. Each element of the information processing device 100 may be implemented using software, hardware, firmware, or a combination thereof. The "part" of each element of the information processing device 100 may be read as "processing," "process," "circuit," or "circuit." [Explanation of Symbols]

[0079] 100 Information processing device, 101 Processor, 102 Memory, 103 Auxiliary storage device, 104 Communication device, 105 Input / Output interface, 110 Basic processing unit, 111 Data acquisition unit, 112 Orthorectification unit, 113 Feature point extraction unit, 114 Matching unit, 115 Position shift amount calculation unit, 120 Improvement judgment unit, 130 Improvement processing unit, 131 Inverse orthorectification unit, 132 Coefficient update unit, 133 Orthorectification unit, 134 Matching unit, 135 Position shift amount calculation unit, 140 Image correction unit, 190 Storage unit, 191 Acquired data storage unit, 201 Image storage unit, 202 Shooting location information storage unit, 203 Ground altitude information storage unit, 204 Ground position information storage unit.

Claims

1. The improvement processing unit performs an inverse orthorectification on a group of basic feature points, which are multiple feature points of an orthoimage corresponding to the aerial image, using basic transformation coefficients, which are orthorectification coefficients of the aerial image, to calculate an inverse transformation feature point group corresponding to the basic feature point group. Based on the basic positional displacement amount, which is the positional displacement amount of the basic feature point group with respect to a group of reference feature points, which are multiple feature points of a reference image, the basic transformation coefficients are updated to obtain improved transformation coefficients. An orthorectification is performed on the inverse transformation feature point group using the improved transformation coefficients to calculate an improved feature point group corresponding to the basic feature point group. The positional displacement amount of the improved feature point group with respect to the reference feature point group is calculated as the improved positional displacement amount obtained by improving the basic feature point group. An information processing device equipped with the following features.

2. The information processing device includes an improvement determination unit that determines whether or not improvement is necessary for the latest improvement feature point cloud based on the latest improvement positional displacement amount. When the improvement processing unit determines that improvement is necessary for the latest improved feature point cloud, it performs the inverse orthorectification on the latest improved feature point cloud using the latest improvement conversion coefficient to calculate a new inverse transformed feature point cloud, updates the latest improvement conversion coefficient based on the latest improved positional shift amount to obtain a new improvement conversion coefficient, performs the orthorectification on the new inverse transformed feature point cloud using the new improvement conversion coefficient to calculate a new improved feature point cloud, and calculates a new improved positional shift amount, which is the positional shift amount of the new improved feature point cloud relative to the reference feature point cloud. The information processing apparatus according to claim 1.

3. The aforementioned information processing device includes an image correction unit, If the improvement processing unit determines that improvement of the latest improvement feature point cloud is unnecessary, it updates the latest improvement conversion coefficient based on the latest improvement position shift amount. If the image correction unit determines that further improvement to the latest improved feature point cloud is unnecessary, it performs an orthorectification on the aerial image using the updated improvement conversion coefficient. The information processing apparatus according to claim 2.

4. The information processing device includes a basic processing unit that performs the orthorectification on the aerial image using the basic transformation coefficients to create the orthoimage, extracts the basic feature point cloud and the reference feature point cloud from the orthoimage and the reference image, and calculates the basic positional displacement amount using the basic feature point cloud and the reference feature point cloud. The improvement determination unit determines whether or not improvement of the basic feature point cloud is necessary based on the basic positional displacement amount. The improvement processing unit calculates the initial improvement position shift amount when it determines that improvement of the basic feature point cloud is necessary. The image correction unit performs alignment correction on the orthoimage using the basic positional displacement amount when it determines that improvement of the basic feature point cloud is unnecessary. The information processing apparatus according to claim 3.

5. Using the basic transformation coefficients, which are orthorectification coefficients of the aerial image, an inverse orthorectification is performed on the basic feature point group, which is a set of multiple feature points in the orthoimage corresponding to the aerial image, to calculate the inverse transformation feature point group corresponding to the basic feature point group. The basic transformation coefficients are updated based on the basic positional displacement amount, which is the positional displacement amount of the basic feature point group with respect to the reference feature point group, which is a set of multiple feature points in the reference image, to obtain improved transformation coefficients. An orthorectification is performed on the inverse transformation feature point group using the improved transformation coefficients to calculate the improved feature point group corresponding to the basic feature point group. The positional displacement amount of the improved feature point group with respect to the reference feature point group is calculated as the improved positional displacement amount obtained by improving the basic feature point group. Information processing methods.

6. Based on the latest improvement amount, determine whether or not improvement is necessary for the latest improvement feature point cloud. If it is determined that further improvement is needed for the latest improved feature point cloud, the latest improvement conversion coefficient is used to perform the inverse orthorectifier on the latest improved feature point cloud to calculate a new inverse transformed feature point cloud, the latest improvement conversion coefficient is updated based on the latest improved positional shift amount to obtain a new improvement conversion coefficient, the new improvement conversion coefficient is used to perform the orthorectifier on the new inverse transformed feature point cloud to calculate a new improved feature point cloud, and a new improved positional shift amount is calculated, which is the positional shift amount of the new improved feature point cloud relative to the reference feature point cloud. The information processing method according to claim 5.

7. If it is determined that further improvement to the latest improved feature point cloud is unnecessary, the latest improvement conversion coefficient is updated based on the latest improvement position shift amount, and the updated improvement conversion coefficient is used to perform orthorectification on the aerial image. The information processing method according to claim 6.

8. Using the basic transformation coefficients, the orthorectification is performed on the aerial image to create the orthoimage, the basic feature point cloud and the reference feature point cloud are extracted from the orthoimage and the reference image, and the basic positional displacement is calculated using the basic feature point cloud and the reference feature point cloud. Based on the aforementioned basic positional displacement, it is determined whether or not improvement of the basic feature point cloud is necessary. If it is determined that improvement of the basic feature point cloud is necessary, the initial amount of the improvement position shift is calculated. If it is determined that improvement of the basic feature point cloud is unnecessary, the basic positional displacement amount is used to perform alignment correction on the orthoimage. The information processing method according to claim 7.

9. The improvement process involves performing an inverse orthorectification on a group of basic feature points, which are multiple feature points in an orthoimage corresponding to the aerial image, using basic transformation coefficients, which are orthorectification coefficients of the aerial image, to calculate an inverse transformation feature point group corresponding to the basic feature point group. The basic transformation coefficients are updated based on the basic positional displacement amount, which is the positional displacement amount of the basic feature point group relative to a group of reference feature points, which are multiple feature points in a reference image, to obtain improved transformation coefficients. An orthorectification is performed on the inverse transformation feature point group using the improved transformation coefficients to calculate an improved feature point group corresponding to the basic feature point group. The positional displacement amount of the improved feature point group relative to the reference feature point group is calculated as the improved positional displacement amount obtained by improving the basic feature point group. An information processing program that causes a computer to execute something.

10. The information processing program includes an improvement determination process that determines whether or not improvement is necessary for the latest improved feature point cloud based on the latest improved positional displacement amount. The improvement process, when it is determined that improvement is needed for the latest improved feature point cloud, involves performing the inverse orthorectification on the latest improved feature point cloud using the latest improvement conversion coefficient to calculate a new inverse transformed feature point cloud, updating the latest improvement conversion coefficient based on the latest improved positional shift amount to obtain a new improvement conversion coefficient, performing the orthorectification on the new inverse transformed feature point cloud using the new improvement conversion coefficient to calculate a new improved feature point cloud, and calculating a new improved positional shift amount which is the positional shift amount of the new improved feature point cloud relative to the reference feature point cloud. The information processing program according to claim 9.

11. The aforementioned information processing program includes image correction processing, The improvement process updates the latest improvement conversion coefficient based on the latest improvement position shift amount if it is determined that no further improvement is needed in the latest improvement feature point cloud. The image correction process performs an orthorectification on the aerial image using the updated improvement conversion coefficients if it is determined that no further improvement is needed for the latest improved feature point cloud. The information processing program according to claim 10.

12. The information processing program includes basic processing that involves performing the orthorectification on the aerial image using the basic transformation coefficients to create the orthoimage, extracting the basic feature point cloud and the reference feature point cloud from the orthoimage and the reference image, and calculating the basic positional displacement amount using the basic feature point cloud and the reference feature point cloud. The improvement judgment process determines whether or not improvement of the basic feature point cloud is necessary based on the basic positional displacement amount. The improvement process calculates the initial improvement position shift amount when it is determined that improvement of the basic feature point cloud is necessary. The image correction process performs alignment correction on the orthomosaic image using the basic positional displacement amount when it is determined that improvement of the basic feature point cloud is unnecessary. The information processing program according to claim 11.

Images