Method and apparatus for generating three-dimensional digital surface model using multi-view satellite image

Abstract

A method for generating a three-dimensional digital surface model of the present invention may comprise the steps of: acquiring a multi-view satellite image of a target place; estimating pose information of a satellite camera that captured the multi-view satellite image on the basis of the multi-view satellite image; generating a primary digital surface model of the target place on the basis of the multi-view satellite image and the pose information of the satellite camera; restoring the distortion of the primary digital surface model by means of a three-dimensional homography transformation matrix; calculating a virtual RPC for the multi-view satellite image on the basis of the multi-view satellite image and the three-dimensional homography transformation matrix; and generating a secondary digital surface model of the target place by using the multi-view satellite image and the virtual RPC.

Description

Method and device for generating a three-dimensional digital surface model using multi-point satellite images

[0001] The present invention relates to a method for generating a three-dimensional digital surface model using a multi-view satellite image, and more specifically, to a method for generating a three-dimensional digital surface model by generating a virtual RPC from a satellite image without RPC information.

[0002] Typically, a Digital Surface Model (DSM) is a numerical representation of the three-dimensional shape of a surface object. Because it represents the three-dimensional shape of the surface, it requires the use of multiple satellite images. For example, by providing multiple satellite images and their associated Rational Polynomial Coefficients (RPCs), a three-dimensional Digital Surface Model (DSM) of a surface object can be created.

[0003] However, acquiring high-resolution satellite imagery for a given area of ​​the Earth's surface is expensive, requiring purchases from companies that own the satellite imagery rights. Furthermore, existing satellite imagery vendors do not possess multi-point imagery for specific locations on Earth, requiring direct requests for imagery or RPC information from satellite-owning companies. This, in turn, incurs significant costs in creating 3D digital surface models.

[0004] The present invention was derived from research conducted as part of the individual basic research of the Ministry of Science and ICT (Project Unique Number: 1711192392, Project Number: 2021R1A2C2009722, Project Management Agency: National Research Foundation of Korea, Research Project Name: Development of High-Precision, High-Density 3D DSM Generation Technology for Metropolitan Areas Using Dasim-Dailsi Satellite Images, Contribution Rate: 60 / 100, Project Performing Organization Name: Kyungpook National University, Research Period: 2021.03.01~2024.02.29).

[0005] In addition, the present invention was derived from research conducted as part of the Ministry of Education's Science and Engineering Academic Research Infrastructure Construction Project (Project Unique Number: 1345365803, Project Number: 2021R1A6A1A03043144, Project Management Agency: National Research Foundation of Korea, Research Project Name: ICT.Automotive Convergence Research Center, Contribution Rate: 40 / 100, Project Performing Organization Name: Kyungpook National University, Research Period: 2021.06.01~2030.05.31).

[0006] Meanwhile, the Korean government, which provided the task, has no property interest in any aspect of the present invention.

[0007] One object of the present invention is to provide a method and device for generating a three-dimensional digital surface model that can generate a three-dimensional digital surface model using multi-view satellite images for which RPC is not provided.

[0008] According to one embodiment of the present invention, a method and device for generating a three-dimensional digital surface model can be provided that can generate a three-dimensional digital surface model using a multi-view satellite image for which RPC is not provided.

[0009] FIG. 1 is a block diagram showing the configuration of a device for generating a three-dimensional numerical surface model using a multi-point satellite image according to one embodiment.

[0010] Figure 2 is a schematic diagram of a process for generating a three-dimensional numerical surface model using multi-point satellite images according to one embodiment.

[0011] FIG. 3 is a diagram showing an example of a three-dimensional inverse depth space generated according to an embodiment.

[0012] Figure 4 is a drawing for explaining distortion restoration of a first-order numerical surface model in one embodiment.

[0013] FIG. 5 is a diagram showing a state in which a first-order numerical surface model in an SFM coordinate system is converted into a geographic coordinate system through 3D homography transformation in one embodiment.

[0014] Figure 6 is a diagram showing a state in which three-dimensional points of a geographic coordinate system are projected into an image coordinate system in one embodiment.

[0015] Figure 7 is a diagram comparing a re-view satellite image, a first-order digital surface model, and a second-order digital surface model in one embodiment.

[0016] FIG. 8 is a block diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments.

[0017] A method for generating a three-dimensional digital surface model according to one embodiment is a three-dimensional digital surface model generating method performed by at least one processor, the method comprising: acquiring a multi-view satellite image of a target location; estimating attitude information of a satellite camera that captured the multi-view satellite image based on the multi-view satellite image; generating a first digital surface model of the target location based on the multi-view satellite image and the attitude information of the satellite camera; restoring distortion of the first digital surface model using a three-dimensional homography transformation matrix; calculating a virtual RPC (Rational Polynomial Coefficient) for the multi-view satellite image based on the multi-view satellite image and the three-dimensional homography transformation matrix; and generating a second digital surface model of the target location using the multi-view satellite image and the virtual RPC.

[0018] Here, the attitude information of the satellite camera is estimated through the SFM (Shape from Motion) method, and the restoring step may be a step of generating the 3D homography transformation matrix by performing a 3D homography transformation to georeferencing the SFM coordinate system, which is the coordinate system of the first numerical surface model, to a geographic coordinate system.

[0019] Here, the step of calculating the virtual RPC may be a step of selecting corresponding points between the geographic coordinate system and the image coordinate system of the re-point satellite image, and calculating the virtual RPC using the corresponding points.

[0020] Here, the step of calculating the virtual RPC may include the steps of: setting a number of 3D points greater than or equal to a preset number in the geographic coordinate system; converting the 3D points into the space of the SFM coordinate system through an inverse matrix of the 3D homography transformation matrix; and projecting the 3D points into the image coordinate system of the satellite image using attitude information of the satellite camera.

[0021] Here, the above virtual RPC can be expressed by the following mathematical formula.

[0022] (mathematical formula)

[0023]

[0024] : Corresponding points in the geographic coordinate system, : Corresponding points in the image coordinate system (SFM coordinate system), : A singular value decomposition function that inputs a preset number of corresponding points in the geographic coordinate system and the image coordinate system and outputs a matrix of a preset size, r1,01 ~ r4,20: RPC information

[0025] Here, the coordinates of the corresponding point in the geographic coordinate system and the coordinates of the image coordinate system can be expressed by the following mathematical formula.

[0026] (mathematical formula)

[0027]

[0028]

[0029] x, y: Coordinates in the image coordinate system, x_o, y_o: Offset for normalizing the image coordinate system, x_s, y_s: Scale information for normalizing the image coordinate system, V, W: Longitude and altitude coordinates in the geographic coordinate system

[0030]

[0031] According to one embodiment, a three-dimensional digital surface model generating device may include: an image acquisition module that acquires a multi-view satellite image of a target location; a first digital surface model generating module that estimates attitude information of a satellite camera that captured the multi-view satellite image based on the multi-view satellite image and generates a first digital surface model of the target location based on the multi-view satellite image and the attitude information of the satellite camera; a restoration module that restores distortion of the first digital surface model using a three-dimensional homography transformation matrix; a virtual RPC (Rational Polynomial Coefficient) generating module that generates a virtual RPC for the multi-view satellite image based on the multi-view satellite image and the three-dimensional homography transformation matrix; and a second digital surface model generating module that generates a second digital surface model of the target location using the multi-view satellite image and the virtual RPC.

[0032] Here, the attitude information of the satellite camera is estimated through the SFM (Shape from Motion) method, and the restoration module can perform a 3D homography transformation to georeferencing the SFM coordinate system, which is the coordinate system of the first numerical surface model, to a geographic coordinate system, thereby generating the 3D homography transformation matrix.

[0033] Here, the virtual RPC calculation module can select corresponding points between the geographic coordinate system and the image coordinate system of the re-point satellite image, and calculate the virtual RPC using the corresponding points.

[0034] Here, the virtual RPC output module sets a number of 3D points greater than or equal to a preset number in the geographic coordinate system, transforms the 3D points into the space of the SFM coordinate system through an inverse matrix of the 3D homography transformation matrix, and projects the 3D points into the image coordinate system of the satellite image using attitude information of the satellite camera.

[0035] Here, the above virtual RPC can be expressed by the following mathematical formula.

[0036] (mathematical formula)

[0037]

[0038] : Corresponding points in the geographic coordinate system, : Corresponding points in the image coordinate system (SFM coordinate system), : A singular value decomposition function that inputs a preset number of corresponding points in the geographic coordinate system and the image coordinate system and outputs a matrix of a preset size, r1,01 ~ r4,20: RPC information

[0039] Here, the coordinates of the corresponding point in the geographic coordinate system and the coordinates of the image coordinate system can be expressed by the following mathematical formula.

[0040] (mathematical formula)

[0041]

[0042]

[0043] x, y: Coordinates in the image coordinate system, x_o, y_o: Offset for normalizing the image coordinate system, x_s, y_s: Scale information for normalizing the image coordinate system, V, W: Longitude and altitude coordinates in the geographic coordinate system

[0044] Since the embodiments described in this specification are intended to clearly explain the idea of ​​the present invention to a person having ordinary skill in the art to which the present invention pertains, the present invention is not limited to the embodiments described in this specification, and the scope of the present invention should be interpreted to include modified or altered examples that do not depart from the idea of ​​the present invention.

[0045] The terms used in this specification have been selected from widely used terms, taking into account their functions in the present invention. However, these terms may vary depending on the intentions of those skilled in the art, precedents, or the emergence of new technologies in the technical field to which the present invention pertains. However, if a specific term is defined and used with an arbitrary meaning, the meaning of that term will be described separately. Therefore, the terms used in this specification should be interpreted based on the actual meaning of the term and the overall content of this specification, rather than simply the name of the term.

[0046] The drawings attached to this specification are intended to facilitate explanation of the present invention, and the shapes depicted in the drawings may be exaggerated as necessary to help understand the present invention, and therefore the present invention is not limited by the drawings.

[0047] In cases where it is determined that a specific description of the composition or function of a public notice related to the present invention in this specification may obscure the gist of the present invention, a detailed description thereof will be omitted as necessary.

[0048]

[0049] FIG. 1 is a block diagram showing the configuration of a device for generating a 3D numerical surface model using a multi-point satellite image according to one embodiment of the present invention, and FIG. 2 is a drawing schematically showing a process for generating a 3D numerical surface model using a multi-point satellite image according to one embodiment of the present invention.

[0050] Referring to FIGS. 1 and 2, a numerical surface model generation device (100) may include an image acquisition module (102), a first numerical surface model generation module (104), a restoration module (106), a virtual RPC (Rational Polynomial Coefficient) calculation module (108), and a second numerical surface model generation module (110).

[0051] The image acquisition module (102) can acquire a multi-point phase image of a target location for generating a three-dimensional digital surface model (DSM) of the ground surface (S1 of FIG. 2). In the disclosed embodiment, the digital surface model is generated using a phase image for which a rational polynomial coefficient (RPC) is not provided.

[0052] For example, the image acquisition module (102) can acquire multi-point satellite images of a target location from a satellite image database that anyone can access and acquire, such as Google Earth. Images purchased through a satellite image sales company are provided with RPCs, but multi-point satellite images published on the Internet, such as Google Earth, are not provided with RPCs. In the disclosed embodiment, a digital surface model can be created using multi-point satellite images that do not provide RPCs.

[0053] Meanwhile, Google Earth provides images of the Earth's surface based on the WGS84 (World Geodetic System 84) coordinate system. Furthermore, Google Earth assumes that the satellite camera is looking straight down at the Earth's surface, providing images of the Earth's surface equivalent to ortho-images. However, in reality, most satellites that acquire Earth's surface images are not located directly above the Earth's surface, but rather capture images of the Earth's surface from an oblique angle. Satellite cameras capture images not only of the Earth's surface directly below, but also by shifting their viewpoint to capture images of the Earth's surface from different angles as seen from the satellite. Therefore, for objects with altitude, the shape of the object appears tilted due to the camera's shooting angle. Even when satellites capture images of objects in the same area, the viewpoint of the satellite camera changes depending on the shooting time, causing the object to appear tilted at different angles.

[0054] The multi-point satellite image (50) of Fig. 2 is a satellite image acquired by capturing the same surface area in Google Earth by the image acquisition module (102). The image acquisition module (102) can acquire multiple multi-point satellite images (50) by changing the shooting date and time of the satellite camera in a satellite image database such as Google Earth and capturing the screen.

[0055] The first digital surface model generation module (104) can estimate the attitude information (internal attitude information and external attitude information) of the satellite camera based on the multi-view satellite image (50) (S2 of FIG. 2). At this time, the first digital surface model generation module (104) can estimate the external attitude of the satellite camera using the optimization method of the SFM (Shape from Motion) algorithm. In addition, the internal attitude of the satellite camera can be set to follow the Perspective Projection Model or the estimated value of the SFM algorithm can be applied. The SFM (Shape from Motion) method is a method of calculating the external attitude information and internal attitude information of the camera of each image under the assumption that any plurality of images are generated through a pinhole camera model.

[0056] The first numerical surface model generation module (104) can generate an inverse depth image (60) using the MVS (Multi-View Stereo matching) technology based on the estimated satellite camera attitude information and the multi-view satellite image (50), and then generate a first numerical surface model based on the inverse depth image (60) and the satellite camera attitude information (S3 of FIG. 2).

[0057] Specifically, the first digital surface model generation module (104) can define a three-dimensional inverse depth space using the estimated satellite camera attitude information and multi-view satellite images (50), and calculate a multi-view stereo matching cost in the three-dimensional inverse depth space. The first digital surface model generation module (104) can input n multi-view images using the EnSoft3D method, which is one of the MVS (Multi-View Stereo matching) technologies, calculate a stereo matching cost using n inverse depth spaces, and generate n inverse depth images as a result.

[0058] FIG. 3 is a diagram showing an example of a 3D inverse depth space generated according to an embodiment of the present invention. Here, the inverse depth space (90) can be composed of the x-axis and y-axis of the image and the d-axis representing the inverse depth. Accordingly, a point of the inverse depth space (90) can be expressed as (x, y, d), and the internal attitude information and external attitude information of the satellite camera can be used in the process of converting (x, y, d) into an actual depth space. Finding the (x, y) coordinates of the same object point among multi-view 2D images (i.e., multi-view phase images) is the same as the multi-view stereo matching (MVS) problem, and (x, y, d) with the minimum multi-view stereo matching cost of the inverse depth space (90) can be determined as the object point on the ground surface.

[0059] The first numerical surface model generation module (104) can generate an inverse depth image (60) by finding coordinates (x, y, d) that minimize the multi-point stereo matching cost in the inverse depth space and determining the coordinates (x, y, d) as object points on the ground surface. In addition, the first numerical surface model generation module (104) can generate a first numerical surface model by calculating an actual depth value from the satellite camera to the ground surface object using the depth value (d value) of the inverse depth image (60) and the attitude information of the satellite camera.

[0060] The restoration module (106) can restore the distortion of the primary digital surface model (S4 of FIG. 2). FIG. 4 is a diagram for explaining the distortion restoration of the primary digital surface model in one embodiment of the present invention. Referring to FIG. 4, since the satellite camera (62) is mostly not located in the vertical direction of the ground surface, the primary digital surface model (66, 72) generated based on the attitude information of the satellite camera is distorted (deformed) compared to the shape of the actual ground surface object. At this time, since the satellite camera is assumed to have a perspective projection model, the primary digital surface model generated from the inverse depth space becomes an asymmetric hexahedral space with distortion. Accordingly, the restoration module (106) can convert the distorted primary digital surface model into the coordinate system of the satellite camera (64) which is assumed to be located vertically above the ground surface, thereby generating the primary digital surface model (68, 74) with the distortion restored.

[0061] The restoration module (106) can restore the inverse depth space based on a satellite camera (virtual satellite camera) (64) assumed to be located vertically above the Earth's surface. Here, since the virtual satellite camera assumes an ortho-image without perspective projection distortion, the camera model can be regarded as an orthographic projection model.

[0062] The restoration module (106) can produce a 3D homography by using a plurality of corner points of the previously generated 1st digital surface model (66, 72) and a plurality of corner points of the digital surface model (model in which the object is orthogonally projected) (68, 74) generated based on the virtual satellite camera to convert the distorted 1st digital surface model into a virtual satellite camera coordinate system without distortion.

[0063] Here, the coordinate system of the primary digital surface model generated using the attitude information of the satellite camera estimated by the SFM (Shape-from-Motion) method may be referred to as the SFM coordinate system. The restoration module (106) may perform a 3D homography transformation to correct the distorted primary digital surface model and georeferencing it to a geographic coordinate system (e.g., WGS84, etc.). The geographic coordinate system may be a coordinate system of the digital surface model generated based on a virtual satellite camera (64). The 3D homography transformation is a transformation that represents a mapping relationship between two 3D coordinate systems and has the form of a matrix.

[0064] The restoration module (106) can search for eight corresponding points between the SFM coordinate system and the geographic coordinate system for 3D homography transformation. The restoration module (106) can use each vertex of the asymmetric hexahedral space in the SFM coordinate system and each vertex of the rectangular hexahedral space in the geographic coordinate system. At this time, the 3D homography transformation matrix (H) can be expressed by the following mathematical expression 1.

[0065] (mathematical formula)

[0066]

[0067]

[0068] SVD: Function using Singular Value Decomposition

[0069] : Corresponding points in the SFM coordinate system

[0070] : Corresponding points of WGS coordinate system (geographic coordinate system)

[0071] Here, if P(p_x, p_y, p_z) is a point in the SFM coordinate system, the corresponding point (P') in the WGS coordinate system can be expressed by the following mathematical expression 2.

[0072] (Equation 2)

[0073]

[0074] FIG. 5 is a diagram illustrating a state in which a first-order numerical surface model in an SFM coordinate system is converted into a geographic coordinate system through a 3D homography transformation in one embodiment of the present invention. Through this, the distortion of the first-order numerical surface model is restored to a certain extent.

[0075] Meanwhile, even if the distortion is restored by the restoration module (106), the first-order numerical surface model is generated without a RPC (Rational Polynomial Coefficient), and thus has limitations in its accuracy and precision. In the disclosed embodiment, a virtual RPC is calculated for a multi-view satellite image, and then the first-order numerical surface model is reconstructed based on the virtual RPC to generate a second-order numerical surface model.

[0076] The virtual RPC (Rational Polynomial Coefficient) calculation module (108) can calculate a virtual RPC for a multi-view satellite image. The RPC refers to the coefficient of a polynomial that can obtain pixel information within a satellite image when latitude, altitude, and altitude are input. The relationship between the normalized pixel coordinates (X, Y) in the satellite image, the coordinates (U (latitude), V (longitude), W (altitude)) of a normalized geographic coordinate system (e.g., WGS84), and the RPC is expressed by mathematical equation 3.

[0077] (Equation 3)

[0078]

[0079]

[0080]

[0081]

[0082]

[0083] Here, represents the offset for normalizing the geographic coordinate system, represents scale information for normalization of the geographic coordinate system. And, represents the offset for normalization of the image coordinate system, represents scale information for normalization of the image coordinate system.

[0084] Also, each polynomial function P i r included in i are RPC information, and there are 20 RPC information for one polynomial function (i.e. r i01 ~ r i20 ), so there are 80 RPC information in total for four polynomial functions (P1, P2, P3, P4). Among these, r in P2 and P4 201 and r 401 Since is fixed to 1, the number of coefficients to be found among 80 RPC information is 78, and therefore, at least 39 correspondence points between geographic coordinates and image coordinates are required. In order to find correspondence points, it is necessary to be able to calculate the coordinates of 3D points existing in the geographic coordinate system from the satellite image of the multi-point, and the projection relationship between the image coordinate system of each image in the geographic coordinate system must be known.

[0085] Specifically, the virtual RPC output module (108) can select matching points (i.e., corresponding points) between the three-dimensional primary digital surface model and the two-dimensional multi-view satellite image (S5 of FIG. 2). The virtual RPC output module (108) can divide the rectangular prism space of the geographic coordinate system (WGS84) into equal intervals in width, length, and height in order to search for uniform corresponding points. Here, the geographic coordinate system is a geographic coordinate system used when restoring the first digital surface model, and the virtual RPC output module (108) can divide the rectangular prism space in the geographic coordinate system in which the distortion of the first digital surface model is restored into a plurality of spaces. For example, the virtual RPC output module (108) can divide the rectangular prism space of the geographic coordinate system into ten equal parts in width, length, and height, and set 100 three-dimensional points. At this time, 39 or more three-dimensional points of the geographic coordinate system can be set.

[0086] The virtual RPC output module (108) converts the set 3D points of the geographic coordinate system into the inverse matrix of the 3D homography transformation (H -1 ) is converted into the space of the SFM coordinate system, and then the SFM pinhole camera attitude information is used to project it into the image coordinate system of each satellite image to create a virtual RPC (S6 in Fig. 2).

[0087] Figures 6 to 8 are diagrams showing a state in which 3D points of a geographic coordinate system are projected onto an image coordinate system in one embodiment of the present invention. Figure 6 shows 3D points sampled in a geographic coordinate system, Figure 7 shows them transformed into an SFM coordinate system through an inverse matrix of a 3D homography transformation, and Figure 8 shows them projected onto image coordinates using SFM pinhole camera attitude information.

[0088] Here, the generation of virtual RPCs using the correspondence points between the uniformly searched geographic coordinate system and the image coordinate system can be expressed as Equation 4 using SVD in the same way as calculating the homography transformation matrix.

[0089] (Equation 4)

[0090]

[0091] : Corresponding points in the geographic coordinate system, : Corresponding points in the image coordinate system (SFM coordinate system), : A singular value decomposition function that inputs a preset number of corresponding points in the geographic coordinate system and the image coordinate system and outputs a matrix of a preset size, r1,01 ~ r4,20: RPC information

[0092] Here, SVD r is a function for calculating virtual RPC, for example, it can be a singular value decomposition function that takes 100 points as input in each coordinate system and outputs a matrix of size 4×20. In this case, each row of the output matrix is ​​divided into P i can be used as a coefficient.

[0093] In addition, if the image coordinates of the satellite image are (x, y) and the coordinates of the normalized geographic coordinate system are (U (latitude), V (longitude), W (altitude)), the coordinates of the satellite image and the coordinates of the geographic coordinate system using the virtual RPC can be expressed by the following mathematical expression 5.

[0094] (Equation 5)

[0095]

[0096]

[0097] x, y: Coordinates in the image coordinate system, x_o, y_o: Offset for normalizing the image coordinate system, x_s, y_s: Scale information for normalizing the image coordinate system, V, W: Longitude and altitude coordinates in the geographic coordinate system

[0098] The secondary digital surface model generation module (110) can generate a secondary digital surface model using a virtual RPC as input from a multi-view satellite image. That is, the secondary digital surface model generation module (110) can generate a secondary digital surface model by applying a virtual RPC to a multi-view satellite image. At this time, the secondary digital surface model generation module (110) can generate a secondary digital surface model using the MVS (Multi-View Stereo matching) technique. Since the generation of a digital surface model using RPC information is a known technology, a detailed description thereof will be omitted. Here, since the secondary digital surface model is generated using the virtual RPC information, it becomes a more accurate model than the primary digital surface model.

[0099] FIG. 9 is a diagram comparing a multi-view satellite image, a first-order digital surface model, and a second-order digital surface model according to an embodiment of the present invention. 201, which represents the left vertical axis diagram of FIG. 9, is a multi-view satellite image, 202, which represents the middle vertical axis diagram of FIG. 9, is the first-order digital surface model, and 203, which represents the right vertical axis diagram of FIG. 9, is the second-order digital surface model. As can be seen in FIG. 9, the second-order digital surface model is more accurate and has improved performance than the first-order digital surface model.

[0100] According to the disclosed embodiment, a first digital surface model is created using a multi-view satellite image for which Rational Polynomial Coefficient (RPC) information is not provided, a virtual RPC is created using a 3D homography transformation used in the restoration of the first digital surface model, and then a second digital surface model is created using the same, thereby enabling the digital surface model to be reconstructed as if RPC information were provided even using a multi-view satellite image for which RPC information is not provided.

[0101]

[0102] FIG. 10 is a block diagram illustrating a computing environment (10) including a computing device suitable for use in exemplary embodiments. In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components other than those described below.

[0103] The illustrated computing environment (10) includes a computing device (12). In one embodiment, the computing device (12) may be a future event prediction device (100, 210). Furthermore, the computing device (12) may be an observation bot (202). Furthermore, the computing device (12) may be a future behavior prediction server (204).

[0104] A computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) may cause the computing device (12) to operate according to the exemplary embodiments mentioned above. For example, the processor (14) may execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, which, when executed by the processor (14), may be configured to cause the computing device (12) to perform operations according to the exemplary embodiments.

[0105] A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data, and / or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by the processor (14). In one embodiment, the computer-readable storage medium (16) may be a memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, any other form of storage medium that can be accessed by the computing device (12) and store desired information, or a suitable combination thereof.

[0106] A communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and computer-readable storage media (16).

[0107] The computing device (12) may also include one or more input / output interfaces (22) that provide interfaces for one or more input / output devices (24) and one or more network communication interfaces (26). The input / output interfaces (22) and the network communication interfaces (26) are connected to the communication bus (18). The input / output devices (24) may be connected to other components of the computing device (12) via the input / output interfaces (22). Exemplary input / output devices (24) may include input devices such as pointing devices (such as a mouse or a trackpad), a keyboard, a touch input device (such as a touchpad or a touchscreen), a voice or sound input device, various types of sensor devices and / or photographing devices, and / or output devices such as display devices, printers, speakers and / or network cards. The exemplary input / output devices (24) may be included within the computing device (12) as a component constituting the computing device (12), or may be connected to the computing device (12) as a separate device distinct from the computing device (12).

[0108]

[0109] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment or may be those known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of the program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

[0110] Although the embodiments described above have been described by way of limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above teachings. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0111] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. A method for generating a three-dimensional numerical surface model performed by at least one processor, A step of acquiring re-point satellite images of a target location; A step of estimating attitude information of a satellite camera that captured the re-pointing satellite image based on the re-pointing satellite image; A step of generating a first numerical surface model of the target location based on the re-point satellite image and the attitude information of the satellite camera; A step of restoring the distortion of the first-order numerical surface model through a three-dimensional homography transformation matrix; A step of calculating a virtual RPC (Rational Polynomial Coefficient) for the re-point satellite image based on the re-point satellite image and the 3D homography transformation matrix; and A step of generating a second numerical surface model of the target location using the above re-point satellite image and the virtual RPC. A method for creating a three-dimensional numerical surface model.

2. In claim 1, The attitude information of the above satellite camera is estimated through the SFM (Shape from Motion) method. The above restoration steps are: A step of generating the 3D homography transformation matrix by performing a 3D homography transformation to georeferencing the SFM coordinate system, which is the coordinate system of the above 1st numerical surface model, to a geographic coordinate system. A method for creating a three-dimensional numerical surface model.

3. In claim 2, The steps for generating the above virtual RPC are: A step of selecting corresponding points between the above geographic coordinate system and the image coordinate system of the above re-point satellite image, and calculating the virtual RPC using the corresponding points. A method for creating a three-dimensional numerical surface model.

4. In claim 3, The steps for generating the above virtual RPC are: A step of setting a three-dimensional point number or more than a preset number in the above geographic coordinate system; A step of transforming the above 3D points into the space of the SFM coordinate system through the inverse matrix of the above 3D homography transformation matrix; and A step of projecting the three-dimensional points into the image coordinate system of the satellite image using the attitude information of the satellite camera. A method for creating a three-dimensional numerical surface model.

5. In claim 3, The above virtual RPC is expressed by the following mathematical formula: (mathematical formula) : Corresponding points in the geographic coordinate system, : Corresponding points in the image coordinate system (SFM coordinate system), : Singular value decomposition function that inputs a preset number of corresponding points in the geographic coordinate system and the image coordinate system and outputs a matrix of a preset size, r1,01 ~ r4,20: RPC information A method for creating a three-dimensional numerical surface model.

6. In claim 5, The coordinates of the corresponding points in the above geographic coordinate system and the coordinates in the image coordinate system are expressed by the following mathematical formulas: (mathematical formula) x, y: Coordinates in the image coordinate system, x_o, y_o: Offset for normalization of the image coordinate system, x_s, y_s: Scale information for normalization of the image coordinate system, V, W: Longitude and altitude coordinates in the geographic coordinate system A method for creating a three-dimensional numerical surface model.

7. Image acquisition module for acquiring re-point satellite images of the target location; A first digital surface model generation module for estimating attitude information of a satellite camera that captured the multi-point satellite image based on the multi-point satellite image, and generating a first digital surface model of the target location based on the multi-point satellite image and the attitude information of the satellite camera; A restoration module that restores the distortion of the first-order numerical surface model through a three-dimensional homography transformation matrix; A virtual RPC calculation module for calculating a virtual RPC (Rational Polynomial Coefficient) for the re-point satellite image based on the re-point satellite image and the 3D homography transformation matrix; and A second digital surface model generation module is included that generates a second digital surface model of the target location using the above re-point satellite image and the virtual RPC. A device for generating three-dimensional numerical surface models.

8. In claim 7, The attitude information of the above satellite camera is estimated through the SFM (Shape from Motion) method. The above restoration module, In order to georeferencing the SFM coordinate system, which is the coordinate system of the above first-order numerical surface model, to a geographic coordinate system, a three-dimensional homography transformation is performed to generate the three-dimensional homography transformation matrix. A device for generating three-dimensional numerical surface models.

9. In claim 8, The above virtual RPC output module is: Select corresponding points between the above geographic coordinate system and the image coordinate system of the above re-point satellite image, and derive the virtual RPC using the corresponding points. A device for generating three-dimensional numerical surface models.

10. In claim 9, The above virtual RPC output module is: A method of setting a number of 3D points greater than or equal to a preset number in the above geographic coordinate system, transforming the 3D points into the space of the SFM coordinate system through an inverse matrix of the 3D homography transformation matrix, and projecting the 3D points into the image coordinate system of the satellite image using attitude information of the satellite camera. A device for generating three-dimensional numerical surface models.

11. In claim 9, The above virtual RPC is expressed by the following mathematical formula: (mathematical formula) : Corresponding points in the geographic coordinate system, : Corresponding points in the image coordinate system (SFM coordinate system), : Singular value decomposition function that inputs a preset number of corresponding points in the geographic coordinate system and the image coordinate system and outputs a matrix of a preset size, r1,01 ~ r4,20: RPC information A device for generating three-dimensional numerical surface models.

12. In claim 11, The coordinates of the corresponding points in the above geographic coordinate system and the coordinates in the image coordinate system are expressed by the following mathematical formulas: (mathematical formula) x, y: Coordinates in the image coordinate system, x_o, y_o: Offset for normalization of the image coordinate system, x_s, y_s: Scale information for normalization of the image coordinate system, V, W: Longitude and altitude coordinates in the geographic coordinate system A device for generating three-dimensional numerical surface models.

Images