A system and method for geometric correction of remote sensing images based on AIS (Automatic Identification System)

Abstract

A geometric correction system for remote sensing images based on AIS according to an embodiment of the present invention comprises: a receiving unit that receives remote sensing images and AIS (Automatic Identification System) data; a processing unit that corrects the received AIS data to the AIS data at the time of capture (ts) of the remote sensing images and matches the ship in the remote sensing images based on the position coordinates of the corrected AIS data to match the information of the matched ship; and an output unit that outputs an image in which geometric correction is performed by the processing unit, wherein the processing unit defines the ship in which the corrected AIS data is matched as a maritime reference point and performs geometric correction by converting the coordinates of the image in which the maritime reference point is located into the actual coordinates of the corrected AIS data.

Description

Technology Field

[0001] The present invention relates to a geometric correction system for remote sensing images based on an Automatic Identification System (AIS) and a method thereof. Background Technology

[0002] AIS (Automatic Identification System, hereinafter referred to as “AIS”) is an automatic ship identification system. Developed as a tool to prevent ship collisions, AIS enables the identification of the locations and travel paths of different vessels and allows for mutual identification.

[0003] Meanwhile, according to the International Maritime Organization (IMO) Ship Safety Act, ships exceeding a certain weight are required to install AIS.

[0004] Generally, the above AIS transmits the collected information to ground receivers or satellite receivers at intervals of seconds to minutes, depending on the vessel's speed of movement.

[0005] Meanwhile, remote sensing can be understood as a technology that observes the Earth's surface based on images from satellites or aircraft.

[0006] In other words, remote sensing imagery can be defined as images of the Earth's surface acquired by satellites, aircraft, etc. Here, the aforementioned Earth surface imagery can be understood in a broad sense that includes not only continental and oceanic crust in a narrow sense but also the ocean. Therefore, remote sensing naturally includes ocean exploration, and for example, the technology of tracking ships passing through the ocean using satellite imagery can also be considered remote sensing.

[0007] As is already known, geometric correction is essential for remote sensing imagery to eliminate geometric distortions occurring during capture and to make it closer to reality. Here, Ground Control Points (hereinafter referred to as “GCPs” or “Ground Control Points”) are widely used as the method for such geometric correction.

[0008] The above ground reference point can be understood as a point that is clearly identifiable within the remote sensing imagery and whose accurate latitude, longitude, and altitude coordinates are already known.

[0009] Generally, ground control points are established at intersections, building corners, steel towers, etc., that can be clearly and distinctly identified in the image, and it must be possible to obtain the precise coordinates, such as latitude, longitude, and altitude, of the aforementioned ground control points using GPS surveying, existing orthophotos, or digital elevation models.

[0010] However, the above-mentioned method of using ground reference points is difficult to apply in ocean exploration that observes the sea. In particular, in ocean environments where it is difficult to find objects to set as reference points and where it is very difficult for people to conduct actual surveys, the above-mentioned ground reference point method will be even more difficult to use. In the above-mentioned ocean environment, objects that can be considered as objects on the sea surface are very limited, such as buoys, stones, and lighthouses, so a problem may arise where it is difficult to apply the definition requirements of the above-mentioned ground reference points in the above-mentioned ocean environment.

[0011] In other words, conventional technology has the problem that it is difficult to perform geometric correction using ground control points in ocean exploration. Therefore, there is a need for a precise geometric correction method that is applicable to ocean exploration within remote sensing imagery.

[0012] Meanwhile, the information transmitted by the aforementioned AIS (hereinafter referred to as “AIS information”) includes not only static information such as the type, width, length, height, name, and IMO number (International Maritime Organization), but also dynamic information such as the vessel's position, time of information reception, course, and speed.

[0013] Accordingly, a geometric correction method is being disclosed that utilizes a vessel in a marine environment as a ground reference point by utilizing AIS information from recent remote sensing images, such as Korean registered patent 10-2723017 B1 (2024.10.23.). The aforementioned prior art document (KR 10-2723017 B1) identifies a vessel using only satellite imagery and matches the closest vessel based on position coordinates among the acquired AIS information to the AIS information.

[0014] However, the above-described method of matching AIS information of the prior art document (KR 10-2723017 B1) to a vessel has problems for the following reasons.

[0015] As mentioned above, since AIS information has intervals ranging from seconds to minutes, there is a difference between the transmission and reception times of AIS information and the capture times of remote sensing images, so conventional technology for matching the nearest vessel has an inaccuracy problem.

[0016] For example, when a large number of vessels exist at a high density in a specific marine space, errors in position coordinates due to the aforementioned time difference inevitably result in inaccurate values.

[0017] Furthermore, remote sensing imagery can also introduce errors due to various environmental influences depending on the capture method or the type of equipment used (e.g., sensors or cameras). Therefore, technology is required to evaluate which information—AIS data or remote sensing imagery—is more reliable and to reflect this in the results.

[0018] Meanwhile, information on relevant prior art literature regarding the geometric correction of remote sensing images using the aforementioned ground reference points is as follows. Prior art literature

[0019] KR 10-0571429 B1 (2006.04.10.), Method for providing an online geometric correction service using a ground control point image chip KR 10-1952326 B1 (2019.02.20.), Method for extracting Ground Control Points (GCPs) for satellite image correction KR 10-2021-0095479 A (2021.08.02.), Satellite image processing device and method for extracting ground control points thereof KR 10-2723017 B1, (2024.10.23.) Method for geometric correction of marine images using an Automatic Identification System and marine image geometric correction system The problem to be solved

[0020] The objective of the present invention is to provide a geometric correction system for remote sensing images and a method thereof that solves the problems of the aforementioned prior art. means of solving the problem

[0021] To achieve the above objective, a geometric correction system for remote sensing images according to an embodiment of the present invention may include: a receiving unit that receives remote sensing images and AIS (Automatic Identification System) data; a processing unit that corrects the received AIS data to the AIS data at the time of capture (ts) of the remote sensing images and matches the vessel in the remote sensing images based on the position coordinates of the corrected AIS data to match the information of the matched vessel; and an output unit that outputs an image in which geometric correction has been performed by the processing unit.

[0022] In addition, the processing unit may define the vessel to which the corrected AIS data is matched as a maritime reference point, and perform geometric correction by converting the coordinates of the image where the maritime reference point is located into the actual coordinates of the corrected AIS data.

[0023] In addition, the geometric correction is characterized by transforming the remote sensing image by moving, rotating, and scaling it based on the coordinates of the corrected AIS data of the vessel, which is the maritime reference point.

[0024] In addition, the processing unit may define only the vessels among those detected in the remote sensing image whose corrected AIS data matches as maritime reference points.

[0025] In addition, the above-mentioned maritime reference points are set as multiple, and geometric correction of the image can be performed based on AIS data from the multiple maritime reference points.

[0026] In addition, the processing unit is characterized by extracting an error (d) through the calculation of the Euclidean distance between two locations when there is a difference between the coordinates of the vessel in the remote sensing image and the coordinates of the corrected AIS data, and when the error (d) is greater than a preset threshold value (a), aligning the vessel coordinates in the image with the coordinates of the corrected AIS data to set them as a maritime reference point.

[0027] In addition, the threshold value (a) is defined by the reliability of the AIS data (C_AIS) and the reliability of the remote sensing image (C_VID), and the reliability of the AIS data is, It can be defined by (C_AIS: result value representing the reliability of AIS data in the range of 0 to 1, Xs and Ys: X,Y coordinates of AIS data after interpolation, Xv and Yv: X,Y coordinates of the vessel detected in the image, σ_GPS is the GPS error range value).

[0028] In addition, the reliability of the above remote sensing image is, It can be defined by (C_VID: result value representing image reliability in the range of 0 to 1, Xs and Ys are AIS data X,Y coordinates on which interpolation is performed, Xv and Yv: X,Y coordinates of the vessel detected in the image, σ_VID is the value for the total error of image reliability).

[0029] In addition, the above threshold value (a) is, It can be defined as.

[0031] In another aspect, a method for geometric correction of an AIS-based remote sensing image according to an embodiment of the present invention may include: receiving a remote sensing image and AIS data; correcting the received AIS data to the AIS data at the time of image capture (ts); searching for a vessel in the image and matching and aligning the corrected AIS data to the searched vessel; determining whether the searched vessel can be set as a maritime reference point; and performing geometric correction of the remote sensing image based on the corrected AIS data of the vessel set as the maritime reference point.

[0032] Additionally, the step of determining whether the above-mentioned searched vessel can be set as a maritime reference point is that when the coordinates of the above-mentioned searched vessel and the coordinates of the above-mentioned corrected AIS data have an error (d), the vessel can be set as a maritime reference point only if the error (d) exceeds a preset threshold value (a).

[0033] In addition, the step of correcting with AIS data at the above-mentioned video capture time (ts) is an equation considering the speed and direction of movement at the location of the AIS data received immediately before the above-mentioned video capture time (ts-1): Correction can be performed along (t_s: time of video capture, X_s: X-plane position at the time of video capture, Y_s: Y-plane position at the time of video capture, X_(AIS-1): X-plane position on the AIS data immediately before the time of video capture, Y_(AIS-1): Y-plane position on the AIS data immediately before the time of video capture, t_(AIS-1): time of reception of the AIS data received immediately before the time of video capture).

[0034] Additionally, the error (d) is extracted by calculating the Euclidean distance between two locations, and the preset threshold value (a) can be defined as a value based on the reliability of the AIS data and the reliability of the remote sensing image.

[0035] In addition, the reliability of the above remote sensing image can be calculated by summing the geometric error values ​​regarding the satellite shooting angle or Earth curvature, the hardware error values ​​regarding sensor noise or lens distortion, the error values ​​due to ship speed, and the error values ​​due to the shooting environment. Effects of the invention

[0036] According to the present invention, although it is relatively difficult to accurately and precisely identify information about each vessel using only satellite imagery, accurate and precise vessel information can be provided within the imagery by aligning AIS information.

[0037] According to the present invention, AIS information can be accurately matched at the time of capturing remote sensing images.

[0038] According to the present invention, error-causing factors in remote sensing images and error-causing factors in AIS information can be mutually offset and reflected in the output result as more accurate data, thereby improving reliability.

[0039] According to the present invention, vessels can be identified more accurately in remote sensing images by utilizing AIS information, and errors caused by the difference between the image and the time of receiving AIS information can be effectively and reliably corrected.

[0040] According to the present invention, geometric correction of remote sensing images can be performed more easily and accurately because a ground reference point is set as a ship in a marine environment such as the ocean.

[0041] According to the present invention, since AIS information can be fused (or aligned) with a vessel in remote sensing images, the problems that occur when identifying a vessel solely from conventional satellite images and the problems that occur when identifying a vessel solely based on AIS information can be mutually resolved.

[0042] According to the present invention, since the vessel in the remote sensing image includes unique identification information (e.g., IMO number), speed, and direction of movement among the AIS information, precise tracking of the actual vessel in the remote sensing image is possible and false positives can be reduced.

[0043] According to the present invention, for remote sensing images captured at a specific point in time, information at the captured specific point in time is obtained from the received AIS information using an interpolation technique, thereby offsetting errors in the ship's position and enabling more precise matching of the ship with the AIS information when a dense ship is detected in a narrow area of ​​interest.

[0044] According to the present invention, the Euclidean distance between vessels in AIS information and remote sensing images is calculated to analyze the error, and a more precise result can be output by comparing it with a threshold value set to evaluate reliability. Brief explanation of the drawing

[0045] FIG. 1 is a diagram illustrating the configuration of a geometric correction system for remote sensing images according to an embodiment of the present invention. FIG. 2 is a flowchart exemplarily showing a geometric correction method for remote sensing images according to an embodiment of the present invention. FIG. 3 is a diagram showing a screen for searching for and setting a target (ship) in an area of ​​interest (ROI) in a remote sensing image according to an embodiment of the present invention. FIG. 4 is a diagram showing a screen in which a target (ship) in a remote sensing image according to an embodiment of the present invention and AIS data matched to the target (ship) are fused (or aligned). FIG. 5 is a diagram showing a remote sensing image before geometric correction according to an embodiment of the present invention. FIG. 6 is a drawing showing a remote sensing image output after geometric correction according to an embodiment of the present invention. Specific details for implementing the invention

[0046] Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that in assigning reference numerals to the components of each drawing, the same components are given the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing the embodiments of the present invention, if it is determined that a detailed description of related known components or functions would hinder understanding of the embodiments of the present invention, such detailed description is omitted.

[0047] In addition, terms such as first, second, A, B, (a), (b), etc., may be used when describing the components of the embodiments of the present invention. These terms are intended only to distinguish the components from other components, and the essence, order, or sequence of the components is not limited by the terms. Where it is stated that a component is "connected," "combined," or "connected" to another component, it should be understood that the component may be directly connected or connected to the other component, but that another component may also be "connected," "combined," or "connected" between each component.

[0048] FIG. 1 is a diagram exemplarily showing the configuration of a geometric correction system for remote sensing images based on AIS according to an embodiment of the present invention.

[0049] For the convenience of explanation, satellite or aerial images may be referred to as “images” below, and information transmitted and received by the AIS (Automatic Identification System) may be referred to as “AIS data or AIS information.”

[0050] Referring to FIG. 1, a geometric correction system (1) for remote sensing images according to an embodiment of the present invention may include a receiving unit (10) that receives AIS data and images.

[0051] The above receiving unit (10) may include an image receiving unit (11) for receiving satellite or aerial images and a data receiving unit (13) for receiving AIS data.

[0052] The above image receiving unit (11) can receive the shooting time (ts) of the received image together. That is, the above image receiving unit (11) can acquire the shooting time (ts) of the image that serves as a time reference for acquiring AIS data.

[0053] In addition, the image receiving unit (11) can receive an image of a region of interest (ROI) that the user is interested in. For example, the image receiving unit (11) can receive a specific area of ​​the sea where multiple vessels are navigating. It can also receive the time of capture (ts) of the received image.

[0054] The data receiving unit (13) can receive AIS data for vessels in an area corresponding to the area of ​​interest (ROI) of the user. Additionally, the data receiving unit (13) can receive AIS data at a time point before and after the time of video capture (ts). For example, the data receiving unit (13) can receive AIS data at a time point ts±n (n=1,2,3…) based on the time of video capture (ts). Preferably, to reflect fast calculation speeds and real-time changing vessel information, AIS data at a time point ts-1 based on the time of video capture (ts) can be received.

[0055] In addition, the above ts±1 time point can be understood as the AIS data immediately before and immediately after ts.

[0056] Meanwhile, depending on the user's needs, AIS data of the target (vessel) after the shooting time (ts) of the above video may be received and used for analysis, so AIS data at time ts+n may also be received.

[0057] As described above, since AIS data is transmitted and received at intervals of seconds to minutes, there may be a difference between the time of video capture (ts) and the time of transmission and reception of AIS data. Therefore, the data receiving unit (13) can receive AIS data before and after the time of video capture (ts).

[0058] In addition, AIS data may include not only static information such as the type, width, length, height, name, and IMO number (International Maritime Organization), but also dynamic information such as the vessel's location, time of information reception, course, and speed.

[0059] The geometric correction system (1) for the above remote sensing image may further include a processing unit (20) that processes the received AIS data and image.

[0060] The above processing unit (20) can perform a preprocessing process to remove noise or abnormal values ​​from AIS data at a time before or after the time of shooting (ts) of satellite and aerial images.

[0061] For example, the received AIS data may contain errors such as duplicate unique information of the vessel or positional errors caused by GPS equipment malfunction. The processing unit (20) can perform a preprocessing process to verify and process such errors in the AIS data.

[0062] AIS data positioning errors may occur due to temporary errors or malfunctions in the vessel's GPS equipment. In this case, the processing unit (20) may exclude it from the target for geometric correction.

[0063] The above image may contain noise caused by floating objects, structures, waves, etc. that may appear in a marine environment. Therefore, the processing unit (20) can perform a preprocessing process to remove such noise.

[0064] In addition, the processing unit (20) can perform a process of correcting the time difference between the video capture time (ts) and the time difference caused by the AIS data transmission and reception interval.

[0065] That is, the processing unit (20) can correct the AIS data. Specifically, the processing unit (20) can process the AIS data at the video shooting time (ts) based on the AIS data received before and after the video shooting time (ts).

[0066] For example, the processing unit (20) can calculate AIS data at the time of video capture (ts) by applying a linear interpolation or velocity-based interpolation method. A detailed method for calculating this will be described later.

[0067] In addition, the processing unit (20) can search for the user's target (vessel) in the received user's region of interest (ROI) image and identify the location of the target.

[0068] In addition, the processing unit (20) can match the target and the corrected AIS data to align (or may be expressed as 'fusion') the vessel and the AIS data at the time of video capture (ts) within the image.

[0069] In addition, if there is a difference (i.e., error) between the position of the target in the image and the position of the corrected AIS data, the processing unit (20) can extract the difference (error) value through Euclidean distance calculation and calculate and compare the reliability of the image and the AIS data.

[0070] Meanwhile, the processing unit (10) may include a data processing unit (22) that preprocesses and corrects the received AIS data (22) described above, and an image processing unit (24) that preprocesses the received image described above and searches for and sets a target (ship) in the user's area of ​​interest (ROI).

[0071] The image processing unit (24) can also perform the function of matching and aligning the AIS data corrected by the data processing unit (22) to targets of the user's region of interest (ROI).

[0072] The above data processing unit (22) can also filter out reception errors, such as location errors caused by signal interference, from the received AIS data during the preprocessing process.

[0073] The above data processing unit (22) can calculate AIS data (especially dynamic information) at the time of video capture (ts) by using a linear interpolation or velocity-based interpolation method when the interval of the received AIS data is not constant.

[0074] Since the above linear interpolation is performed when the ship moves in a straight line at a constant speed, it can quickly derive a result value, making it convenient for users to utilize in large-capacity and wide-area images.

[0075] The above processing unit (20) can calculate AIS data at the time of image capture (ts) using linear interpolation with the following mathematical formula 1 when a vessel in a designated window in the user's region of interest (ROI) can be evaluated as moving in a straight line at a constant speed.

[0076]

[0077] Here, t_s is the time of video recording, X_s and Y_s are the positions (X,Y) at the time of video recording (ts), X_(AIS-1) and Y_(AIS-1) are the (X,Y) positions of the vessel in the AIS data immediately before the time of video recording (ts), X_(AIS+1) and Y_(AIS+1) are the (X,Y) positions of the vessel in the AIS data immediately after the time of video recording (ts), t_(AIS-1) is the time of reception of the AIS data received immediately before the time of video recording (ts), and t_(AIS+1) is the time of reception of the AIS data received immediately after the time of video recording (ts).

[0078] According to the above mathematical formula 1, AIS data (especially dynamic information) at the time of video recording (ts) can be calculated using AIS data immediately before and after the time of video recording (ts).

[0079] The above interpolation considering speed and direction of movement is a method that considers the ship's speed and direction of movement based on AIS data at time t-1, and may be desirable for users who require more precise analysis and matching.

[0080] The above processing unit (20) can calculate AIS data at the time of image capture (ts) by Velocity-Based Interpolation considering speed and direction of movement using the following mathematical formula 2 in order to precisely track or analyze a target (ship) in the user's area of ​​interest (ROI).

[0081]

[0082] Here, t_s is the time of video recording, X_s and Y_s are the positions (X,Y) at the time of video recording (ts), X_(AIS-1) and Y_(AIS-1) are the (X,Y) positions of the vessel in the AIS data immediately before the time of video recording (ts), V_(AIS-1) is the vessel speed (m / s) in the AIS data immediately before the time of video recording (ts), and θ_(AIS-1) is the direction of movement (azimuth, degrees) in the AIS data immediately before the time of video recording (ts).

[0083] The above mathematical formula 2 can be corrected to the position of the satellite image point (ts) considering the direction of movement of the target (ship) by calculating the variables cos(θ) and sin(θ).

[0084] Accordingly, according to the above mathematical formula 2, the AIS data (in particular, dynamic information) at the time of video capture (ts) can be calculated using the AIS data immediately prior to the time of video capture (ts).

[0085] The image processing unit (24) can perform geometric correction by rotating and scaling the image based on the AIS data at the image capture time (ts) calculated (or corrected) by the data processing unit (22).

[0086] In addition, the processing unit (20) can perform geometric correction by matching and aligning the AIS data (hereinafter referred to as “corrected AIS data”) at the time of video capture (ts) with the target (ship) in the video, thereby using the target (ship) as a ‘sea reference point’.

[0087] In detail, the actual latitude, longitude, and altitude coordinates of the target (vessel) to which the above-mentioned corrected AIS data is matched can be known. Accordingly, the processing unit (20) can define multiple targets (vessels) as 'sea reference points' in the remote sensing image.

[0088] The above processing unit (20) can define a transformation model for geometric correction by comparing the pixel coordinates (x,y) of an image where a plurality of sea reference points (ships) are located with the actual ship coordinates (X,Y).

[0089] In detail, the processing unit (20) can convert pixel coordinates (x,y) and actual coordinates (X,Y) by setting a polynomial based on the remote sensing image and the AIS data of the sea reference point. For example, the processing unit (20) can perform a first-order conversion (linear conversion) and a second-order conversion (non-linear conversion).

[0090] The processing unit (20) can perform movement, rotation, and scale adjustment in the first transformation to align the image coordinates of the target with the coordinates (actual coordinates) of the AIS data aligned with the target.

[0091] Therefore, the accuracy and precision of geometric correction can be improved even in marine, particularly oceanic, environments where it is difficult to find objects satisfying the traditional definition of a 'ground reference point'.

[0092] Additionally, when multiple vessels are densely searched within the user's area of ​​interest (ROI), the processing unit (20) can determine whether the vessel in the image matches the vessel based on the unique vessel number, etc., among the AIS data.

[0093] Additionally, the processing unit (20) can identify cases where there is a difference between the position of the target in the image and the position of the corrected AIS data. The processing unit (20) can define only targets with more reliable AIS data as maritime reference points by extracting the error (d) between the two positions through Euclidean distance calculation and performing a reliability evaluation process.

[0094] The geometric correction system (1) of the above remote sensing image may further include an output unit (30) and a storage unit (40).

[0095] The output unit (30) can output the result processed by the processing unit (20) as an image. For example, the output unit (30) can output an image in which corrected AIS data is aligned for each target (ship) at the time of image capture (ts).

[0096] The storage unit (40) can store the results of each process performed by the processing unit (20). Additionally, information stored in the storage unit (40) can be loaded at the request of the processing unit (20).

[0097] FIG. 2 is a flowchart exemplarily showing a geometric correction method for remote sensing images according to an embodiment of the present invention.

[0098] Referring to FIG. 2, a geometric correction method for remote sensing images according to an embodiment of the present invention may include the step (S10) of receiving satellite or aerial images.

[0099] The above satellite or aerial image can be received by the receiver (10) along with the time of image capture (ts). The receiver (10) can also receive an image of a region of interest to the user, i.e., a user's area of ​​interest (ROI).

[0100] For example, the image receiving unit (10) can receive images such as an optical satellite or synthetic aperture radar (SAR) for a user's region of interest (ROI) and receive the time of capture (ts) of the image.

[0101] In addition, a step (S20) of receiving and preprocessing AIS data may be performed simultaneously with or after securing an image of the user's region of interest (ROI).

[0102] The above AIS data can be received by the receiving unit (10) for all vessels existing in the user's area of ​​interest (ROI).

[0103] In addition, a preprocessing step can be performed to remove noise or anomalies from AIS data at points in time before or after the video capture time (ts). For example, position errors caused by signal interference in the received AIS data can also be filtered out during the preprocessing step.

[0104] In addition, a preprocessing process for the image may also be performed in step S20 as needed, such as for noise. As described above, the preprocessing process can be performed in the processing unit (20).

[0105] Meanwhile, among the vessels in the image within the user's region of interest (ROI), vessels for which AIS data is not received can be detected in the image. In this case, the vessel can be processed to be excluded from the target in the preprocessing step (S20).

[0106] After the above AIS data is received and / or preprocessed, a step (S30) of correcting the AIS data can be performed.

[0107] The above AIS data correction step (S30) can be understood as a step of correcting the time difference between the image capture time (ts) and the time difference caused by the AIS data transmission and reception interval for each vessel, as described above.

[0108] For example, step S30 can calculate AIS data (especially dynamic information) at the image capture time (ts) by using a linear interpolation or velocity-based interpolation method based on AIS data received before or after the image capture time (ts).

[0109] In step S30, it can be determined whether a vessel within a window designated as a user region of interest (ROI) can be evaluated as moving in a straight line at constant velocity.

[0110] For example, depending on the area covered by the video of the user's region of interest (ROI) (e.g., window size) or if the time interval for tracking the target (ship) is larger than a preset standard, the ship in the video can be approximated as moving in a straight line at a constant speed and calculated as AIS data at the time of video capture (ts), thereby improving the processing speed of large volumes.

[0111] Therefore, when the above vessel is evaluated as moving in a constant linear motion, the AIS data at the time of image capture (ts) can be calculated using Equation 1, which is the linear interpolation method described above.

[0112] Meanwhile, if the above linear interpolation method cannot be used in step S30, or if one wishes to calculate AIS data at the time of image capture (ts) with relative precision by considering the ship's speed and direction of movement, the AIS data at the time of image capture (ts) can be calculated using Equation 2, which is a velocity-based interpolation method that considers the speed (including the direction of movement) described above.

[0113] In this case, based on the AIS data at the time point (t-1) immediately preceding the image capture time point (ts), the AIS data at the image capture time point (ts) can be calculated considering the ship's speed and direction of movement.

[0114] AIS data calculated using the above method can be named “corrected AIS data.”

[0115] After the above AIS data correction step (S30) is completed, a step (S40) of searching for and setting a target (vessel) within the image of the user's region of interest (ROI) can be performed. That is, in step S40, a vessel existing within the user's region of interest (ROI) is searched for and extracted, and a vessel that the user can define as a maritime reference point can be targeted.

[0116] For example, the image processing unit (24) can search for and extract a ship within the image of the user's region of interest (ROI). Here, the search and extraction of the ship image within the image can be performed using a known deep learning (machine learning) method.

[0117] Additionally, in step S40, a target (vessel) for matching the corrected AIS data can be set. For example, the image processing unit (24) can set a target for matching the corrected AIS data among the vessels searched within the image of the user's region of interest (ROI).

[0118] When the above target is set, a step (S50) of matching and aligning the corrected AIS data with the target can be performed.

[0119] According to the above-described step S30, AIS information (i.e., corrected AIS data) for each vessel at the time of video capture (ts) can be obtained, and accordingly, the position (coordinates) of each vessel based on the corrected AIS data at the time of video capture (ts) can be obtained. Therefore, the target vessel searched and set in the image can be matched with the AIS data calculated as the position (coordinates) of the same area.

[0120] For example, the processing unit (20) can find a vessel in an image of a user's region of interest (ROI) corresponding to a location (coordinates) based on the corrected AIS data. The processing unit (20) can also match the corrected AIS data to the vessel.

[0121] And the corrected AIS data can be matched with the information of the target (vessel) in the matched image.

[0122] A target (vessel) having the above-mentioned matched AIS data can be defined as a maritime reference point. The target (vessel) can be defined as multiple, and accordingly, multiple maritime reference points can also be established.

[0123] That is, geometric correction can be performed based on the AIS data of the above-mentioned maritime reference point (target). (S60)

[0124] The above target (vessel) can output not only static information but also dynamic information by aligning AIS data.

[0125] In addition, accurate location information of the vessel in the remote sensing image can be obtained by matching the time difference between the reception of AIS information and the time of capture of the satellite image.

[0126] Meanwhile, based on the above-mentioned video capture time (ts), the position (coordinates) of the vessel in the corrected AIS data may not exactly match the coordinates of the vessel in the video due to errors that may occur in the video and / or errors that may occur in the AIS data.

[0127] Therefore, in the process of aligning corrected AIS data with the target (vessel), it is necessary to determine whether the AIS data can be trusted and used as a maritime reference point by considering the aforementioned error influence.

[0128] Specifically, if the position coordinates of a target (vessel) in a remote sensing image differ from the position coordinates in the corrected AIS data, the processing unit (20) can extract and store the difference between the two positions (hereinafter referred to as “error (d)”) through the calculation of Euclidean distance. The processing unit (20) can then compare the error (d) with a preset threshold value (a).

[0129] Specifically, the above error (d) can be calculated as the Euclidean distance between the corrected AIS data and the target (ship) position in the image, as shown in Equation 3 below.

[0130]

[0131] Here, X_im and Y_im represent position coordinates per plane in the image, and X_(AIS Data) and Y_(AIS Data) represent the XY plane position coordinates of the AIS data corrected for the image capture time (ts).

[0132] Meanwhile, after calculating the error (d), the processing unit (20) can determine whether the error (d) exceeds a threshold value (a).

[0133] The above threshold value (a) can be defined as Equation 6 by the reliability of the AIS data (C_AIS, Equation 4) and the reliability of the image (C_VID, Equation 5).

[0134]

[0135]

[0136]

[0137] Here, C_AIS is a result value representing the reliability of AIS data in the range of 0 to 1, Xs and Ys are the AIS data coordinates after interpolation, Xv and Yv are the coordinates of the vessel detected in the image, and σ_GPS is the GPS error range value (e.g., standard GPS receiver = 3m, RTK GPS using high-precision positioning = 1m). Also, C_VID is a result value representing the reliability of the image in the range of 0 to 1, Xs and Ys are the AIS data coordinates after interpolation, Xv and Yv are the coordinates of the vessel detected in the image, and σ_VID is the value for the total error of the image reliability.

[0138] Meanwhile, the value for the total image reliability error (σ_VID) is defined as the sum of geometric error values ​​due to satellite shooting angle or Earth curvature, hardware error values ​​due to sensor noise or lens distortion, error values ​​due to ship speed, and error values ​​due to shooting environment.

[0139] Therefore, the above threshold value (a) can be understood as a value that takes into account AIS data reliability and image reliability.

[0140] If the above error (d) exceeds the above threshold value (a), the corrected AIS data may be determined to be more reliable (or have a lower error or probability of error) information than the ship information derived from the image.

[0141] In this case, even if there is a difference (i.e., an error (d)) between the coordinates of the vessel in the corrected AIS data based on the time of image capture (ts) and the coordinates of the vessel in the image, it can be determined that this is a distortion phenomenon caused by various factors other than geometric phenomena, and the information for performing geometric correction can be determined to be clear and clean. Therefore, the position of the vessel (target) in the above remote sensing image can be aligned with the coordinates based on the corrected AIS data, and geometric correction can be performed based on this.

[0142] Meanwhile, if the above error (d) is smaller than the above threshold value (a), it can be understood that the risk due to the error in the corrected AIS data is higher than the risk due to the error in the image. In other words, the probability of error in the AIS data may be high.

[0143] Therefore, if the above error (d) is smaller than the threshold value (a), performing geometric correction based on AIS data may result in a problem where the output image becomes more distorted.

[0144] Therefore, if the position (coordinates) of the vessel in the corrected AIS data differs from the coordinates of the vessel in the image based on the above image capture time (ts) (i.e., has an error (d)), and the error (d) is smaller than the threshold value (a), it can be excluded from the target for setting the maritime reference point.

[0145] That is, the processing unit (20) can perform matching only for targets (ships) for which the corrected AIS data can be trusted.

[0146] Ultimately, once the aforementioned marine reference point is established, geometric correction of remote sensing images can be performed effectively and precisely even in a marine environment.

[0147] And the geometrically corrected image can be output (S70).

[0148] FIG. 3 is a diagram showing a screen for searching for and setting a target (ship) in an area of ​​interest (ROI) in a remote sensing image according to an embodiment of the present invention, and FIG. 4 is a diagram showing a screen in which a target (ship) in a remote sensing image and AIS data matched to the target (ship) are fused (or aligned) according to an embodiment of the present invention.

[0149] Figure 4 is an image of a user's region of interest (ROI) for a specific port, captured by a synthetic aperture radar.

[0150] Referring to Fig. 4, the ships discovered in the image (IP, defined by blue squares) are defined, and a point (orange) that can activate information about each ship is defined.

[0151] One of the vessels shown in Fig. 4 can be set as a target.

[0152] In addition, if the user selects the above point, an information screen of the target (vessel) with the corrected AIS data matched can be displayed.

[0153] In this regard, referring to FIG. 5, when one of the vessels searched in FIG. 4 is set as a target, a screen (W1) in which corrected AIS data for the set target (T1) is matched can be seen. The information for the target (T1) includes time (time point), IMO number, vessel number, location (latitude, longitude, altitude), etc.

[0154] Therefore, when exploring a sea area that cannot satisfy traditional ground reference point requirements in remote sensing imagery, the user can set a vessel with aligned AIS data as described above as a sea reference point (target) and perform geometric correction through image translation, rotation, scale adjustment, and coordinate transformation based on relational equations.

[0155] FIG. 5 is a drawing showing a remote sensing image before geometric correction according to an embodiment of the present invention, and FIG. 6 is a drawing showing a remote sensing image output after geometric correction according to an embodiment of the present invention.

[0156] Referring to FIG. 5, a distorted remote sensing image can be seen before geometric correction is performed using a target (ship) according to the embodiment of the present invention described above.

[0157] In detail, when looking at the breakwater lines (b1, b2) forming the harbor in the user interest area (R), it can be seen that due to the distortion phenomenon described above, multiple shapes appear to be superimposed differently from reality.

[0158] Referring to FIG. 6, when looking at the breakwater lines (a1, a2) where geometric correction is performed by defining the target (ship) according to the embodiment of the present invention as a sea reference point in the same user interest area (R), it can be seen that the distorted shape is removed to be identical to the actual shape.

[0159] Therefore, the output image of Fig. 6 can be provided as a more precise and accurate remote sensing image. Explanation of the symbols

[0160] 10: Receiver 20: Processing unit 30: Output section 40: Storage section

Claims

Claim 1 A geometric correction system for remote sensing images, comprising: a receiving unit for receiving remote sensing images and AIS (Automatic Identification System) data; a processing unit for correcting the received AIS data to the AIS data at the time of capture (ts) of the remote sensing images, and matching the ship in the remote sensing images based on the position coordinates of the corrected AIS data to match the information of the matched ship; and an output unit for outputting an image in which geometric correction is performed by the processing unit, wherein the processing unit defines the ship in which the corrected AIS data is matched as a maritime reference point, and performs geometric correction by converting the coordinates of the image in which the maritime reference point is located into the actual coordinates of the corrected AIS data. Claim 2 A geometric correction system for remote sensing images according to claim 1, wherein the geometric correction is characterized by converting the remote sensing image by moving, rotating, and scaling it based on the coordinates of the corrected AIS data of the vessel, which is the maritime reference point. Claim 3 In claim 2, the processing unit defines only the vessels among the vessels detected in the remote sensing image that match the corrected AIS data as maritime reference points, a geometric correction system for remote sensing images. Claim 4 A geometric correction system for remote sensing images according to claim 2, wherein the above-mentioned maritime reference points are set as a plurality and perform geometric correction of the image based on AIS data from the plurality of maritime reference points. Claim 5 A geometric correction system for remote sensing images according to claim 4, wherein the processing unit extracts an error (d) by calculating the Euclidean distance between two locations when there is a difference between the coordinates of a vessel in the remote sensing image and the coordinates of the corrected AIS data, and when the error (d) is greater than a preset threshold value (a), aligns the coordinates of the vessel in the image with the coordinates of the corrected AIS data to set them as a sea reference point. Claim 6 In claim 5, the threshold value (a) is defined by the reliability of the AIS data and the reliability of the remote sensing imagery, and the reliability of the AIS data is, A geometric correction system for remote sensing images defined by (C_AIS: result value representing the reliability of AIS data in the range of 0 to 1, Xs and Ys: X,Y coordinates of AIS data after interpolation, Xv and Yv: X,Y coordinates of vessels detected in the image, σ_GPS is the GPS error range value). Claim 7 In claim 6, the reliability of the remote sensing image is, A geometric correction system for remote sensing images defined by (C_VID: result value representing image reliability in the range of 0 to 1, Xs and Ys are AIS data X,Y coordinates on which interpolation is performed, Xv and Yv: X,Y coordinates of vessels detected in the image, σ_VID is a value for the total error of image reliability). Claim 8 In claim 7, the threshold value (a) is, A geometric correction system for remote sensing images defined as. Claim 9 A method for geometric correction of remote sensing images, comprising: receiving remote sensing images and AIS data; correcting the received AIS data to the AIS data at the time of image capture (ts); searching for a vessel in the image and matching and aligning the corrected AIS data to the searched vessel; determining whether the searched vessel can be set as a maritime reference point; and performing geometric correction of the remote sensing images based on the corrected AIS data of the vessel set as the maritime reference point. Claim 10 A geometric correction method for remote sensing images according to claim 9, wherein the step of determining whether the searched vessel can be set as a sea reference point is characterized by setting it as a sea reference point only when the error (d) between the coordinates of the searched vessel and the coordinates of the corrected AIS data exceeds a preset threshold value (a). Claim 11 In claim 10, the step of correcting with AIS data at the image capture time (ts) comprises an equation considering the speed and direction of movement at the position of the AIS data received immediately before the image capture time (ts-1): A geometric correction method for remote sensing images that corrects along (t_s: time of image capture, X_s: X-plane position at the time of image capture, Y_s: Y-plane position at the time of image capture, X_(AIS-1): X-plane position on AIS data immediately before the time of image capture, Y_(AIS-1): Y-plane position on AIS data immediately before the time of image capture, t_(AIS-1): time of reception of AIS data received immediately before the time of image capture). Claim 12 A geometric correction method for a remote sensing image according to claim 10, wherein the error (d) is extracted by calculating the Euclidean distance between two locations, and the preset threshold value (a) is defined as a value based on the reliability of the AIS data and the reliability of the remote sensing image. Claim 13 A method for geometric correction of remote sensing images according to claim 12, characterized in that the reliability of the remote sensing image is calculated by the sum of a geometric error value for the satellite shooting angle or Earth curvature, a hardware error value for sensor noise or lens distortion, an error value due to ship speed, and an error value due to the shooting environment.

Images