Method for calculating sea target speed based on virtual control points of high-orbit satellite images

Abstract

The present invention discloses a method for calculating the speed of maritime targets based on virtual control points of high-orbit satellite images, which includes: acquiring high-orbit satellite images at different imaging times, extracting maritime targets from the images to obtain maritime target information; at each imaging time, according to the maritime target information, calculating the trajectory intersection points of any two of the pre-selected maritime targets, and selecting the trajectory intersection points as virtual control points; using the virtual control points at different imaging times for point set matching, and calculating the relative transformation relationship between the images at different imaging times according to the matching results; correcting the positions of the maritime targets at the specified imaging time according to the relative transformation relationship between the images, and calculating the speed and direction of the maritime targets based on the corrected positions of the maritime targets. The method of the present invention can eliminate the relative position error between high-orbit satellite images in the case of no available ground control points, and improve the calculation accuracy of the speed of maritime targets in the case of no available ground control points.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to a method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images. Background Art

[0002] High-orbit optical satellites have the characteristics of wide imaging range and high temporal resolution, and can continuously monitor a large range of sea areas to obtain continuous maritime situations, having broad application prospects in space-based ocean surveillance.

[0003] Currently, when using high-orbit optical satellites for sea area surveillance, when the surveillance sea area of the high-orbit optical satellite includes geographical conditions such as land and islands, that is, when the satellite images taken by the high-orbit optical satellite include geographical conditions such as land and islands, fixed ground control points on the land or islands are used for image registration to eliminate the relative position deviation between image sequences, and then the motion state of the maritime target is estimated based on the satellite image sequences after eliminating the deviation. When the surveillance sea area of the high-orbit optical satellite does not include geographical conditions such as land and islands, that is, when the satellite images taken by the high-orbit optical satellite do not include geographical conditions such as land and islands, since fixed ground control points cannot be used for image registration, the motion state of the maritime target is directly estimated based on the satellite image sequences at this time.

[0004] However, due to the fact that high-orbit optical satellites are far from the earth, errors such as platform attitude and orbital operation lead to large geometric positioning errors in satellite images, and the error level is generally of the order of kilometers, which in turn leads to large relative position errors between satellite image sequences. If the motion state of the maritime target is directly estimated based on the unregistered satellite image sequences, the estimation accuracy of the motion state of the maritime target is poor, and the error of the calculated speed of the maritime target is large. Summary of the Invention

[0005] To solve some or all of the technical problems existing in the above-mentioned prior art, the present invention provides a method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images.

[0006] The technical solution of the present invention is as follows:

[0007] A method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images is provided, and the method includes:

[0008] Obtain high-orbit satellite images at different imaging times, extract maritime targets from the images, and obtain maritime target information;

[0009] At each imaging time, according to the maritime target information, calculate the trajectory intersection points of any two of the pre-selected maritime targets, and select the trajectory intersection points as virtual control points;

[0010] Perform point set matching using virtual control points at different imaging times, and calculate the relative transformation relationship between images at different imaging times according to the matching results;

[0011] Correct the position of the maritime target at the specified imaging time according to the relative transformation relationship between the images, and calculate the speed and direction of the maritime target based on the corrected position of the maritime target.

[0012] In some possible implementation manners, extract the maritime target from the image to obtain maritime target information, including:

[0013] Use the rotated bounding box object detection algorithm to extract the maritime target from the image, and obtain the pixel position, length, and direction information of the maritime target in the image;

[0014] Determine the actual position, length, and direction information of the maritime target according to the pixel position, length, and direction information of the maritime target, as well as the high-orbit satellite image resolution and projection information.

[0015] In some possible implementation manners, the intersection point of the trajectories of two maritime targets is determined in the following manner:

[0016] Pass through the position point of one maritime target and draw a straight line along the direction of the current maritime target;

[0017] Pass through the position point of the other maritime target and draw a straight line along the direction of the current maritime target;

[0018] Take the intersection point of the two straight lines as the intersection point of the trajectories of the two maritime targets.

[0019] In some possible implementation manners, the intersection point of the trajectories of two maritime targets is calculated using the following formula:

[0020]

[0021] where, (x1, y1) represents the x-axis and y-axis coordinate positions of a maritime target at a certain imaging time in the set position coordinate system, (x2, y2) represents the x-axis and y-axis coordinate positions of another maritime target at the same imaging time in the set position coordinate system, (x 12 , y 12 ) represents the x-axis and y-axis coordinate positions of the intersection point of the trajectories of the two maritime targets at the same imaging time in the set position coordinate system, θ1 represents the angle between a maritime target and the y-axis of the set position coordinate system, and θ2 represents the angle between the other maritime target and the y-axis of the set position coordinate system.

[0022] In some possible implementation manners, select the maritime targets in the image with a length dimension greater than the preset pixel value to calculate the intersection point of the trajectories.

[0023] In some possible implementation manners, the intersection point of the trajectories of two corresponding maritime targets whose direction angle difference is greater than a preset angle threshold and whose distance from each other is less than a preset distance threshold is selected as the virtual control point.

[0024] In some possible implementation manners, point set matching is performed using the virtual control points at different imaging times, including:

[0025] Performing point set matching on the virtual control points at different imaging times using the Iterative Closest Point (ICP) algorithm to determine the corresponding matching point pairs;

[0026] Using the Random Sample Consensus (RANSAC) algorithm to screen and refine the corresponding matching point pairs to obtain the matching point pairs.

[0027] In some possible implementation manners, the position of the maritime target at a specified imaging time is corrected according to the relative transformation relationship between the images, including:

[0028] Taking the position of the maritime target at the previous imaging time of the specified imaging time as a reference, and correcting the position of the maritime target at the specified imaging time according to the relative transformation relationship between the images at the two imaging times.

[0029] In some possible implementation manners, the following method is used to calculate the speed and direction of the maritime target:

[0030] Taking the ratio of the distance between the positions of the maritime target at the two corrected imaging times to the imaging time interval as the speed of the maritime target at the specified imaging time;

[0031] Taking the direction from the position of the maritime target at the corrected specified imaging time to the position of the maritime target at the previous imaging time as the direction of the maritime target at the specified imaging time.

[0032] The main advantages of the technical solution of the present invention are as follows:

[0033] The method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images of the present invention uses the intersection point of the maritime target trajectories as the virtual control point to estimate the relative transformation relationship between image sequences, and performs the estimation of the motion state of the maritime target based on the determined relative transformation relationship, which can eliminate the relative position error between high-orbit satellite images in the case of no available ground control points and improve the calculation accuracy of the speed of the maritime target in the case of no available ground control points. Description of the Drawings

[0034] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, and constitute a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0035] Figure 1 Flowchart of a method for calculating the speed of a maritime target based on virtual control points in high-orbit satellite images according to an embodiment of the present invention;

[0036] Figure 2 Schematic diagram of the calculation principle of the trajectory intersection point of a maritime target according to an embodiment of the present invention;

[0037] Figure 3 Schematic diagram of the relative position relationship of virtual control points at the t imaging moment and the t+Δt imaging moment given by an embodiment of the present invention;

[0038] Figure 4 Given by an embodiment of the present invention corresponding to Figure 3 Schematic diagram of the point set matching result of virtual control points at two imaging moments. Detailed implementation manners

[0039] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0040] The following will detail the technical solutions provided by the embodiments of the present invention in conjunction with the drawings.

[0041] Refer to Figure 1 , an embodiment of the present invention provides a method for calculating the speed of a maritime target based on virtual control points in high-orbit satellite images, and the method includes the following steps S1-S4:

[0042] Step S1, obtain high-orbit satellite images at different imaging moments, extract maritime targets from the images, and obtain maritime target information.

[0043] When using a high-orbit optical satellite for sea area surveillance, the high-orbit optical satellite will take a series of images. When it is necessary to estimate the motion state of a maritime target for maritime target surveillance, first obtain high-orbit satellite images at different imaging moments, and then extract maritime targets from the high-orbit satellite images to obtain maritime target information.

[0044] In an embodiment of the present invention, the maritime target information includes: the position, length, and direction information of the maritime target.

[0045] Further, in an embodiment of the present invention, extracting maritime targets from high-orbit satellite images to obtain maritime target information includes the following steps:

[0046] Step S11: Use the rotated bounding box object detection algorithm to extract marine targets from the image, and obtain the pixel positions, lengths, and direction information of the marine targets in the image.

[0047] Step S12: Determine the actual positions, lengths, and direction information of the marine targets based on the pixel positions, lengths, and direction information of the marine targets, as well as the high-orbit satellite image resolution and projection information.

[0048] In an embodiment of the present invention, the rotated bounding box object detection algorithm can adopt any existing rotated bounding box object detection algorithm, as long as it can achieve object detection and extraction to obtain object information. For example, the R2CNN deep learning object detection algorithm can be adopted.

[0049] Step S2: At each imaging moment, according to the marine target information, calculate the trajectory intersection points of any two marine targets among the pre-selected marine targets, and select the trajectory intersection points as virtual control points.

[0050] In an embodiment of the present invention, the trajectory intersection points of two marine targets are determined in the following manner:

[0051] Draw a straight line passing through the position point of one marine target and along the direction of the current marine target.

[0052] Draw a straight line passing through the position point of the other marine target and along the direction of the current marine target.

[0053] Take the intersection point of the two straight lines as the trajectory intersection point of the two marine targets.

[0054] Refer to Figure 2 Take marine target 1 and marine target 2 in Figure 2 as an example. Assume that the positions of marine target 1 and marine target 2 at the t imaging moment are (x1, y1) and (x2, y2), and the positions at the t+Δt imaging moment are (x1′, y1′) and (x2′, y2′), where the position coordinate system is set according to the actual situation, and the position coordinate systems set at different imaging moments are the same.

[0055] In practical applications, when Δt is small, it can be approximately considered that the marine targets move in a uniform straight line. Therefore, further assume that the direction angles of marine target 1 and marine target 2 from the t imaging moment to the t+Δt imaging moment are θ1 and θ2. Among them, the direction angle of the marine target represents the angle between the direction of the marine target and the y-axis of the set position coordinate system.

[0056] Based on the above assumptions, according to the above determination method of the trajectory intersection points of the two marine targets, the trajectory intersection points of the two marine targets at the t imaging moment and the t+Δt imaging moment are the same point.

[0057] Furthermore, taking the maritime target 1 and the maritime target 2 at the t imaging moment as examples, in an embodiment of the present invention, the trajectory intersection point of the two maritime targets is calculated using the following formula:

[0058]

[0059] where (x 12 , y 12 ) represents the x-axis and y-axis coordinate positions of the trajectory intersection point of the two maritime targets at the corresponding imaging moment in the set position coordinate system.

[0060] In step S1, when using the rotated bounding box object detection algorithm to extract maritime targets from the image, since the angle range of the rotated bounding box is 0 to 180 degrees, there may still be a 180-degree ambiguity in the direction of the extracted maritime targets. In an embodiment of the present invention, the above-defined method is used to determine the calculation of the trajectory intersection point of the maritime target. Even if the direction of the maritime target rotates 180 degrees, the obtained trajectory intersection point will not change, and the 180-degree ambiguity problem existing in the rotated bounding box object detection algorithm can be eliminated.

[0061] When a high-orbit optical satellite images, pose errors, orbit errors, etc. will cause geometric misalignment between image sequences. When the imaging time interval is short, the position of the trajectory intersection point of the maritime target can be considered fixed. For this reason, in an embodiment of the present invention, taking the satellite image at one imaging moment as a reference, the trajectory intersection points of the maritime targets in the satellite images at different imaging moments are used as virtual control points to solve the relative geometric transformation relationship between the image sequences.

[0062] Furthermore, considering that the position error of the virtual control point is affected by factors such as the target direction estimation error and the distance between the target and the virtual control point, in an embodiment of the present invention, in order to improve the solution accuracy of the relative geometric transformation relationship between the satellite image sequences, maritime targets with a length dimension greater than a preset pixel value in the image are selected to calculate the trajectory intersection point.

[0063] Since the direction estimation error is generally related to the target length, the longer the target length, the higher the direction accuracy estimated by the rotated bounding box. By selecting maritime targets with a length dimension greater than a preset pixel value in the image to calculate the trajectory intersection point, it can be ensured that the position of the obtained trajectory intersection point has a high accuracy, thereby reducing the position error of the selected virtual control point.

[0064] Furthermore, in an embodiment of the present invention, the trajectory intersection point corresponding to the two maritime targets with a direction angle difference greater than a preset angle threshold and a distance less than a preset distance threshold is selected as the virtual control point.

[0065] Since the closer the target is to the virtual control point, the smaller the position error of the virtual control point. By selecting the intersection point of the trajectories of two corresponding maritime targets with a direction angle difference greater than a preset angle threshold and a distance less than a preset distance threshold as the virtual control point, the position error of the selected virtual control point can be further reduced, thereby improving the solution accuracy of the relative geometric transformation relationship between satellite image sequences.

[0066] Among them, the preset pixel value, preset angle threshold, and preset distance threshold are specifically set according to actual requirements.

[0067] Step S3: Use the virtual control points at different imaging times for point set matching, and calculate the relative transformation relationship between the images at different imaging times according to the matching results.

[0068] The virtual control points of satellite images at different imaging times form different point sets. In one embodiment of the present invention, a topology-based point set matching algorithm is used for matching.

[0069] Reference Figure 3-4 , considering that the transformation between high-orbit satellite image sequences is mainly translation transformation and rotation transformation (rigid transformation), and basically translation transformation. In one embodiment of the present invention, using the virtual control points at different imaging times for point set matching includes the following steps:

[0070] Use the Iterative Closest Point algorithm (ICP algorithm) to perform point set matching on the virtual control points at different imaging times to determine the corresponding matching point pairs;

[0071] Use the Random Sample Consensus algorithm (RANSAC algorithm) to screen and refine the corresponding matching point pairs to obtain the matching point pairs.

[0072] By performing point set matching in the above manner, the number of incorrect matches and low-precision matching point pairs can be significantly reduced.

[0073] Furthermore, in one embodiment of the present invention, calculating the relative transformation relationship between the images at different imaging times according to the matching results includes the following steps:

[0074] Construct a relationship formula based on the matching point pairs and the relative transformation relationship between the preset satellite images, and use the least squares method to solve the relationship formula to determine the actual relative transformation relationship between the satellite images.

[0075] In theory, the translation relationship can be calculated using a pair of matching virtual control points, the translation relationship and rotation relationship can be calculated using two pairs of matching virtual control points, and more complex transformation relationships can be calculated using more matching point pairs.

[0076] Step S4: Correct the position of the maritime target at the specified imaging time according to the relative transformation relationship between the images, and calculate the speed and direction of the maritime target based on the corrected position of the maritime target.

[0077] In one embodiment of the present invention, correcting the position of the maritime target at the specified imaging time according to the relative transformation relationship between the images includes the following steps:

[0078] Taking the position of the maritime target at the previous imaging time of the specified imaging time as a reference, correct the position of the maritime target at the specified imaging time according to the relative transformation relationship between the images at the two imaging times.

[0079] Specifically, by correcting the target position at one of the imaging times according to the relative transformation relationship between the images at the two imaging times, the relative position deviation between the images at the two imaging times can be eliminated.

[0080] Further, based on the corrected position of the maritime target, in one embodiment of the present invention, the following method is used to calculate the speed and direction of the maritime target:

[0081] Taking the ratio of the distance between the positions of the maritime target at the two corrected imaging times to the imaging time interval as the speed of the maritime target at the specified imaging time;

[0082] Taking the direction from the position of the maritime target at the corrected specified imaging time to the position of the maritime target at the previous imaging time as the direction of the maritime target at the specified imaging time.

[0083] Specifically, taking the specified imaging time as the (t + Δt) moment and the previous imaging time as the t moment as an example, determine the distance Δl between the position of the maritime target at the (t + Δt) moment after correction and the position of the maritime target at the t moment, and calculate the ratio of the position distance Δl to the time interval Δt Taking the ratio as the actual speed of the corresponding maritime target at the (t + Δt) moment, and taking the direction from the position of the maritime target at the (t + Δt) moment after correction to the position of the maritime target at the t moment as the actual direction of the corresponding maritime target at the (t + Δt) moment.

[0084] The method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images provided by one embodiment of the present invention uses the intersection points of the maritime target trajectories as virtual control points to estimate the relative transformation relationship between image sequences, and estimates the motion state of the maritime target based on the determined relative transformation relationship, which can eliminate the relative position error between high-orbit satellite images in the case of no available ground control points and improve the calculation accuracy of the speed of the maritime target in the case of no available ground control points.

[0085] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images, characterized in that, Including: Obtain high-orbit satellite images at different imaging times, extract maritime targets from the images, and obtain maritime target information. At each imaging time, according to the maritime target information, calculate the trajectory intersection points of any two of the pre-selected maritime targets, and select the trajectory intersection points as virtual control points. Use the virtual control points at different imaging times for point set matching, and calculate the relative transformation relationship between the images at different imaging times according to the matching results. Correct the positions of the maritime targets at the specified imaging time according to the relative transformation relationship between the images, and calculate the speed and direction of the maritime targets based on the corrected positions of the maritime targets.

2. The method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images according to claim 1, wherein Extract maritime targets from the images to obtain maritime target information, including: Use the rotated bounding box object detection algorithm to extract maritime targets from the images, and obtain the pixel positions, lengths, and direction information of the maritime targets in the images. Determine the actual positions, lengths, and direction information of the maritime targets according to the pixel positions, lengths, and direction information of the maritime targets, as well as the high-orbit satellite image resolution and projection information.

3. The method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images according to claim 1, wherein The trajectory intersection points of two maritime targets are determined in the following way: Pass through the position point of one maritime target and draw a straight line along the direction of the current maritime target. Pass through the position point of the other maritime target and draw a straight line along the direction of the current maritime target. Take the intersection point of the two straight lines as the trajectory intersection point of the two maritime targets.

4. The method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images according to claim 3, characterized in that, The trajectory intersection points of two maritime targets are calculated using the following formula: Among them, (x1, y1) represents the x-axis and y-axis coordinate positions of a maritime target at an imaging moment in the set position coordinate system, (x2, y2) represents the x-axis and y-axis coordinate positions of another maritime target at the same imaging moment in the set position coordinate system, (x 12 , y 12 ) represents the x-axis and y-axis coordinate positions of the trajectory intersection point of two maritime targets at the same imaging moment in the set position coordinate system, θ1 represents the angle between a maritime target and the y-axis of the set position coordinate system, and θ2 represents the angle between another maritime target and the y-axis of the set position coordinate system.

5. The method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images according to claim 3, wherein, Select the maritime targets in the images with length dimensions greater than the preset pixel value to calculate the trajectory intersection points.

6. The method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images according to claim 5, wherein Select the trajectory intersection points where the difference in the direction angles of the corresponding two maritime targets is greater than the preset angle threshold and the distance between the corresponding two maritime targets is less than the preset distance threshold as virtual control points.

7. The method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images according to any one of claims 1-6, characterized in that Use the virtual control points at different imaging times for point set matching, including: Use the iterative closest point algorithm to perform point set matching on the virtual control points at different imaging times to determine the corresponding matching point pairs. Use the random sample consensus algorithm to screen and refine the corresponding matching point pairs to obtain the matching point pairs.

8. The method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images according to any one of claims 1-6, characterized in that, Correct the positions of the maritime targets at the specified imaging time according to the relative transformation relationship between the images, including: Based on the positions of the maritime targets at the previous imaging time of the specified imaging time, correct the positions of the maritime targets at the specified imaging time according to the relative transformation relationship between the images at the two imaging times.

9. The method for calculating the speed of a maritime target based on virtual control points of high-orbit satellite images according to claim 8, wherein Calculate the speed and direction of the maritime targets in the following way: Take the ratio of the distance between the positions of the maritime targets at the two corrected imaging times to the imaging time interval as the speed of the maritime targets at the specified imaging time. Take the direction from the position of the maritime target at the corrected specified imaging time to the position of the maritime target at the previous imaging time as the direction of the maritime target at the specified imaging time.

Images