A fast ship target detection method for GEO remote sensing satellites

Abstract

The present application provides a ship target detection and tracking method for GEO remote sensing satellites, comprising: a ground station receives imaging data transmitted in the form of a real-time code stream in real time, and the received imaging data is transmitted to a user end in the form of real-time code stream data; in a preprocessing module of the user end, the imaging data is preprocessed to generate a sequence of image products, wherein the preprocessing of the imaging data includes the screening of imaging spectrum bands; the preprocessing module performs geographic location registration on each frame of image product generated, and simultaneously and in parallel performs ship target extraction on the frame of image product, wherein the ship target extraction includes: slicing the frame of image product according to a preset position point to obtain a frame of image slices; detecting and extracting the coordinates of the ship target in the frame of image slices; and correcting the coordinates of the ship target in the frame of image product using the geographic location registration result of the frame of image product.

Description

Technical Field

[0001] The present application relates to the rapid processing, detection and tracking of ship targets using GEO remote sensing satellites, and in particular to the detection and tracking of large ship targets in the mid- and far-seas using medium- and low-resolution sequence optical remote sensing images. Background Art

[0002] Geostationary orbit remote sensing satellites (GEO remote sensing satellites) are located in geosynchronous orbits and can stay in a fixed area for a long time. They have the characteristics of observation width of hundreds of kilometers, continuous monitoring of targets within the observation range, and real-time data transmission to ground stations. Compared with low-orbit observation systems in terms of time resolution and observation range, they have the ability to respond to tasks quickly and continuously monitor multiple targets over a large area. At present, their applications are mainly to ensure meteorology, environment, and disaster reduction, and they have broad application prospects in ship target detection and tracking.

[0003] Task response speed, data processing timeliness, target detection accuracy, and target association correctness are important technical indicators that affect ship target detection and tracking. Research on ship target detection and tracking is mostly focused on the research of detection and tracking algorithms. Due to the small number of GEO remote sensing satellites in orbit and the fact that my country only has one GF-4 satellite for civilian use, the conventional mission process is not designed for ship target detection and tracking needs. In actual applications, the timeliness is greatly reduced due to the long task response, data transmission, processing, and application links. The time from obtaining imaging data to generating image products is extended to several hours. Due to the lack of accurate and efficient processing application systems, the accuracy of target detection and tracking is also difficult to guarantee. This seriously restricts the actual application effect of ship target detection and tracking.

[0004] Figure 1 The conventional mission process is shown. After the full-spectrum satellite data (real-time code stream data) of a single mission is landed, the satellite control department will centrally perform batch preprocessing to generate standard Level 2 image products (Level 2 full-spectrum sequence images). The generated image products are packaged and delivered to the user. On the user side, the entire batch of full-spectrum sequence images are used to perform ship target detection and target tracking for panoramic images of multiple spectral bands to obtain target tracking information.

[0005] The following problems exist in the target's regular task flow:

[0006] 1. The code stream data processing process is independent of the target detection and tracking process. The serialization of these two processes increases the delay. After the satellite control department completes the production of the image products of a single mission, it will be packaged and transmitted to the user. After receiving all the image products of a single mission, the user will detect and track the target, which will cause a large delay in the task process.

[0007] 2. Since the satellite control department needs to batch process the single-mission imaging data set to generate standard Level 2 image products before distributing them to users, there is a queue for processing tasks. The current processing delay is generally in the order of hours.

[0008] 3. Since the second-level image products are generated using the full-spectrum code stream data, the geometric precision correction process in the preprocessing program is time-consuming.

[0009] 4. It takes a long time to use multi-spectral data for panoramic image detection.

[0010] Therefore, there is a problem of poor timeliness of task flow in the target's regular task flow. Summary of the invention

[0011] At least one object of the present application is to provide a ship target detection and tracking method, comprising:

[0012] The satellite control department will upload the imaging mission instructions to the geostationary orbit remote sensing satellite;

[0013] The imaging data transmitted in the form of real-time code stream is received by the ground station in real time, and the received imaging data is transmitted to the user end in the form of real-time code stream data;

[0014] In a preprocessing module at the user end, the imaging data is preprocessed to generate a sequence image product, wherein the sequence image product includes a multi-frame image product, wherein the preprocessing of the imaging data includes screening of an imaging spectrum segment, and the imaging spectrum segment is selected from at least one of a visible spectrum segment and a near infrared spectrum segment;

[0015] Each time the preprocessing module generates a frame of image product, the geographical location of the frame of image product is registered, and the ship target is extracted from the frame of image product in parallel, wherein the ship target extraction includes: slicing the frame of image product according to a preset position point to obtain a frame of image slices; detecting the frame of image slices and extracting the coordinates of the ship target in the frame of image product;

[0016] Using the geographical location registration result of the frame image product, the coordinates of the ship target in the frame image product are corrected;

[0017] Using the target tracking module, starting from the second frame image slice, the time and space information of the target are associated. After the association is completed for three consecutive frames of image slices, the target tracking is confirmed to be successful;

[0018] Update the output target tracking information to achieve scrolling output of target tracking information.

[0019] According to an embodiment of the present invention, each time a new image product is generated by the preprocessing module, the target detection module detects the image product.

[0020] The detection of the frame image product includes: performing geographic location registration on the frame image product and extracting a ship target from the frame image product in parallel; and using the geographic location registration result of the frame image product to correct the coordinates of the ship target in the frame image slice of the frame image product.

[0021] According to an embodiment of the present invention, control points are selected by using a loaded control point library, thereby performing geographic location registration using a standard map.

[0022] According to an embodiment of the present invention, the sequence image product is a level 1A sequence image product.

[0023] According to an embodiment of the present invention, the process of obtaining a frame of image slices includes:

[0024] The coordinates of the ship target at the starting time are determined through prior information, and the radius of the active area at the current time is extrapolated according to the ship's movement speed. The image slice size is the circumscribed rectangle of a circle with the coordinates of the starting time as the center and the radius of the active area as the radius.

[0025] According to an embodiment of the present invention, the radius of the activity area is 20km to 50km.

[0026] According to one embodiment of the present invention, the preprocessing also includes data formatting, attitude and orbit determination, radiation correction, and RPC orthorectification calculation.

[0027] According to an embodiment of the present invention, the target tracking information includes target point position, target heading, and target speed.

[0028] The present invention also provides a computer-readable storage medium having software instructions stored thereon, wherein the software instructions implement the above method when executed.

[0029] The present invention also provides a ship target detection and tracking system for executing the above method, wherein the system comprises a user terminal, and the user terminal comprises:

[0030] Preprocessing module;

[0031] Object detection module;

[0032] Target tracking module.

[0033] The method provided by the present application can greatly improve the processing timeliness, shorten the processing delay, and realize the real-time detection and tracking of moving targets. In addition, it can also perform rapid detection and identification, reduce the false alarm rate, and improve the accuracy of target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The following will further explain the above characteristics, technical features, advantages and implementation methods of the present application in a clear and understandable manner through the description of the preferred embodiments and in combination with the accompanying drawings. The following drawings are only intended to illustrate and explain the present application, and do not limit the scope of the present application. Among them:

[0035] Figure 1 The general task flow of ship target detection and tracking is shown.

[0036] Figure 2 The flowchart of the ship target detection and tracking method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0037] In order to have a clearer understanding of the technical features, purposes and effects of the present application, the specific implementation methods of the present application are now described with reference to the accompanying drawings.

[0038] The present application provides a method for rapid processing, detection and tracking of ship targets based on GEO remote sensing satellites, which can improve data processing speed, reduce processing delay, have strong timeliness, and give full play to the advantages of GEO remote sensing satellites in rapid response and continuous tracking.

[0039] The process of the ship target detection and tracking method provided in this application is as follows Figure 2 The ship target detection and tracking method provided by the present invention includes data preprocessing, target detection and target tracking.

[0040] Among them, data preprocessing and target detection adopt multi-threaded parallel processing, and the data preprocessing and target detection threads are started by receiving data as a signal.

[0041] The target tracking operation needs to be started after the target detection operation of at least three frames of image data is completed. After the target tracking operation is started, the target motion information is updated accordingly for each new frame of target detection result received, thereby achieving continuous scrolling output of the target position and speed and heading motion information.

[0042] like Figure 2 As shown, the ship target detection and tracking method provided by this application includes:

[0043] S1: Ship target detection requirements are submitted by users: The user determines that a suspicious ship target needs to be continuously tracked, and after knowing the position of the suspicious ship target at a certain time through prior information (such as information obtained by shore-based radar, AIS and other detection methods), the user generates task requirements (including target discovery time, target point location, tracking start time, and continuous tracking duration) and directly submits them to the satellite control department.

[0044] S2: The satellite control department conducts mission planning: After the satellite control department conducts mission planning, it generates an imaging mission plan. The ground station generates imaging mission instructions and uploads them to the geostationary orbit remote sensing satellite.

[0045] S3: Imaging and real-time code stream data transmission by geostationary orbit remote sensing satellite: The satellite is turned on at the specified time according to the imaging task instruction to shoot the area where the target point is located. The satellite transmits the imaging data in the form of real-time code stream while shooting.

[0046] S4: The ground station receives the imaging data in real time and transmits it to the user: The ground station receives the original real-time code stream data transmitted by the satellite in real time, and transmits it to the satellite control department synchronously, and transmits the imaging data in the form of original real-time code stream data transmitted by the satellite to the user end through the ground dedicated line.

[0047] S5: In the preprocessing module of the user end, the imaging data is preprocessed to generate a sequence image product: the received imaging data is preprocessed using the preprocessing module. The preprocessing includes: screening of imaging spectrum bands; generation of 1A-level image products. Among them, in the screening of imaging spectrum bands, two imaging spectrum bands are optional (one or both of the B1 visible spectrum band and the B5 near-infrared spectrum band). The generation steps of the 1A-level image product include data formatting, attitude and orbit determination, radiation correction, and RPC orthorectification calculation, and finally generate a 1A-level sequence image product (a standard product with an RPC file).

[0048] S6: Detecting ship targets: When the preprocessing module generates each frame of 1A-level image products (the frame of 1A-level image products is a panoramic image), the frame of 1A-level image products is geographically aligned, and the frame of 1A-level image products is simultaneously extracted from the ship targets. The geographical alignment and ship target extraction steps are performed simultaneously and in parallel.

[0049] In the geographic location registration step, control points are selected through the loaded control point library, so as to perform geographic location registration on the frame of 1A level image product using a standard map.

[0050] Among them, in the ship target extraction step, it includes: slicing the frame 1A level image product according to the preset position point to obtain an image slice; detecting the image slice and extracting the coordinates of the ship target in the image product. Specifically, in the slicing process, the possible distribution area of ​​the ship target is determined by prior information, and the frame 1A level image product (panoramic image) is sliced ​​to reduce the detection range. Through the prior information, the longitude and latitude coordinates of the ship target at the starting time T0 can be obtained (the coordinates can be the preset position point), and the radius of the activity area at the current time T1 can be extrapolated according to the ship movement speed V (R=V×(T1-T0)). The slice size is a circumscribed rectangle of a circle with the coordinates at the starting time T0 as the center and R as the radius. The image slice radius R is adjustable between 20-50km (for example, 30km), and is determined according to the continuous tracking time and the ship speed.

[0051] S7: Accurately locate the ship target: Use the geographic location registration result of a frame of 1A-level image product to correct the coordinates of the ship target in the image slice of the frame of 1A-level image product, so as to improve the positioning accuracy. For example, according to the geographic location registration result, the frame of 1A-level image product has an offset (Δx, Δy) relative to the standard map, then the offset (Δx, Δy) is applied to the coordinates of the ship target in the frame of 1A-level image product to correct the coordinates of the ship target in the frame of 1A-level image product relative to the standard map.

[0052] In the prior art, the two steps of geographic location registration and ship target extraction are performed in series. That is, firstly, a certain frame of 1A-level image product is geographically registered using a standard map, and then the ship target is extracted from the registered frame of 1A-level image product. Since these two steps are time-consuming and serial, the task process is delayed greatly.

[0053] In the present application, since the two steps of geographic location registration and ship target extraction are performed in parallel, the time consumed can be greatly reduced.

[0054] S8: Target association tracking, calculation of heading and speed: Associating the time and space information of the target, so as to track the target, starting from the second frame image slice, after completing the detection and association of three consecutive frames of image slices, confirming that the target tracking is successful. Calculate the mean and variance of heading and speed, and the default minimum variance is the final track.

[0055] S9: Target continuous tracking: Target continuous tracking: After each new image product is generated in the preprocessing stage, the image slice of the image product is detected and associated with the detected target. If the target association is successful, the target tracking information is updated and output to achieve rolling output of the target tracking information. The target tracking information includes the target point position, heading, and speed. If the target association fails, it means that the target is lost.

[0056] The ship target detection and tracking method provided in this application has the following technical features:

[0057] (1) Integrate data preprocessing functions into the user end, optimize the process according to task requirements, and realize real-time and fast data processing on the user end. Select B1 or B5 single-spectrum imaging data processing to generate 1A-level image products.

[0058] (2) The target detection process improves timeliness while also improving positioning accuracy.

[0059] Taking advantage of the large single-shot imaging width of geostationary remote sensing satellite array imaging, when the captured image contains land, the control points are selected for geographic location registration and correction to improve positioning accuracy. The control point registration (geolocation registration) process is carried out in parallel with ship target extraction, and only the position of the detected target is corrected to improve target positioning accuracy and detection timeliness.

[0060] (3) In the existing target detection algorithms, it is generally necessary to first use the sea-land segmentation algorithm to identify and distinguish sea and land features. In this application, an image slicing solution is adopted. In the detection of ship targets in sparse waters in the middle and far seas, there is no land area in the image slice, so there is no need to consider the sea-land segmentation problem, and only the visible light single spectrum image characteristics can be used for target detection.

[0061] (4) Data preprocessing is performed in parallel with target detection and tracking. The 1A-level image products generated by preprocessing are transmitted to the target detection module and the target tracking module in real time. The target detection module starts target detection. After receiving three consecutive frames of image products, the target tracking module associates the three frames and generates continuous motion information of the target, including position, heading, speed, track, etc. After that, each frame of image product is detected and the current information is updated to achieve real-time rapid detection and tracking.

[0062] The method provided in this application has the following beneficial effects:

[0063] (1) Only use B1 or B5 single-spectrum sequence image products for target detection and tracking to improve processing efficiency.

[0064] (2) Based on prior information, target slicing and windowing detection is performed on the image, which can quickly detect and identify the target, reduce the false alarm rate, and improve the accuracy of target tracking.

[0065] (3) By integrating preprocessing, target detection, and target tracking, the processing flow is optimized, and each module runs in parallel according to the set business logic, greatly improving the processing application efficiency. The conventional several hours are shortened to less than 10 minutes, achieving real-time detection and tracking of moving targets.

[0066] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0067] The above description is only an illustrative embodiment of the present application and is not intended to limit the scope of the present application. Any equivalent changes, modifications and combinations made by any technician in the field without departing from the concept and principle of the present application shall fall within the scope of protection of the present application.

Claims

1. A ship target detection and tracking method for GEO remote sensing satellites, comprising: The satellite control department will upload the imaging mission instructions to the geostationary orbit remote sensing satellite; The imaging data transmitted in the form of real-time code stream is received by the ground station in real time, and the received imaging data is transmitted to the user end in the form of real-time code stream data; In a preprocessing module at the user end, the imaging data is preprocessed to generate a sequence image product, wherein the sequence image product includes a multi-frame image product, wherein the preprocessing of the imaging data includes screening of an imaging spectrum segment, and the imaging spectrum segment is selected from at least one of a visible spectrum segment and a near infrared spectrum segment; Each time the preprocessing module generates a frame of image product, the geographical location of the frame of image product is registered, and the ship target is extracted from the frame of image product in parallel, wherein the ship target extraction includes: slicing the frame of image product according to a preset position point to obtain a frame of image slices; detecting the frame of image slices and extracting the coordinates of the ship target in the frame of image product; Using the geographical location registration result of the frame image product, the coordinates of the ship target in the frame image product are corrected; Using the target tracking module, starting from the second frame image slice, the time and space information of the target are associated. After the association is completed for three consecutive frames of image slices, the target tracking is confirmed to be successful; Update the output target tracking information to achieve rolling output of target tracking information, Each time the preprocessing module generates a new image product, the target detection module detects the image product. The detection of the frame image product includes: performing geographic location registration on the frame image product and extracting a ship target from the frame image product in parallel; using the geographic location registration result of the frame image product, correcting the coordinates of the ship target in the frame image slice of the frame image product, The process of obtaining a frame of image slice includes: The coordinates of the ship target at the starting time are determined by prior information, and the radius of the active area at the current time is extrapolated according to the ship's movement speed. The image slice size is the circumscribed rectangle of a circle with the coordinates at the starting time as the center and the radius of the active area as the radius. The radius of the activity area is 20km to 50km.

2. The method according to claim 1, wherein: Geo-reference using standard maps by selecting control points using a loaded control point library.

3. The method according to claim 1, wherein: The sequence image product is a 1A level sequence image product.

4. The method according to claim 1, wherein: The preprocessing also includes data formatting, attitude and orbit determination, radiation correction, and RPC orthorectification calculation.

5. The method according to claim 1, wherein: The target tracking information includes the target point position, target heading, and target speed.

6. A computer-readable storage medium having stored thereon software instructions which, when executed, implement the method according to any one of claims 1-5.

7. A ship target detection and tracking system for GEO remote sensing satellites, used to execute the method according to any one of claims 1 to 5, the system comprising a user end, the user end comprising: Preprocessing module; Object detection module; Target tracking module.

Images