A method and system for relay tracking application for large and medium-sized ships in the open sea

Abstract

The present invention discloses a method and system for relay tracking applications for large and medium-sized ships in the open sea. The method includes the following steps: S1. Obtain the current position information of the target ship, and delimit a circular area of interest based on this current position information; S2. Calculate the overhead satellites within half an hour based on the area of interest; for the uplink task of the satellites that will pass over the area of interest in the short term, determine the current position of the target ship; after the microsatellite receives the AIS signal of the target ship, transmit it to the ground; S3. According to the AIS signal reception situation, select to enter the regular imaging tracking mode under stable AIS signal reception or the search imaging tracking mode under short-term loss of AIS signal, and conduct relay tracking on the target ship. The advantages are: it can greatly compress the amount of remote sensing image data transmitted for such tasks and the time required to complete the tasks, and support high-timeliness task requirements such as escorting ships in the open sea and far areas and searching for and rescuing ships at sea.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent interpretation and application of remote sensing images, and particularly to a method and system for relay tracking application of large and medium-sized ships in the open sea. Background Art

[0002] Carrying out escort guarantee for ships in the open sea and remote areas and emergency search and rescue of ships in the wide sea area based on remote sensing means is of great significance for safeguarding maritime rights and interests, ensuring national and people's livelihood security, etc. At present, the development of low-earth orbit satellite constellations is rapid, and many low-earth orbit satellite constellation missions have been proposed and launched into orbit; with the support of on-board intelligent processing and inter-satellite communication technologies, through the multi-satellite cooperation method, low-earth orbit micro-nano satellite constellations support various space-based accompanying guarantee missions in orbit, such as relay tracking of ships in the open sea. Space-based relay tracking of ships in the open sea refers to using multiple earth observation satellites passing over the vicinity of the ship's position to obtain the AIS signal or remote sensing imaging of the large and medium-sized ships far from land and in the open sea at short time intervals and continuously for multiple times. By processing the obtained information, the continuous mastery of information such as the geographical location and identity of the ship can be realized.

[0003] In recent years, the rise of artificial intelligence has driven the rapid development and technological innovation in fields such as computer vision and natural language processing. In the field of remote sensing, many intelligent interpretation methods based on deep learning have been proposed and achieved breakthroughs and applications in fields such as remote sensing target detection, target fine recognition, and target segmentation. Using lightweight intelligent interpretation algorithms deployed on remote sensing satellites for ship detection and fine recognition in the sea is the key means to realize ship relay tracking. This can greatly reduce the amount of data transmitted between the satellite and the ground, greatly reduce the workload of personnel, and meet the requirements of rapid processing and high-timeliness application tasks. At the same time, it is also the only way to develop an intelligent satellite system that does not rely on ground stations and has the ability of on-orbit autonomous intelligent processing.

[0004] However, searching for and detecting ships in wide - area sea remote - sensing images in the traditional way of traversing remote - sensing images with a sliding window is undoubtedly like looking for a needle in a haystack. This method will consume a large amount of computing and storage resources of the satellite platform, with poor timeliness of information acquisition, seriously restricting the effectiveness of satellite applications. The Automatic Identification System (AIS) is a maritime radio communication system that can automatically broadcast ship dynamic information (position, speed, heading, etc.) and static information (ship name, nationality, call sign, draft, etc.). Due to the mandatory installation and self - reporting characteristics of AIS, AIS has been gradually popularized and applied in various aspects such as sea area perception and sea - surface surveillance, and its maritime application value has become increasingly prominent. Equipping a remote - sensing satellite with an on - board ship identification subsystem (on - board AIS system) can receive, process, and extract AIS information transmitted by ship AIS devices in real - time on the satellite, realizing the identification of ship target attributes, position acquisition, etc. Currently, the coverage range of the on - board AIS receiver is about 2000 km (at an orbital altitude of 500 km), and the positioning accuracy is better than 20 m (GPS positioning accuracy). Therefore, combining the two means of AIS signal acquisition and on - orbit remote - sensing imaging, and at the same time using the ship positioning information provided by AIS as a guide, designing the search direction and regional scope specifically, and performing intelligent on - orbit interpretation of the remote - sensing images in this regional scope is a better way to conduct wide - area sea - ship target search, detection, and identification.

[0005] With the continuous development of satellite mass - production capabilities and inter - satellite communication network technologies, the Earth - observing satellite system is gradually evolving towards large - scale, flexible, and sustainable development. Based on satellite constellations and high - speed inter - satellite communication, constructing an instant remote - sensing satellite constellation has become the current mainstream. By giving full play to the advantages of satellite constellation cooperation and combining intelligent on - orbit processing to reduce time delays in each link, improving the timeliness and accuracy of data processing, and realizing fully automated and intelligent near - real - time satellite constellation intelligent services, it can lay a foundation for realizing all - weather, all - day - long, fast, efficient, and intelligent services for Earth observation.

[0006] Therefore, the present invention proposes a method and system for relay tracking of large and medium - sized ships in the open sea, which is based on multi - source information support, cooperation between high - and low - orbit multiple satellites, and cooperation between space, ground, human, and machine. The low - orbit micro - nano remote - sensing satellite is equipped with an on - board AIS receiving payload. In addition to the remote - sensing imaging ability, the satellite also has the ability to receive, analyze, and compare AIS signals. Using the on - board AIS receiver and the optical remote - sensing imaging payload, through the fusion and complementarity of ship AIS information and optical remote - sensing imaging information, relay tracking of large and medium - sized ships in the open sea is realized. Summary of the Invention

[0007] The purpose of the present invention is to provide a method and system for relay tracking of large and medium - sized ships in the open sea, so as to solve the foregoing problems existing in the prior art.

[0008] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0009] A method for relay tracking application of large and medium-sized ships in the far sea, comprising the following steps:

[0010] S1. Set a concerned area according to the current position information of the target ship obtained, and calculate the satellites passing over the concerned area within half an hour; determine the current position of the target ship for the satellite uploading task of passing over the concerned area in the short term; after the microsatellite receives the AIS signal of the target ship, transmit it to the ground.

[0011] S2. When the AIS signal reception is in a stable reception situation, enter the regular imaging tracking mode in the case of stable AIS signal reception to perform relay tracking on the target ship. If the AIS signal is lost during operation, switch to the search imaging tracking mode in the case of short-term loss of the AIS signal; when the AIS signal reception is in a short-term loss situation, enter the search imaging tracking mode in the case of short-term loss of the AIS signal to perform relay tracking on the target ship. If the AIS signal is restored during operation, switch to the regular imaging tracking mode in the case of stable AIS signal reception.

[0012] In both modes, if the planned preset operation time is exceeded, return to step S1, and use the updated ship position as a reference to perform the task planning for the next preset operation time. Otherwise, continue to run in the current mode.

[0013] Preferably, step S1 specifically includes the following contents:

[0014] S11. If it has been more than half an hour since the last satellite overpass calculation, set a concerned area centered on the current navigation position of the target ship obtained, and use this concerned area as a reference to calculate the low-earth remote sensing satellites that will pass over this concerned area within the next half hour and have an observation side-sway angle less than 45° according to the prior information such as the coverage range of the satellite-borne AIS receiver and the satellite orbit parameters.

[0015] S12. Upload tasks to the above satellites that will pass over the concerned area in the short term, that is, receive and analyze the ship AIS signal, and determine the current position of the target ship by comparing and obtaining the ship identity in the AIS information.

[0016] S13. After the microsatellite stably and continuously receives and identifies the AIS signal of the target ship, it will regularly transmit the information to the ground through the inter-satellite topology communication link or the geostationary communication satellite.

[0017] Preferably, the regular imaging tracking mode in the case of stable AIS signal reception specifically includes the following contents: A1. Turn on the satellite AIS receiver for the overpass circular concerned area.

[0018] A2. The satellite that has recognized the AIS signal of the target ship transmits the latest obtained ship navigation information to the ground every minute at regular intervals.

[0019] A3. Check whether it is the time for regular imaging. If it is not the time for regular imaging, return to A1 for execution; if it is the time for regular imaging, start the imaging tracking process.

[0020] A4. Predict the future position of the target ship based on the recently obtained target ship status, plan and upload the imaging task to the overpass satellite.

[0021] A5. The overpass satellite receives the AIS signal of the target ship and images the position area of the target ship.

[0022] A6. With the position of the target ship broadcast by the AIS at the imaging moment as the center, a 1024*1024 pixel area is used as the range of an intelligent detection and recognition unit to form multiple candidate detection and recognition units, and the adjacent candidate detection and recognition units overlap by 25% in area; start the detection and recognition layer by layer and unit by unit from the center outwards; the candidate detection and recognition units that are more consistent with the course direction of the target ship and closer in distance have a higher detection and recognition priority, so they are detected and recognized first.

[0023] A7. If the target ship is not recognized, return to A3 for execution; if the target ship is recognized, transmit the sliced image of the target ship and the AIS information data of the target ship obtained at the imaging moment to the ground, and update the navigation status of the target ship.

[0024] A8. If the target ship tracking task is not completed, return to A1 for execution, otherwise end.

[0025] Preferably, step A6 is specifically as follows: with the real-time position of the ship broadcast by the AIS of the target ship at the imaging moment as the center, a 1024*1024 pixel area is used as the range of an intelligent detection and recognition unit to form multiple candidate detection and recognition units, and the adjacent candidate detection and recognition units overlap by 25% in area; start the detection and recognition layer by layer and unit by unit from the center outwards; the candidate detection and recognition units that are more consistent with the course direction of the target ship and closer in distance have a higher detection and recognition priority, so they are detected and recognized first.

[0026] Preferably, the detection and recognition is implemented based on a deep learning network, and the deep learning network is the improved YOLO_v10 network; among them,

[0027] The scale of the input image of the network is 1024×1024×3, and the scales of the prediction output layers are 128×128, 64×64, and 32×32.

[0028] The network uses the Circular Smooth Label algorithm to modify the target detection to rotated bounding box detection; the non-maximum suppression algorithm for rotated targets is used in the output part of the network.

[0029] After the HEAD module at the top layer of the network and before the prediction layer, a plug-and-play lightweight convolutional block attention module is added, and this module combines the channel attention mechanism and the spatial attention mechanism.

[0030] Preferably, the search imaging tracking mode in the case of short-term loss of AIS signals specifically includes the following content:

[0031] B1. According to the obtained recent navigation status of the target ship, predict the possible route of the target ship within the next 15 minutes;

[0032] B2. Taking the predicted route of the target ship as a reference, calculate the satellites passing over the target ship within 15 minutes; select three satellites with an interval close to 5 minutes from them, and plan and upload imaging tasks to the selected imaging satellites;

[0033] B3. The three satellites image the predicted positions in sequence, and with the predicted position corresponding to the target ship at the imaging moment as the center, conduct intelligent detection and identification of the target ship in regions from the inside out;

[0034] B4. If a satellite among the three satellites detects and identifies the target ship, transmit the sliced image of the target ship and the information data of the imaging moment to the ground, and update the navigation status of the target ship;

[0035] B5. If none of the three satellites search and identify the target ship, end the task, otherwise return to B3 for execution; B6. If the target ship tracking task is not ended, return to B1 for execution, otherwise end the task.

[0036] Preferably, step B3 is specifically as follows:

[0037] When each satellite conducts intelligent detection and identification of the target ship in the remote sensing image, with the predicted position of the target ship at the current imaging moment as the center, search layer by layer and unit by unit from the inside out. The units that are more consistent with the heading direction of the target ship and closer in distance have a higher detection and identification priority and are detected and identified first;

[0038] Assume S n-5 to S n-1 are the actual positions of the target ship at the most recent times when it was searched and identified. Sn-1 is the actual position of the target ship at the most recent time when it was searched and identified; S n-1 S' is the track line of uniform linear navigation starting from the actual position of the target ship at the most recent time when it was searched and identified, with its heading as the navigation direction and the average speed as the navigation speed; S n-1 The three positions PA, PB, and PC on S' are the predicted positions of the target ship corresponding to the imaging moments of the three satellites selected with an interval close to 5 minutes; then if the satellite imaging at the PA position fails to search and identify the target ship, and the satellite imaging at the PB position searches and identifies the target ship, and the actual position of the target ship is at S n , then next, according to the actual position S n of the target ship at the most recent time when it was searched and identifiedStarting from [starting point], with its then course as the navigation direction and the average speed as the navigation speed, perform uniform linear navigation to obtain a new track line S n S”; Repeat the above process to achieve the search, imaging, and tracking of the target ship in the case of short-term loss of AIS signals.

[0039] Preferably, in step S2, when operating in the regular imaging and tracking mode under stable reception of AIS signals, if the ground gives an end task instruction, stop the current task; otherwise, continue to operate in the current mode.

[0040] When operating in the search and imaging mode in the case of short-term loss of AIS signals, if the ground gives an end task instruction, stop the current task; otherwise, first judge the situation of ship target search, imaging, and tracking. If the target ship is lost, stop the current task. If it is possible to maintain tracking of the target ship by combining the predicted position with imaging search, continue to operate in the current mode.

[0041] The purpose of the present invention also lies in providing a system for relay tracking applications of large and medium-sized ships in the open sea. The system can implement the above-mentioned method. The system includes a remote sensing image access unit, a spaceborne AIS receiver, a remote sensing image intelligent processing unit, and a comprehensive interface backplane.

[0042] The remote sensing image access unit is composed of a field programmable gate array, a random access memory, and a solid-state drive. On the one hand, it provides a high-speed data transmission channel for reading the original remote sensing image from the optical remote sensing imaging payload, and also provides a high-speed data transmission channel for the transfer of the original remote sensing image between the remote sensing image intelligent processing unit and the satellite platform. On the other hand, it is used for storing the original remote sensing image, the sliced remote sensing image after intelligent processing, and the operating system, software, and models required for intelligent processing.

[0043] The spaceborne AIS receiver receives ship AIS signals and completes data parsing.

[0044] The remote sensing image intelligent processing unit is composed of a central processing unit and a graphics processing unit, mainly providing an algorithm operation platform for remote sensing image preprocessing and intelligent detection and recognition of ships in remote sensing images; it runs the Linux kernel operating system, deploys image preprocessing algorithms and deep learning algorithms related to intelligent detection and recognition of large and medium-sized ship targets, and provides strong program control management, data stream processing, and parallel computing functions for image intelligent processing.

[0045] The integrated interface baseboard consists of a power control unit and multiple types of interface channels. On the one hand, it provides power control for the remote sensing image access unit, on-board AIS receiver, and remote sensing image intelligent processing unit in the system, and provides interface management between the remote sensing image access unit, on-board AIS receiver, and remote sensing image intelligent processing unit and the satellite platform. On the other hand, it provides mechanical installation support for the remote sensing image access unit, on-board AIS receiver, and remote sensing image intelligent processing unit, supports modular and stacked design, and is easy to implement personalized function combination design.

[0046] Preferably, the files stored in the solid-state drive include the operating system, software programs, and dependency libraries required for the operation of application programs in the remote sensing image intelligent processing unit; relevant parameter files for preprocessing newly acquired remote sensing images; preprocessed remote sensing images; deep learning network structures, model parameter files, and configuration files for ship target detection and recognition.

[0047] The random access memory is used to provide a cache for the read and write operations of the original remote sensing image data, preprocessed remote sensing images, and intelligent processing result data.

[0048] The beneficial effects of the present invention are as follows: (1) Compared with traditional wide-area ship search and recognition, the present invention integrates ship AIS signal data and satellite remote sensing image data, and uses AIS positioning data to guide the imaging satellite to complete ship target search and recognition faster and autonomously in orbit. Compared with the traditional cumbersome data processing and application processes such as remote sensing data downlink, ground processing, and uplink control, the present invention can greatly compress the transmission volume of remote sensing image data for carrying out such tasks and the time required to complete the tasks, and supports high-timeliness task requirements such as ship escort in the far sea and far area and ship search and rescue at sea. (2) The present invention proposes a ship tracking process based on AIS information-guided remote sensing imaging, and designs the tracking process according to the AIS signal status, dividing it into two cases of "stable AIS signal reception" and "short-term loss of AIS signal" and their state transitions. The regular imaging tracking process under stable AIS reception can achieve all-weather, all-day, and global ship target tracking capabilities; the search imaging tracking process under short-term loss of AIS signal can achieve ship target tracking capabilities under good weather (cloudless or thin clouds), daylight conditions, and globally. The two tracking modes can be converted well to ensure the smooth execution of ship target tracking tasks. (3) The system proposed by the present invention has a simple composition and low cost, and can provide a modular and intelligent solution for small satellites to achieve relay tracking of large and medium-sized ships at sea. The system has good portability, compatibility, and expandability. Description of the Drawings

[0049] Figure 1 is a flowchart of the method for relay tracking application in an embodiment of the present invention;

[0050] Figure 2 It is a flowchart of the regular imaging tracking mode under the condition of stable reception of AIS signals in the embodiments of the present invention;

[0051] Figure 3 It is a schematic diagram of a method for searching and identifying ship targets in remote sensing images guided by position information in the embodiments of the present invention;

[0052] Figure 4 It is a structural diagram of a deep learning network in the embodiments of the present invention;

[0053] Figure 5 It is a flowchart of the search imaging tracking mode under the condition of short-term loss of AIS signals in the embodiments of the present invention;

[0054] Figure 6 It is a schematic diagram of a method for searching and identifying ship targets guided by ship track prediction in the embodiments of the present invention;

[0055] Figure 7 It is a structural diagram of a system for relay tracking application in the embodiments of the present invention. Specific implementation manners

[0056] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific implementation manners described herein are only used to explain the present invention and are not used to limit the present invention.

[0057] Currently, satellite platform and payload technologies, communication technologies, etc. related to low-earth orbit micro-nano remote sensing satellites are developing rapidly. Low-earth orbit micro-nano remote sensing satellites have technical capabilities such as high-resolution imaging, large swath imaging, inter-satellite high-speed communication, and multi-payload compatibility; combined with the empowerment of on-orbit intelligent processing technology, large-scale remote sensing satellite constellations can achieve global high revisit frequency, on-orbit autonomous intelligent processing, multi-source information fusion application, near-real-time data downlink, etc. The above-mentioned ability advantages of large-scale remote sensing satellite constellations can provide solutions for tasks with high timeliness requirements, such as carrying out escort guarantees for large and medium-sized ships in the open sea and far sea areas based on space-based information, and wide-area emergency search and rescue of ships at sea. Therefore, the present invention proposes a method and system for relay tracking application of large and medium-sized ships in the open sea, which is based on multi-source information support, high and low-orbit multi-satellite cooperation, and space-ground-human-machine collaboration. The low-earth orbit micro-nano remote sensing satellite is equipped with an on-board AIS receiving payload. In addition to the remote sensing imaging ability, the satellite also has the ability to receive, analyze and compare AIS signals. By using the on-board AIS receiver and optical remote sensing imaging payload, through the fusion and complementarity of ship AIS information and optical remote sensing imaging information, relay tracking of large and medium-sized ships in the open sea is realized. As Figure 1 shown, in this embodiment, the method of the present invention includes the following four parts:

[0058] I. Acquisition of target ship position information

[0059] Obtain the current position (latitude and longitude) information of the target ship.

[0060] II. Calculation of over - head satellites based on the area of interest and the tasks of receiving and verifying the uplinked AIS signals

[0061] Set the area of interest according to the current position information of the target ship, and calculate the over - head satellites within half an hour based on the area of interest; determine the current position of the target ship for the satellite uplink task that will pass over the area of interest in the short term; after the microsatellite receives the AIS signal of the target ship, transmit it to the ground. The specific contents are as follows.

[0062] 2.1. If the time since the last satellite over - head calculation exceeds half an hour, set the area of interest centered on the current navigation position of the ship target (such as a circular area with a diameter of 20 km), and based on this area, calculate the low - earth - orbit remote - sensing satellites that will pass over this area of interest (the AIS signal reception range covers this area of interest) within the next half hour and the observation side - swing angle is less than 45° according to prior information such as the coverage range of the satellite - borne AIS receiver and the satellite orbit parameters.

[0063] 2.2. Uplink the task to the above - mentioned satellites that will pass over the area of interest in the short term, that is, receive and analyze the AIS signal of the ship, determine the current position of the target ship by comparing and obtaining the ship identity in the AIS information; the signal reception coverage range of the satellite - borne AIS receiver is wide (the satellite - borne AIS signal reception range at an orbital altitude of 500 km can reach more than 2000 km). In the constellation background, multiple satellites can achieve multiple coverage of the area of interest centered on the ship, and can continuously monitor the AIS signal of the ship.

[0064] 2.3. After the microsatellite stably and continuously receives and identifies the AIS signal of the target ship, it will transmit the information to the ground at regular intervals (such as once a minute) through the inter - satellite topology communication link or the geostationary communication satellite.

[0065] III. Select the corresponding mode to relay - track the target ship according to the AIS signal reception situation

[0066] When the AIS signal reception is in a stable reception state, enter the periodic imaging tracking mode under the stable reception of the AIS signal to perform relay tracking on the target ship. If the AIS signal is lost during operation, switch to the search imaging tracking mode under the short-term loss of the AIS signal; when the AIS signal reception is in a short-term loss state, enter the search imaging tracking mode under the short-term loss of the AIS signal to perform relay tracking on the target ship. If the AIS signal is restored during operation, switch to the periodic imaging tracking mode under the stable reception of the AIS signal. In these two modes, if the planned preset operation time is exceeded, return to step S2, and use the updated ship position as a reference to perform the task planning for the next preset operation time. Otherwise, continue to run in the current mode.

[0067] When running in the periodic imaging tracking mode under the stable reception of the AIS signal, if the ground gives an end task instruction, stop the current task. Otherwise, continue to run in the current mode; when running in the search imaging mode under the short-term loss of the AIS signal, if the ground gives an end task instruction, stop the current task. Otherwise, first judge the search imaging tracking situation of the ship target. If the target ship is lost, stop the current task. If it is possible to maintain tracking of the target ship by combining the predicted position with the imaging search, continue to run in the current mode.

[0068] In this embodiment, as Figure 2 shown, the periodic imaging tracking mode under the stable reception of the AIS signal specifically includes the following contents.

[0069] (1). Turn on the satellite AIS receiver in the overhead circular attention area; under the condition of the micro-nano satellite constellation, coupled with the wide coverage of the on-board AIS signal reception, multiple satellites can continuously acquire, analyze, and identify the AIS signals of target ships.

[0070] (2). The satellite that identifies the AIS signal of the target ship will transmit the latest obtained ship navigation information, such as ship position, heading, speed, etc., to the ground every minute through inter-satellite high-speed communication or geostationary orbit communication satellites.

[0071] (3). Check whether it is the timing imaging moment (once every 10 minutes). If it is not the timing imaging moment, return to A1 to execute; if it is the timing imaging moment, start the imaging tracking process.

[0072] (4) According to the recently obtained ship status, assume that the ship sails at a constant speed in a straight line approximately according to the recently obtained course and speed, and predict the position of the ship at a certain future moment, which is the moment when the satellite that will pass over this position closest to the predicted position of the target ship is closest to the target ship. Thus, the most suitable satellite for the remote sensing imaging task can be selected; upload the remote sensing imaging task to the satellite undertaking the imaging tracking task. The imaging payload startup time can be selected as the moment calculated above. The imaging side-sway angle is determined based on the real-time position of the target ship obtained by the satellite at this moment. The satellite can perform preliminary rough pose maneuvers according to the real-time position of the target ship to ensure that the position of the target ship is near the center position of the remote sensing image.

[0073] (5) The satellite that has received the imaging task instruction adjusts its attitude according to the continuously received AIS information of the target ship. The goal is to keep the target ship within the range near the center of the remote sensing image. Before approaching the imaging startup moment, it will stabilize the attitude in advance to ensure the imaging quality; at the planned imaging moment, the satellite remote sensing payload starts up for imaging and records the AIS information of the target ship obtained at the imaging moment.

[0074] (6) The imaging satellite preprocesses the obtained remote sensing image on board; according to the AIS information of the target ship obtained at the imaging moment, information such as the position, course, and speed of the ship target can be obtained; with the position of the target ship at the imaging moment as the center, the target ship is detected and recognized intelligently in regions from the inside out.

[0075] The intelligent detection and recognition in this process adopt a method for searching and recognizing ship targets in remote sensing images guided by position information. The specific method is as follows: Figure 3 As shown in the figure, point D in the figure is the real-time position of the ship broadcast by the AIS of the target ship at the imaging moment, and the error between this position and the actual position is better than 20 meters. With position D as the center, a 1024*1024 pixel is the range of an intelligent detection and recognition unit, forming 9 candidate detection and recognition units, and the adjacent monitoring and recognition units overlap by 25% in area. The numbers 1-6 within the box in the figure show the priority order of detection and recognition, and the smaller the number, the higher the priority. The unit where position D is located has the highest priority and is detected and recognized first. It can be seen from the figure that the units more consistent with the course direction of the target ship and closer in distance have higher detection and recognition priorities, so they are detected and recognized earlier.

[0076] The above detection and recognition process is realized relying on a deep learning network. As Figure 4 shown, the deep learning network is improved based on the YOLO_v10 network. Compared with the YOLO_v10 network, the deep learning network for detecting and recognizing large and medium-sized ship targets at sea has the following main improvements:

[0077] (1) The scale of the input image of the network is 1024×1024×3, and the scales of the prediction output layers are 128×128, 64×64, and 32×32.

[0078] (2) Using the Circular Smooth Label (CSL) algorithm, the object detection is modified to rotated bounding box detection to achieve better envelopment of rotated objects. At the same time, the non-maximum suppression algorithm (Non-Maximum Suppression, NMS) in the network output part is correspondingly modified to the NMS_obb algorithm for rotated objects.

[0079] (3) After the HEAD module at the top layer of the network and before the prediction layer, a plug-and-play lightweight convolutional block attention module (Convolutional Block Attention Module, CBAM) is added. This attention module that combines channel attention mechanism and spatial attention mechanism can promote the improvement of the representation ability of the convolutional neural network in the feature extraction process, enabling the model to pay more attention to the significant features that play a greater role in the detection and recognition of target ships and improving the detection and recognition performance.

[0080] (7) If the target ship is not recognized, return to (3) for execution. If the target ship is recognized, the sliced image of the target ship and the AIS information data of the target ship obtained at the imaging time are transmitted to the ground through the inter-satellite communication link or the geostationary orbit communication satellite; update the navigation state of the target ship, including the sliced image, position, heading, speed, etc.

[0081] (8) If the target ship tracking task is not completed, return to (1) for execution, otherwise end.

[0082] In this embodiment, in the regular imaging tracking mode under stable reception of AIS signals, guided by using AIS signals to obtain the real-time position of the ship, the imaging satellite can be scheduled near-real-time to image the area where the ship is located regularly (such as every 10 minutes), and through on-board intelligent processing, ship target detection, recognition, and comparison and confirmation are realized. If the weather conditions are good (such as cloudless or thin cloud conditions, daytime), after accurately recognizing and confirming the ship, the sliced image of the target ship can be transmitted to the ground through the inter-satellite topological communication link or the geostationary orbit communication satellite; if the target ship cannot be detected and recognized due to poor weather (such as thick cloud conditions, night), but since the satellite can stably receive the AIS signal of the target ship at this time, it is considered that the state of the target ship is normal and no other supplementary means for imaging are required.

[0083] In this embodiment, such asFigure 5 As shown in the figure, the search imaging tracking mode in the case of short-term loss of AIS signals specifically includes the following content:

[0084] (1) Predict the possible route of the target ship within the next 15 minutes based on the recently obtained navigation status of the target ship; the predicted route approximately conducts uniform linear navigation with the recently obtained ship heading and speed as the initial state.

[0085] (2) Taking the predicted route of the target ship as a reference, calculate the satellites that pass over the target ship within 15 minutes, that is, calculate the corresponding moments, ship predicted positions, and satellite positions when the distance between the passing-over satellite and the predicted position of the target ship is the closest; select 3 satellites with an interval close to 5 minutes from them as the satellites to carry out the imaging search and tracking tasks.

[0086] Upload the remote sensing imaging task to the satellites undertaking the search imaging tracking tasks. The imaging payload startup time can select the moments calculated above, and the imaging side-sway angle is determined based on the predicted position of the target ship; the satellite can perform pre-positioning and pose maneuvers according to the predicted position of the target ship to ensure that the position of the target ship is near the center position of the remote sensing image.

[0087] (3) For the three satellites undertaking the search imaging tracking tasks, respectively complete the preprocessing of the obtained remote sensing satellite images on board; with the predicted position corresponding to the target ship at the imaging moment as the center, conduct intelligent detection and recognition of the target ship in regions from the inside to the outside.

[0088] The intelligent detection and recognition in this process adopts the ship target search and recognition method guided by ship trajectory prediction. The specific method is as follows: Figure 6 As shown in the figure, S n-5 to S n-1 are the actual positions of the target ship when it was most recently searched and recognized; Sn-1 is the actual position of the target ship when it was most recently searched and recognized; S n-1 S’ is the track line of uniform linear navigation starting from the actual position of the target ship when it was most recently searched and recognized, with its heading as the navigation direction and the average speed as the navigation speed; S n-1 The three positions PA, PB, and PC on S’ are the predicted positions of the target ship corresponding to the imaging moments of the three selected satellites with an interval close to 5 minutes. When the satellite conducts intelligent detection and recognition of the target ship in the remote sensing image, with the predicted position of the target ship at the current imaging moment as the center, search layer by layer and unit by unit from the inside to the outside (similar to Figure 3 as shown in the figure). The units that are more consistent with the heading direction of the target ship and closer in distance have a higher detection and recognition priority and are detected and recognized first. Figure 6A typical possible result is given, that is, the satellite imaging the PA position fails to search and identify the target ship, while the satellite imaging the PB position searches and identifies the target ship, and the actual position of the target ship is at S n . Then, next, the actual position S n when the target ship was last searched and identified will be used as the starting point, its heading at that time will be used as the sailing direction, and the average speed will be used as the sailing speed to perform uniform linear sailing to obtain a new track line S n S". Repeat the above process to achieve the search, imaging and tracking of the target ship in the case of short-term loss of AIS signals.

[0089] (4) If a satellite among the three satellites detects and identifies the target ship, the sliced image of the target ship and the information data of the imaging time will be transmitted to the ground through the inter-satellite communication link or the geostationary orbit communication satellite; update the navigation status of the target ship, including the sliced image and position. According to the sliced image, the target heading information can also be obtained. By comparing with the position, time and other information when the target ship was last searched and identified, the average speed and other information of the target ship can be roughly obtained.

[0090] (5) If none of the three satellites search and identify the target ship, the mission will end; otherwise, return to (3) to execute.

[0091] (6) If the target ship tracking mission has not ended, return to (1) to execute; otherwise, end the mission.

[0092] In this embodiment, in the search imaging tracking mode under the condition of short-term loss of AIS signals, when the AIS signals of the target ship cannot be received from more than three overpassing satellites, the imaging means is activated to search for the target ship to achieve tracking and accompaniment. Under the background of the satellite constellation, based on the information such as the position and course of the ship target received most recently, the running direction and position of the ship are predicted, and the micro-nano satellite with the fastest overpass is determined based on this information. After the micro-nano satellite images, it will search, identify and confirm the target ship in the remote sensing image layer by layer from the inside out with the predicted position of the target ship as the center; the current speed and course of the target ship are updated and calculated based on the information such as the current position and the position information at the previous moment of the target, and the target detection azimuth. After confirming the identity of the target ship, the sliced image of the target ship and the navigation status information (such as speed, course, etc.) are transmitted to the ground through the inter-satellite topology communication link or the geostationary communication satellite in a timely manner. If the weather conditions are good (such as cloudless or thin cloud conditions, daytime), the relay tracking of the target ship is achieved through the above "position prediction" and "search and identification" process methods. If the weather conditions are poor (such as thick cloud conditions, night), then consider scheduling the radar imaging satellite for imaging search (since this invention focuses on using AIS signals as the main means and optical remote sensing imaging as the auxiliary means to solve the task of tracking and accompanying the target ship under good weather conditions, the relay tracking of the target ship based on radar imaging is not described in detail in this invention).

[0093] In this embodiment, as Figure 7 shown, a system for relay tracking application for large and medium-sized ships in the open sea is also provided. The system consists of four parts: a remote sensing image access unit, an on-board AIS receiver, a remote sensing image intelligent processing unit, and an integrated interface backplane.

[0094] (1) The remote sensing image access unit is composed of a field programmable gate array (FPGA), a random access memory (RAM), and a solid state drive (SSD). On the one hand, it provides a high-speed data transmission channel for reading the original remote sensing image from the optical remote sensing imaging payload, and also provides a high-speed data transmission channel for the transfer of the original remote sensing image between the remote sensing image intelligent processing unit and the satellite platform. On the other hand, it is used for storing the original remote sensing image (read from the remote sensing imaging payload), the sliced remote sensing image after intelligent processing (output by the remote sensing image intelligent processing unit), as well as the operating system, software, models, etc. required for intelligent processing.

[0095] The remote sensing image access unit is connected to the remote sensing image intelligent processing unit and the integrated interface backplane through a high-speed LVDS communication interface, supporting the remote sensing image intelligent processing unit to read remote sensing image data from the solid state drive for preprocessing and intelligent detection and identification applications, supporting the processed data to be written into and stored in the solid state drive, and supporting the satellite platform to read relevant data from the optical remote sensing imaging payload or the solid state drive.

[0096] The solid-state drive (SSD) is used to store the following files: ① basic files such as the operating system, software programs, and dependent libraries required for the operation of application programs in the intelligent remote sensing image processing unit; ② parameter files related to the preprocessing (radiometric correction and geometric correction) of newly acquired remote sensing images; ③ preprocessed remote sensing images; ④ deep learning network structures, model parameter files, and configuration parameter files for ship target detection and recognition; ⑤ result data such as ship target slice images and detection and recognition result files cropped after intelligent processing.

[0097] The random access memory (RAM) is used to provide a cache for the read and write operations of raw remote sensing image data, preprocessed remote sensing images, and result data after intelligent processing.

[0098] (2) The spaceborne AIS receiver is a receiving device for receiving ship AIS signals. Its receiving antenna is installed on the outer layer of the satellite structure and is connected to the spaceborne AIS receiver via a radio frequency cable. This device uses commercial off-the-shelf products (COTS) and is managed, controlled, and data-interacted through a CAN interface.

[0099] (3) The intelligent remote sensing image processing unit consists of a central processing unit (CPU) and a graphics processing unit (GPU), and mainly provides an algorithm operation platform for remote sensing image preprocessing and intelligent detection and recognition of ships in remote sensing images. This unit runs the Linux kernel operating system and deploys image preprocessing algorithms (radiometric correction, geometric correction, etc.) and deep learning algorithms related to intelligent detection and recognition of large and medium-sized ship targets, providing functions such as strong program control management, data stream processing, and parallel computing for image intelligent processing.

[0100] (4) The integrated interface backplane consists of a power control unit (PCU) and multiple types of interface channels. On the one hand, it provides power control for the remote sensing image access unit, on-board AIS receiver, and remote sensing image intelligent processing unit in the system, and provides interface management between the remote sensing image access unit, on-board AIS receiver, and remote sensing image intelligent processing unit and the satellite platform. On the other hand, it provides mechanical installation support for the remote sensing image access unit, on-board AIS receiver, and remote sensing image intelligent processing unit, supports modular and stacked design, and is easy to implement personalized function combination design. The power control of the power control unit is managed by the satellite platform through the CAN1 interface. By adjusting the external power provided by the satellite platform end, it can provide the required power supply voltage and current for each unit in the system, and provide power control management functions according to the satellite platform CAN communication instructions. The integrated interface backplane also provides multiple and multiple types of data interaction interfaces such as CAN and LVDS, supporting system payload or unit expansion; the remote sensing image access unit, on-board AIS receiver, and remote sensing image intelligent processing unit are interconnected through CAN2, supporting status information data interaction between each payload or unit, and this CAN communication network is connected to the satellite platform through the communication interface provided by the integrated interface backplane, supporting the satellite platform to manage the payload and unit, and supporting data interaction between it and the satellite platform. This system can be used as a subsystem of the satellite, compatible with various remote sensing satellite platforms, with good portability and strong compatibility, having strong expansion ability, supporting the integration of multiple payloads and processing units, and adapting to multi-scenario applications.

[0101] By adopting the above technical solutions disclosed in the present invention, the following beneficial effects are obtained:

[0102] The present invention provides a method and system for relay tracking applications of large and medium-sized ships in the open sea. Compared with traditional wide-area ship search and identification, the present invention integrates ship AIS signal data and satellite remote sensing image data, and uses AIS positioning data to guide imaging satellites to complete ship target search and identification faster and autonomously in orbit. Compared with the cumbersome data processing and application processes such as traditional remote sensing data downlink, ground processing, and uplink control, the present invention can greatly compress the transmission volume of remote sensing image data for carrying out such tasks and the time required to complete the tasks, and support high-timeliness task requirements such as open-sea and far-sea ship escort and maritime ship search and rescue. The present invention proposes a ship tracking process based on AIS information to guide remote sensing imaging, and designs the tracking process according to the AIS signal status, which is divided into two cases of "stable AIS signal reception" and "short-term AIS signal loss" and their state transitions. The regular imaging tracking process under stable AIS reception can achieve all-weather, all-day, and global ship target tracking capabilities; the search imaging tracking process under short-term AIS signal loss can achieve ship target tracking capabilities under good weather (cloudless or thin clouds), daylight conditions, and globally. The two tracking modes can be converted well to ensure the smooth execution of ship target tracking tasks. The system proposed by the present invention has a simple composition and low cost, and can provide a modular and intelligent solution for small satellites to achieve relay tracking of large and medium-sized ships at sea. The system has good portability, compatibility, and expandability.

[0103] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for relay tracking application of large and medium-sized ships in the open sea, characterized by: The following steps are included: S1. Delineate a focus area based on the current position information of the target ship, and calculate the satellites that will pass overhead within half an hour based on the focus area; assign tasks to satellites that will pass overhead in the focus area in the short term to determine the current position of the target ship; After receiving the AIS signal from the target ship, the micro-nano satellite transmits it to the ground; S2. When the AIS signal reception is stable, enter the periodic imaging tracking mode under the condition of stable AIS signal reception, and relay track the target ship. If the AIS signal is lost during operation, switch to the search imaging tracking mode under the condition of short-term loss of AIS signal; When the AIS signal reception is short-term loss, enter the search imaging tracking mode under the condition of short-term loss of AIS signal, and relay track the target ship. If the AIS signal is restored during operation, switch to the periodic imaging tracking mode under the condition of stable AIS signal reception; In both modes, if the planned preset running time is exceeded, the process returns to step S1, and the task planning for the next preset running time is performed with reference to the updated ship position, otherwise, the current mode is maintained and continued to run; The search imaging tracking mode in the case of short-term loss of AIS signal includes the following: B1. Based on the recent navigation status of the target ship, predict the possible route of the target ship in the next 15 minutes; B2. Using the predicted target ship route as a reference, calculate the satellites that will pass over the target ship within 15 minutes; select three satellites with an interval of nearly 5 minutes, and plan and assign imaging tasks to the selected imaging satellites; B3. The three satellites image the predicted position in turn, and use the predicted position of the target ship at the time of imaging as the center to perform intelligent detection and identification of the target ship in different areas from the inside to the outside; B4. If one of the three satellites detects and identifies the target ship, the target ship slice image and imaging time information data will be transmitted to the ground, and the navigation status of the target ship will be updated; B5. If none of the three satellites can find the target ship, the mission ends. Otherwise, the mission returns to B3. B6. If the target ship tracking mission is not completed, return to B1 for execution; otherwise, the mission is terminated.

2. The method for relay tracking of large and medium-sized ships at sea according to claim 1 is characterized in that: Step S1 specifically includes the following contents: S11. If the time since the last satellite pass is more than half an hour, a focus area is delineated with the current navigation position of the target ship as the center, and the focus area is used as a reference. Based on the coverage of the satellite-borne AIS receiver and the satellite orbit parameters, the low-orbit remote sensing satellite that will pass over the focus area in the next half an hour and have an observation side swing angle of less than 45° is calculated. S12, assigning a task to the satellites that will be over the area of ​​interest in the short term, i.e., receiving and analyzing the AIS signals of ships, obtaining the identities of the ships in the AIS information by comparison, and determining the current position of the target ship; S13. After the micro-nano satellite stably and continuously receives and identifies the AIS signal of the target ship, it will transmit the information to the ground through the inter-satellite topology communication link or the geostationary orbit communication satellite at a regular interval.

3. The method for relay tracking of large and medium-sized ships at sea according to claim 1 is characterized in that: The periodic imaging tracking mode under the condition of stable AIS signal reception specifically includes the following contents: A1. Turn on the satellite AIS receiver over the circular area of ​​interest; A2: The satellite that recognizes the AIS signal of the target ship will transmit the latest navigation information of the ship to the ground every minute; A3, check whether it is the scheduled imaging time, if not, return to A1 to execute; if it is the scheduled imaging time, start the imaging tracking process; A4. Predict the future position of the target ship based on the most recently acquired state of the target ship, and plan and upload the imaging mission to the overhead satellite; A5, the overhead satellite receives the AIS signal of the target ship and images the target ship's location area; A6. Overhead satellites use the position of the target ship at the time of imaging as the center and perform intelligent detection and identification of the target ship in different areas from the inside to the outside; A7. If the target ship is not identified, return to A3 for execution; if the target ship is identified, transmit the target ship slice image and the target ship AIS information data acquired at the imaging time to the ground, and update the navigation status of the target ship; A8. If the target ship tracking mission is not completed, return to A1 for execution; otherwise, end.

4. The method for relay tracking of large and medium-sized ships at sea according to claim 3 is characterized in that: Step A6 is specifically as follows: taking the real-time position of the target ship AIS broadcast at the imaging moment as the center, 1024*1024 pixels as an intelligent detection and identification unit range, forming multiple candidate detection and identification units, and adjacent candidate detection and identification units overlap by 25% area; starting from the center, detection and identification are performed layer by layer and unit by unit from the inside to the outside; the candidate detection and identification unit that is more consistent with the heading direction of the target ship and the closer the distance is, the higher the detection and identification priority is, and therefore the detection and identification is performed first.

5. The method for relay tracking application of large and medium-sized ships at sea according to claim 4 is characterized in that: Detection and recognition are based on a deep learning network, which is an improved YOLO_v10 network. The scale of the network input image is 1024×1024×3, and the scale of the prediction output layer is 128×128, 64×64, and 32×32; The network uses the Circular Smooth Label algorithm to modify the target detection into the rotation box detection; the network output part uses the non-maximum suppression algorithm for rotating target applications; After the HEAD module at the top layer of the network and before the prediction layer, a plug-and-play lightweight convolutional block attention module is added, which combines the channel attention mechanism and the spatial attention mechanism.

6. The method for relay tracking of large and medium-sized ships at sea according to claim 1 is characterized in that: Step B3 specifically comprises: When each satellite performs intelligent detection and identification of target ships in remote sensing images, it takes the predicted position of the target ship at the current moment of imaging as the center, and searches from the inside to the outside layer by layer and unit by unit. The unit with the more consistent heading direction and the closer distance to the target ship has a higher detection and identification priority, and is detected and identified first. Assume S n-5 To S n-1 Sn is the actual position of the target ship when it was searched and identified most recently, Sn-1 is the actual position of the target ship when it was searched and identified most recently; S n-1 S' is the track line of uniform straight-line navigation, starting from the actual position of the target ship when it was last searched and identified, with its heading as the navigation direction and the average speed as the navigation speed; S n- The three positions PA, PB, and PC on 1S' are the predicted positions of the target ship corresponding to the imaging moments of the three satellites selected with an interval of nearly 5 minutes; if the satellite imaging at the PA position fails to search and identify the target ship, and the satellite imaging at the PB position searches and identifies the target ship, the actual position of the target ship is S n Then the actual position S of the target ship when it was last searched and identified will be used as the n As the starting point, with its current heading as the navigation direction and the average speed as the navigation speed, it sails in a straight line at a uniform speed to obtain the new track line S n S”; thus repeating the above process, the target ship search, imaging and tracking can be realized when the AIS signal is lost for a short time.

7. The method for relay tracking application of large and medium-sized ships at sea according to claim 1 is characterized in that: In step S2, when the AIS signal is stably received in the periodic imaging tracking mode, if the ground gives an end-task instruction, the current task is stopped, otherwise, the current mode is maintained and continued to operate; When the AIS signal is lost for a short time and the search imaging mode is in operation, if the ground gives an instruction to end the mission, the current mission will be stopped. Otherwise, the ship target search imaging tracking situation will be judged first. If the target ship is lost, the current mission will be stopped. If the target ship can be tracked by predicting the position combined with the imaging search, the current mode will be maintained.

8. A system for relay tracking of large and medium-sized ships at sea, characterized by: The system can implement the method described in any one of claims 1 to 7 above, and the system includes a remote sensing image access unit, a satellite-borne AIS receiver, a remote sensing image intelligent processing unit and an integrated interface baseboard. The remote sensing image access unit is composed of a field programmable gate array, a random access memory and a solid state drive. On the one hand, it provides a high-speed data transmission channel for reading the original remote sensing image from the optical remote sensing imaging payload, and also provides a high-speed data transmission channel for the flow of the original remote sensing image between the remote sensing image intelligent processing unit and the satellite platform. On the other hand, it is used to store the original remote sensing image, the remote sensing image slices after intelligent processing, and the operating system, software and models required for intelligent processing. The satellite-borne AIS receiver receives the ship's AIS signal and completes data analysis; The remote sensing image intelligent processing unit is composed of a central processing unit and a graphics processing unit, and mainly provides an algorithm operation platform for remote sensing image preprocessing and intelligent detection and identification of ships in remote sensing images. It runs the Linux kernel operating system and is equipped with image preprocessing algorithms and deep learning algorithms related to intelligent detection and identification of large and medium-sized ship targets, providing powerful program control management, data flow processing, and parallel computing functions for image intelligent processing. The integrated interface baseboard consists of a power control unit and multiple types of interface channels. On the one hand, it provides power control for the remote sensing image access unit, satellite-borne AIS receiver and remote sensing image intelligent processing unit in the system, and provides interface management between the remote sensing image access unit, satellite-borne AIS receiver and remote sensing image intelligent processing unit and the satellite platform; on the other hand, it provides mechanical installation support for the remote sensing image access unit, satellite-borne AIS receiver and remote sensing image intelligent processing unit, supports modular and stacked design, and is easy to realize personalized functional combination design.

9. The system for relay tracking of large and medium-sized ships at sea according to claim 8 is characterized in that: The files stored in the solid-state hard disk include the operating system, software programs and dependent libraries required for the application program to run in the remote sensing image intelligent processing unit; and relevant parameter files for preprocessing newly acquired remote sensing images; Preprocessed remote sensing images; deep learning network structure, model parameter files and configuration files for ship target detection and recognition; The random access memory is used to provide a high-speed cache for the reading and writing operations of the original remote sensing image data, the pre-processed remote sensing image and the result data after intelligent processing.

Images