Optimization method and system for collaborative observation of maritime targets based on data fusion

Abstract

The present invention discloses a method and system for optimizing collaborative observation of maritime targets based on data fusion, which is used to improve the observation efficiency and emergency response capabilities of multiple platforms for mobile targets at sea. The method includes: tracking targets through maritime observation platforms, constructing a time-series trajectory diagram, and predicting the target trajectory within a period of time in the future to generate a trajectory prediction map; identifying observation blind spots based on the prediction results to form an observation blind spot distribution map; constructing a collaborative compensation observation network based on the blind spot distribution to achieve multi-platform collaborative observation; and obtaining fused observation data for target status judgment and emergency response. This method improves observation coverage and data integrity and is suitable for scenarios such as maritime target monitoring and emergency management.

Description

Technical Field

[0001] The present invention relates to the technical field of maritime target tracking, and in particular to a method and system for optimizing collaborative observation of maritime targets based on data fusion. Background Art

[0002] With the increasing development of marine resources, maritime transportation, and maritime security, the need for dynamic observation and management of maritime targets (such as ships, floating objects, and illegal intruders) is becoming increasingly urgent. Maritime targets often exhibit complex motion trajectories, experience significant environmental interference, and experience frequent occlusions, presenting significant challenges to traditional observation and recognition systems.

[0003] Existing marine observations mainly rely on a single platform, such as fixed marine radar stations, shore-based camera equipment, drone cruise systems or satellite remote sensing platforms. Although these devices can monitor targets in specific scenarios, in cases of complex sea conditions, limited field of view or frequent blind spots, observation blind spots or missing target information are prone to occur, resulting in insufficient stability and continuity of the overall monitoring system.

[0004] Furthermore, some current research has attempted to introduce multi-platform joint observation and target tracking technologies, but these efforts have primarily focused on multi-sensor data fusion, failing to effectively address the coordinated scheduling and resource optimization issues among multiple observation sources. In particular, significant room for improvement remains in areas such as blind spot compensation, target trajectory prediction, and emergency response. Traditional systems struggle to identify and coordinate responses to emergencies or unusual behavior, posing a significant safety risk.

[0005] Therefore, there is an urgent need for a technical solution that can integrate multi-source observation data, dynamically build target prediction models, automatically identify observation blind spots, and realize compensatory observation through a collaborative platform to enhance the continuous monitoring capability, global perception capability, and intelligent response capability of maritime targets. Summary of the Invention

[0006] In order to solve at least one of the above technical problems, the present invention proposes a method and system for optimizing collaborative observation of marine targets based on data fusion.

[0007] A first aspect of the present invention provides a method for optimizing collaborative observation of marine targets based on data fusion, comprising:

[0008] Tracking a mobile target at sea based on a maritime observation platform, constructing a time-series trajectory diagram of the mobile target at sea, and predicting the movement trajectory of the mobile target at sea in a preset time period in the future based on the time-series trajectory diagram to obtain a trajectory prediction map;

[0009] Determine the observation blind spots of the offshore observation platform for the offshore moving target based on the trajectory prediction map, and construct an observation blind spot distribution map;

[0010] Constructing a collaborative compensation observation network according to the observation blind area distribution map;

[0011] Conducting collaborative observation of mobile targets at sea based on the collaborative compensation observation network to construct fused observation data;

[0012] An emergency response is performed on mobile targets at sea based on the fused observation data.

[0013] In this solution, the offshore mobile target is tracked based on the offshore observation platform, and a time series trajectory diagram of the offshore mobile target is constructed. The movement trajectory of the offshore mobile target in the future preset time period is predicted based on the time series trajectory diagram to obtain a trajectory prediction map, which is specifically:

[0014] Acquire infrared video image data of a target sea area using an infrared camera device on a marine observation platform, extract frame image data of the infrared video image data, and model a static background area in the frame image data based on a Gaussian mixture model to generate a background model;

[0015] Comparing the current frame image data with the background model, extracting the pixel area in the current frame image data whose difference from the background model exceeds a preset threshold as the foreground target area, generating a foreground target mask, performing a morphological opening operation on the foreground target mask, and constructing binary image data;

[0016] Detecting a moving target at sea based on the binary image data, extracting the center coordinates and bounding box information of the moving target at sea, performing trajectory matching on the center coordinates and bounding box information of the moving target at sea between adjacent frames using the Hungarian algorithm, and calculating the Euclidean distance and bounding box overlap between the targets in adjacent frames;

[0017] Associating the maritime mobile targets whose distances and overlaps satisfy preset condition values into the same trajectory sequence, generating a time-series trajectory point set of the maritime mobile targets, and constructing a time-series trajectory graph of the maritime mobile targets based on the time-series trajectory point set;

[0018] A long short-term memory (LSTM) network is introduced, and the time series trajectory graph is learned and trained according to the LSTM network to construct a trajectory prediction model. The movement trajectory of the maritime mobile target in the target sea area within a preset time period in the future is predicted based on the trained trajectory prediction model to obtain a trajectory prediction map.

[0019] In this solution, the observation blind spots of the marine observation platform for the marine moving target are determined based on the trajectory prediction map, and the observation blind spot distribution map is constructed, specifically:

[0020] Obtain the geographical environment data of the target sea area, the geographical coordinates and appearance data of the offshore observation platform to construct a three-dimensional geographical model of the target sea area;

[0021] Acquire effective observation distance and observation coverage angle data of the offshore observation platform, generate a visual ray emission model of the observation platform based on the effective observation distance and observation coverage angle, map the visual ray emission model to the offshore observation platform in the three-dimensional geographic model, and construct a blind spot scanning system;

[0022] determining the apparent size of the maritime moving target based on bounding box information of the maritime moving target, constructing a simulation model of the maritime moving target based on the bounding box information and the apparent size, mapping the simulation model to the three-dimensional geographic model, and simulating navigation of the maritime moving target based on the trajectory prediction map;

[0023] The blind spot scanning system emits simulated rays at preset angular intervals toward the three-dimensional geographic model of the simulated navigation, and determines, based on the coverage area and occlusion relationship of the simulated rays in the three-dimensional geographic model, blank areas of the target sea area not covered by the rays at each time point of the simulated navigation, and marks the blank areas as preselected observation blind spots;

[0024] According to the predicted position coordinates of the maritime mobile target in the trajectory prediction map, determining whether there is an area of the preselected observation blind spot that overlaps with the predicted position of the maritime mobile target at each moment in a preset future time period; if there is an overlapping area, associating the overlapping area with the corresponding moment and marking it as an observation blind spot;

[0025] According to the time series distribution and spatial coordinates of the observation blind spots, a distribution map of the observation blind spots evolving over time in a future preset time period is constructed, wherein the distribution map of the observation blind spots includes spatial boundary information and time window information of the observation blind spots.

[0026] In this solution, the collaborative compensation observation network is constructed according to the observation blind area distribution map, specifically:

[0027] Extracting spatial boundary information and time window information of each observation blind spot according to the observation blind spot distribution map, and calculating the coverage area, time window length and predicted moving speed of the maritime moving target within the blind spot of each observation blind spot;

[0028] When the coverage area is smaller than the preset area threshold and the time window length is smaller than the preset time threshold, the corresponding observation blind spot is calibrated as a short-term local blind spot; when the coverage area exceeds the preset area threshold or the predicted moving speed of the mobile target at sea exceeds the preset speed threshold, the corresponding observation blind spot is calibrated as a long-term wide-area blind spot;

[0029] For short-term local blind spots, the real-time position coordinates and endurance of dispatchable UAVs in the target sea area are obtained. Based on the spatial boundary coordinates and time window information of the short-term local blind spot, the shortest navigation path and navigation time of the UAV from the current position to the boundary of the short-term local blind spot are calculated;

[0030] When the flight time is less than a preset ratio of the time window length and the drone's endurance meets the round-trip observation requirements, the corresponding drone is marked as an available compensation node, and a drone observation path planning instruction is generated to obtain a short-term blind spot compensation observation strategy;

[0031] For long-term wide-area blind spots, obtain the historical transit time window data and coverage area data of Beidou satellites, match the coverage area and transit time window of Beidou satellites according to the spatial boundary information and time window information of the long-term wide-area blind spot, and screen out Beidou satellite nodes whose coverage area spatially overlaps with the long-term wide-area blind spot and whose transit time window intersects with the blind spot time window;

[0032] According to the remote sensing image resolution of the screened Beidou satellite node, when the remote sensing image resolution meets the preset observation accuracy requirement, the corresponding Beidou satellite node is marked as an available compensation node, a satellite remote sensing data acquisition instruction is generated, and a long-term blind area compensation observation strategy is obtained;

[0033] According to the short-term blind spot compensation observation strategy and the long-term blind spot compensation observation strategy, a collaborative compensation observation network including UAVs and Beidou satellite nodes is constructed.

[0034] In this solution, the collaborative observation of the mobile targets at sea is carried out based on the collaborative compensation observation network to construct fused observation data, specifically:

[0035] Acquire real-time observation video data of a mobile target at sea in a target sea area based on a marine observation platform, extract real-time observation video frames, and acquire compensated observation data of an observation blind spot based on the collaborative compensation observation network, wherein the compensated observation data includes UAV compensated observation video data and satellite remote sensing image data;

[0036] Extracting local feature points of the real-time observation video frame and the compensated observation data based on a scale-invariant feature transformation algorithm, matching the local feature points, and establishing a spatial correspondence between the video frame and the compensated observation data;

[0037] Performing an affine transformation on the pixel area covering the observation blind area in the compensated observation data according to the spatial correspondence to generate a compensated image block that is spatially aligned with the real-time observation video frame;

[0038] The compensated image blocks are embedded into the corresponding blind spot positions of the real-time observation video frames using a weighted fusion algorithm to generate fused observation data of a continuous time series.

[0039] In this solution, the emergency response to the mobile target at sea based on the fused observation data is specifically as follows:

[0040] Extracting a multi-dimensional motion feature vector of a mobile target at sea based on the fused observation data, including a target heading angle change rate, a speed fluctuation amplitude, and a trajectory deviation;

[0041] Acquire historical motion feature data of mobile targets at sea during different abnormal events, build an abnormal event recognition model based on a decision tree, import the motion feature data of the different abnormal events into the abnormal event recognition model, and calculate the contribution of each motion feature to the classification of the abnormal event based on the information gain rate;

[0042] According to the classification contribution, motion features with information gain rates exceeding a preset threshold are selected as split nodes of a decision tree, and a decision tree is constructed using the Gini coefficient as a node splitting criterion to obtain a trained abnormal event recognition model;

[0043] Importing the multidimensional motion feature vector into the trained abnormal event recognition model to perform abnormal event recognition to obtain an abnormal event recognition result;

[0044] The abnormal event location information is determined according to the abnormal event identification result, and emergency warning data is constructed by combining the abnormal event identification result and the abnormal event location information and sent to nearby rescue ships for emergency response.

[0045] A second aspect of the present invention further provides a system for collaborative observation optimization of marine targets based on data fusion, the system comprising: a memory and a processor, wherein the memory includes a program for collaborative observation optimization of marine targets based on data fusion, and when the program for collaborative observation optimization of marine targets based on data fusion is executed by the processor, the following steps are implemented:

[0046] Tracking a mobile target at sea based on a maritime observation platform, constructing a time-series trajectory diagram of the mobile target at sea, and predicting the movement trajectory of the mobile target at sea in a preset time period in the future based on the time-series trajectory diagram to obtain a trajectory prediction map;

[0047] Determine the observation blind spots of the offshore observation platform for the offshore moving target based on the trajectory prediction map, and construct an observation blind spot distribution map;

[0048] Constructing a collaborative compensation observation network according to the observation blind area distribution map;

[0049] Conducting collaborative observation of mobile targets at sea based on the collaborative compensation observation network to construct fused observation data;

[0050] An emergency response is performed on mobile targets at sea based on the fused observation data.

[0051] The present invention discloses a method and system for optimizing collaborative observation of maritime targets based on data fusion, which is used to improve the observation efficiency and emergency response capabilities of multiple platforms for mobile targets at sea. The method includes: tracking targets through maritime observation platforms, constructing a time-series trajectory diagram, and predicting the target trajectory within a period of time in the future to generate a trajectory prediction map; identifying observation blind spots based on the prediction results to form an observation blind spot distribution map; constructing a collaborative compensation observation network based on the blind spot distribution to achieve multi-platform collaborative observation; and obtaining fused observation data for target status judgment and emergency response. This method improves observation coverage and data integrity and is suitable for scenarios such as maritime target monitoring and emergency management. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flow chart of a method for optimizing collaborative observation of marine targets based on data fusion according to the present invention is shown;

[0053] Figure 2 The flowchart of constructing fused observation data according to the present invention is shown;

[0054] Figure 3 A flow chart showing an emergency response to a mobile target at sea according to the present invention is shown;

[0055] Figure 4 A block diagram of a marine target collaborative observation optimization system based on data fusion according to the present invention is shown. DETAILED DESCRIPTION

[0056] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0057] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0058] Figure 1 The flowchart of the data fusion-based marine target collaborative observation optimization method of the present invention is shown.

[0059] like Figure 1 As shown, the first aspect of the present invention provides a method for optimizing collaborative observation of marine targets based on data fusion, comprising:

[0060] S102, tracking the maritime mobile target based on the maritime observation platform, constructing a time series trajectory diagram of the maritime mobile target, and predicting the movement trajectory of the maritime mobile target in a future preset time period based on the time series trajectory diagram to obtain a trajectory prediction map;

[0061] S104, determining the observation blind spots of the offshore observation platform for the offshore moving target based on the trajectory prediction map, and constructing an observation blind spot distribution map;

[0062] S106, constructing a collaborative compensation observation network according to the observation blind area distribution map;

[0063] S108, performing collaborative observation of the marine mobile target according to the collaborative compensation observation network to construct fused observation data;

[0064] S110: Perform emergency response to the mobile target at sea based on the fused observation data.

[0065] It should be noted that by constructing a time-series trajectory diagram, accurate modeling and prediction of the motion laws of mobile targets at sea can be achieved, effectively improving the tracking accuracy and prediction reliability of mobile trajectories under complex sea conditions, and solving the problem of trajectory loss caused by target maneuverability in traditional observation methods; through dynamic fusion modeling of three-dimensional geographical environment and observation platform parameters, the spatiotemporal blind spots caused by geographical obstruction or observation equipment limitations during target movement can be accurately identified, significantly enhancing the coverage capability of continuous monitoring of targets; a multi-level collaborative compensation observation network constructed based on the spatiotemporal distribution characteristics of blind spots can intelligently dispatch multi-source observation resources such as drones and satellites, realize dynamic compensation and resource optimization allocation of observation blind spots, and break through the bottleneck of limited observation capabilities of a single platform; through the spatiotemporal alignment and fusion processing of multi-source heterogeneous observation data, information loss or contradiction caused by differences in observation perspectives is eliminated, forming a complete and reliable target status perception data chain; finally, by combining multi-dimensional motion characteristics with intelligent decision-making models, abnormal target behavior can be quickly identified and a hierarchical emergency response mechanism can be triggered, greatly improving the timeliness of early warning and disposal of maritime emergencies.

[0066] According to an embodiment of the present invention, the offshore mobile target is tracked based on the offshore observation platform, a time series trajectory diagram of the offshore mobile target is constructed, and the movement trajectory of the offshore mobile target in a future preset time period is predicted based on the time series trajectory diagram to obtain a trajectory prediction map, specifically:

[0067] Acquire infrared video image data of a target sea area using an infrared camera device on a marine observation platform, extract frame image data of the infrared video image data, and model a static background area in the frame image data based on a Gaussian mixture model to generate a background model;

[0068] Comparing the current frame image data with the background model, extracting the pixel area in the current frame image data whose difference from the background model exceeds a preset threshold as the foreground target area, generating a foreground target mask, performing a morphological opening operation on the foreground target mask, and constructing binary image data;

[0069] Detecting a moving target at sea based on the binary image data, extracting the center coordinates and bounding box information of the moving target at sea, performing trajectory matching on the center coordinates and bounding box information of the moving target at sea between adjacent frames using the Hungarian algorithm, and calculating the Euclidean distance and bounding box overlap between the targets in adjacent frames;

[0070] Associating the maritime mobile targets whose distances and overlaps satisfy preset condition values into the same trajectory sequence, generating a time-series trajectory point set of the maritime mobile targets, and constructing a time-series trajectory graph of the maritime mobile targets based on the time-series trajectory point set;

[0071] A long short-term memory (LSTM) network is introduced, and the time series trajectory graph is learned and trained according to the LSTM network to construct a trajectory prediction model. The movement trajectory of the maritime mobile target in the target sea area within a preset time period in the future is predicted based on the trained trajectory prediction model to obtain a trajectory prediction map.

[0072] It should be noted that, firstly, the Gaussian mixture model is used to model the background of the infrared video frame, effectively separating the static sea environment from the dynamic target, and combining the morphological opening operation to eliminate noise interference, generate robust binary target features, and significantly improve the detection accuracy of moving targets in complex sea conditions; then, the Hungarian algorithm is used to correlate the center coordinates and bounding box information of multi-frame targets across frames, and the dual criteria of Euclidean distance and bounding box overlap are used to achieve accurate matching of multi-target trajectories, and construct a time-series trajectory graph with spatiotemporal continuity, which effectively solves the problem of continuous identity tracking in scenarios such as target occlusion and cross-trajectory. tracking problem; finally, by introducing the long short-term memory network (LSTM) to perform time series modeling on the historical trajectory data, the implicit features of the target movement, such as speed, heading angle, acceleration, and their nonlinear evolution laws are fully learned, which can accurately capture the movement pattern of the maritime target affected by environmental factors such as ocean currents and wind and waves, thereby generating a trajectory prediction map containing position distribution, movement trend and probability density in the future time window; the foreground target mask is a binary mask extracted from the dynamic target area by comparing the current frame image with the background model, in which the target area is white (1) and the background area is black (0).

[0073] According to an embodiment of the present invention, determining the observation blind spots of the marine observation platform for the marine mobile target based on the trajectory prediction map and constructing the observation blind spot distribution map are specifically as follows:

[0074] Obtain the geographical environment data of the target sea area, the geographical coordinates and appearance data of the offshore observation platform to construct a three-dimensional geographical model of the target sea area;

[0075] Acquire effective observation distance and observation coverage angle data of the offshore observation platform, generate a visual ray emission model of the observation platform based on the effective observation distance and observation coverage angle, map the visual ray emission model to the offshore observation platform in the three-dimensional geographic model, and construct a blind spot scanning system;

[0076] determining the apparent size of the maritime moving target based on bounding box information of the maritime moving target, constructing a simulation model of the maritime moving target based on the bounding box information and the apparent size, mapping the simulation model to the three-dimensional geographic model, and simulating navigation of the maritime moving target based on the trajectory prediction map;

[0077] The blind spot scanning system emits simulated rays at preset angular intervals toward the three-dimensional geographic model of the simulated navigation, and determines, based on the coverage area and occlusion relationship of the simulated rays in the three-dimensional geographic model, blank areas of the target sea area not covered by the rays at each time point of the simulated navigation, and marks the blank areas as preselected observation blind spots;

[0078] According to the predicted position coordinates of the maritime mobile target in the trajectory prediction map, determining whether there is an area of the preselected observation blind spot that overlaps with the predicted position of the maritime mobile target at each moment in a preset future time period; if there is an overlapping area, associating the overlapping area with the corresponding moment and marking it as an observation blind spot;

[0079] According to the time series distribution and spatial coordinates of the observation blind spots, a distribution map of the observation blind spots evolving over time in a future preset time period is constructed, wherein the distribution map of the observation blind spots includes spatial boundary information and time window information of the observation blind spots.

[0080] It should be noted that during maritime target observation, observation platforms are susceptible to obstructions from the geographic environment, inter-target occlusion, detection range limitations, and target dynamic trajectories, resulting in observation blind spots. This makes it difficult to predict the spatiotemporal coupling between target trajectories and blind spots in real time, leading to delays in emergency monitoring. Therefore, by simulating the navigation of a maritime moving target within a three-dimensional geographic model according to a trajectory prediction map, ray tracing technology is used to simulate the three-dimensional spatial coverage characteristics of the sensor's physical detection range based on the observation platform's effective detection range and coverage angle parameters. A visual ray emission model uses the observation platform's location as the origin and emits dense clusters of rays at preset angular intervals into the three-dimensional geographic model. By detecting collisions between the rays and terrain obstacles (such as islands and reefs) and the target simulation model, the obstruction effects and range attenuation boundaries along the ray propagation path are dynamically resolved, thereby accurately quantifying the geometric extent of the sensor's line of sight. The blind spot scanning system analyzes the blind spots of the ray clusters in three dimensions frame by frame. Combined with the spatiotemporal evolution of the simulated target trajectory, it captures continuous blank areas caused by terrain obstruction, insufficient detection range, or target movement in real time and maps them into pre-selected observation blind spots associated with the time axis. The pre-selected blind spots are further analyzed in spatiotemporal correlation with the future positions of the targets in the trajectory prediction map. By checking the overlap of spatial coordinates and matching the time windows, the possible blind spots on the actual navigation path of the target are screened out, eliminating invalid blind spot marks caused by the target not arriving or the path deviation. Through the spatial topological analysis of three-dimensional ray clusters and the spatiotemporal coupling modeling of dynamic trajectories, the limitations of the traditional two-dimensional plane projection method that ignores the correlation between terrain undulations and target motion are broken through, and high-precision dynamic calibration of blind spot boundaries is achieved. At the same time, relying on the physical simulation characteristics of ray tracing, the attenuation law of sensor detection capabilities under complex sea conditions is truly restored, effectively improving the reliability of blind spot determination.

[0081] According to an embodiment of the present invention, the collaborative compensation observation network is constructed according to the observation blind area distribution map, specifically:

[0082] Extracting spatial boundary information and time window information of each observation blind spot according to the observation blind spot distribution map, and calculating the coverage area, time window length and predicted moving speed of the maritime moving target within the blind spot of each observation blind spot;

[0083] When the coverage area is smaller than the preset area threshold and the time window length is smaller than the preset time threshold, the corresponding observation blind spot is calibrated as a short-term local blind spot; when the coverage area exceeds the preset area threshold or the predicted moving speed of the mobile target at sea exceeds the preset speed threshold, the corresponding observation blind spot is calibrated as a long-term wide-area blind spot;

[0084] For short-term local blind spots, the real-time position coordinates and endurance of dispatchable UAVs in the target sea area are obtained. Based on the spatial boundary coordinates and time window information of the short-term local blind spot, the shortest navigation path and navigation time of the UAV from the current position to the boundary of the short-term local blind spot are calculated;

[0085] When the flight time is less than a preset ratio of the time window length and the drone's endurance meets the round-trip observation requirements, the corresponding drone is marked as an available compensation node, and a drone observation path planning instruction is generated to obtain a short-term blind spot compensation observation strategy;

[0086] For long-term wide-area blind spots, obtain the historical transit time window data and coverage area data of Beidou satellites, match the coverage area and transit time window of Beidou satellites according to the spatial boundary information and time window information of the long-term wide-area blind spot, and screen out Beidou satellite nodes whose coverage area spatially overlaps with the long-term wide-area blind spot and whose transit time window intersects with the blind spot time window;

[0087] According to the remote sensing image resolution of the screened Beidou satellite node, when the remote sensing image resolution meets the preset observation accuracy requirement, the corresponding Beidou satellite node is marked as an available compensation node, a satellite remote sensing data acquisition instruction is generated, and a long-term blind area compensation observation strategy is obtained;

[0088] According to the short-term blind spot compensation observation strategy and the long-term blind spot compensation observation strategy, a collaborative compensation observation network including UAVs and Beidou satellite nodes is constructed.

[0089] It should be noted that when the coverage area is less than the preset area threshold and the time window length is less than the preset time threshold, it means that the observation blind spot has the characteristics of limited spatial range and short duration. Such blind spots are usually caused by local obstacles or short-term environmental interference. The scope of influence is controllable and the existence time is relatively short. It is suitable for the use of fast-response means such as maneuverable and flexible drones for precise compensation observation. When the coverage area exceeds the preset area threshold or the predicted moving speed of the mobile target at sea exceeds the preset speed threshold, it means that the blind spot has the characteristics of a wide spatial range or rapid expansion over time. Such blind spots are often caused by large-scale terrain obstruction, continuous severe sea conditions or high-speed moving targets beyond the conventional observation range. The scope of influence is large and the duration is long or the dynamic changes are drastic. It requires the use of platforms with wide-area continuous observation capabilities such as satellites to achieve effective coverage. For short-term, local blind spots, dynamic path planning based on the drone's real-time position and endurance is used. By verifying the matching of the shortest flight time with the blind spot time window, the drone is ensured to arrive at the designated location before the target enters the blind spot. This provides a fast-response, flexible, and maneuverable near-field compensation observation capability, effectively addressing the inability of traditional fixed observation equipment to cover sudden, small-scale blind spots. For long-term, wide-area blind spots, high-precision satellite remote sensing coverage of large, dynamic blind spots is achieved by coupling the Beidou satellite transit time window with the blind spot's spatiotemporal parameters, combined with remote sensing resolution threshold constraints. This overcomes the bottleneck in the ability of a single observation platform to provide continuous, wide-area blind spot monitoring. The two strategies complement each other in spatiotemporal dimensions to form a three-dimensional observation network. The drone strategy focuses on ensuring real-time blind spot compensation and positioning accuracy, while the satellite strategy addresses the needs of large-scale spatial coverage and long-term monitoring. By complementing these strategies in spatiotemporal dimensions to build a collaborative network, the team leverages the advantages of drones' high-precision near-field observations and the satellites' macroscopic remote sensing capabilities, significantly improving blind spot data integrity and the timeliness of emergency response.

[0090] Figure 2 A flow chart of constructing fused observation data according to the present invention is shown.

[0091] According to an embodiment of the present invention, the collaborative observation of the mobile target at sea based on the collaborative compensation observation network and the construction of fused observation data are specifically as follows:

[0092] S202, acquiring real-time observation video data of a mobile target at sea in a target sea area from a marine observation platform, extracting real-time observation video frames, and acquiring compensated observation data of an observation blind spot based on the collaborative compensation observation network, wherein the compensated observation data includes compensated observation video data from a drone and satellite remote sensing image data;

[0093] S204, extracting local feature points of the real-time observation video frame and the compensated observation data based on a scale-invariant feature transformation algorithm, matching the local feature points, and establishing a spatial correspondence between the video frame and the compensated observation data;

[0094] S206, performing an affine transformation on the pixel area covering the observation blind area in the compensated observation data according to the spatial correspondence, to generate a compensated image block that is spatially aligned with the real-time observation video frame;

[0095] S208 , using a weighted fusion algorithm to embed the compensated image block into a corresponding blind spot position of the real-time observation video frame, and generating fused observation data of a continuous time series.

[0096] It should be noted that the collaborative compensation observation network, based on drone video and satellite remote sensing data, uses a scale-invariant feature transformation algorithm to extract local feature points from multi-source data and establish precise spatial correspondences. This effectively addresses the data registration challenges caused by differences in viewing angles and resolutions across different observation platforms. Affine transformation is used to geometrically correct the compensation observation data and the main observation video frame, ensuring precise spatial alignment of the blind spot compensation image blocks with the original video, eliminating target deformation or positional misalignment caused by coordinate deviations in traditional methods. An adaptive weighted fusion algorithm is used to embed the compensation image into the blind spot. By adjusting the pixel-level fusion weights, the accuracy of the original video observation is maintained while ensuring a natural transition between the compensation area and the surrounding environment, avoiding artifacts of splicing. The fusion accuracy of the multi-platform observation data is improved to the sub-pixel level, achieving an optimal balance between temporal continuity and spatial consistency in the fused observation data. This not only fully restores the target's motion trajectory in the blind spot, but also enhances the recognition of target features through the complementarity of multi-source data. The weighted fusion algorithm includes a multi-scale fusion method based on the Gaussian-Laplacian pyramid. The fused observation data is the fused observation video data.

[0097] Figure 3 A flow chart of the present invention for emergency response to a mobile target at sea is shown.

[0098] According to an embodiment of the present invention, performing an emergency response to a mobile target at sea based on the fused observation data is specifically:

[0099] S302, extracting a multi-dimensional motion feature vector of the maritime moving target based on the fused observation data, including the target heading angle change rate, speed fluctuation amplitude, and trajectory deviation;

[0100] S304, obtaining historical motion feature data of the mobile target at sea during different abnormal events, building an abnormal event recognition model based on a decision tree, importing the motion feature data of the different abnormal events into the abnormal event recognition model, and calculating the contribution of each motion feature to the classification of the abnormal event based on the information gain rate;

[0101] S306, selecting motion features with information gain rates exceeding a preset threshold based on the classification contribution as splitting nodes of a decision tree, and constructing a decision tree using the Gini coefficient as a node splitting criterion to obtain a trained abnormal event recognition model;

[0102] S308, importing the multi-dimensional motion feature vector into the trained abnormal event recognition model to perform abnormal event recognition and obtain an abnormal event recognition result;

[0103] S310: Determine the abnormal event location information according to the abnormal event identification result, construct emergency warning data using the abnormal event identification result and the abnormal event location information, and send it to nearby rescue ships for emergency response.

[0104] It should be noted that the multi-dimensional motion feature vector extracted based on the fusion observation data comprehensively characterizes the abnormal behavior pattern of the target through dynamic parameters such as the heading angle change rate and speed fluctuation; the decision tree model is used to select the most discriminative feature nodes based on the information gain rate, and the Gini coefficient is used to optimize the decision tree splitting process, which significantly improves the accuracy of abnormal event identification; by importing real-time motion features into the trained model, it can quickly identify a variety of typical abnormal events such as collision risks and ship failures, and generate structured early warning data in combination with location information; the system automatically pushes early warning information to surrounding rescue ships, realizing full-link automated processing from abnormal detection to emergency response, greatly shortening the response time required for traditional manual analysis and judgment, and effectively reducing the false alarm rate through the feature contribution ranking mechanism, significantly improving the timeliness and accuracy of handling maritime emergencies. The different abnormal events mentioned include ship collision risks, illegal crossing of boundaries, mechanical failure drift, illegal fishing operations, and maritime search and rescue targets.

[0105] According to an embodiment of the present invention, the further embodiment includes:

[0106] Based on the historical communication delay data of each node in the collaborative compensation observation network, a communication delay time distribution model is constructed, and the delay time window of data transmission of each node is determined according to the communication delay time distribution model;

[0107] Collect historical blind spot status change data for each time window in the observation blind spot distribution map, including blind spot spatial boundaries, target motion vectors, and compensation node observation parameters, and construct a blind spot status time series training set;

[0108] Introducing a long short-term memory network to perform time series learning on the blind spot state time series training set to establish a blind spot state prediction model;

[0109] Predicting the blind spot state within a future preset time period according to the blind spot state prediction model, generating a blind spot state prediction sequence, and storing the blind spot state prediction sequence in a compensation data buffer;

[0110] Real-time monitoring of the communication delay state of the current node in the collaborative compensation observation network, and extracting blind zone state prediction data corresponding to the delay time point from the compensation data buffer according to the communication delay time window;

[0111] Based on the blind spot state prediction data, the compensated observation data is spatially affine transformed to generate a predicted compensation image block that is spatiotemporally aligned with the real-time observation video frame. The predicted compensation image block is dynamically embedded into the corresponding blind spot area of the real-time observation video frame to achieve data synchronization fusion under delay compensation.

[0112] The determining of the delay time window for data transmission of each node according to the communication delay time distribution model is specifically as follows:

[0113] The historical communication delay data series between the UAV nodes and the satellite nodes in the cooperative compensation observation network are statistically analyzed, and the kernel density estimation method is used to perform probability distribution fitting on the historical communication delay data series to generate the delay time probability density function of each node;

[0114] Calculating a delay time fluctuation interval under a preset confidence level according to the delay time probability density function, and dividing the delay time fluctuation interval into a plurality of continuous time windows;

[0115] Based on the occurrence frequency and duration of delay time in each time window, a Markov state transition matrix of node communication delay is established. The most likely subsequent time window sequence of the current delay state is decoded using the Viterbi algorithm to generate dynamic delay time window prediction results.

[0116] The blind area state prediction sequence index table of the compensation data buffer is updated according to the dynamic delay time window prediction result, and a mapping relationship between the delay time window and the prediction compensation image block is established.

[0117] It should be noted that in the scenario of collaborative observation of targets at sea, due to the instability of the communication links between drones and satellite nodes and the difference in transmission distance, the communication delay of the collaborative compensation observation network will cause the compensation observation data (such as satellite remote sensing images and drone video streams) to be unable to align with the real-time observation video frames in time and space, thereby causing data fusion asynchrony problems. For example, the high latency of satellite data transmission may cause a time lag when the compensation image block is embedded in the real-time video, making the blind spot compensation ineffective or the target tracking trajectory broken, reducing the timeliness of emergency response. To address this problem, this technology constructs a communication delay probability model and a blind spot state prediction mechanism, combines the long short-term memory network (LSTM) to perform time series modeling of the dynamic evolution of the blind spot, and uses Markov state transition and dynamic time window prediction to generate a compensation data buffer that matches the future delay period in advance. Its technical effect is: by predicting the future blind spot status and pre-generating compensation data, it can effectively offset the time difference caused by communication delay and ensure the spatiotemporal consistency of compensated observation data and real-time video frames; at the same time, by dynamically adjusting the compensation strategy, it can improve the robustness of multi-node collaborative observation, ensure the accuracy and real-time nature of emergency response instructions, and ultimately optimize the continuity of maritime target tracking and global situational awareness capabilities.

[0118] Figure 4 A block diagram of a marine target collaborative observation optimization system based on data fusion according to the present invention is shown.

[0119] The second aspect of the present invention further provides a data fusion-based marine target collaborative observation optimization system 4, which includes: a memory 41 and a processor 42. The memory includes a data fusion-based marine target collaborative observation optimization method program. When the data fusion-based marine target collaborative observation optimization method program is executed by the processor, the following steps are implemented:

[0120] Tracking a mobile target at sea based on a maritime observation platform, constructing a time-series trajectory diagram of the mobile target at sea, and predicting the movement trajectory of the mobile target at sea in a preset time period in the future based on the time-series trajectory diagram to obtain a trajectory prediction map;

[0121] Determine the observation blind spots of the offshore observation platform for the offshore moving target based on the trajectory prediction map, and construct an observation blind spot distribution map;

[0122] Constructing a collaborative compensation observation network according to the observation blind area distribution map;

[0123] Conducting collaborative observation of mobile targets at sea based on the collaborative compensation observation network to construct fused observation data;

[0124] An emergency response is performed on mobile targets at sea based on the fused observation data.

[0125] The present invention discloses a method and system for optimizing collaborative observation of maritime targets based on data fusion, which is used to improve the observation efficiency and emergency response capabilities of multiple platforms for mobile targets at sea. The method includes: tracking targets through maritime observation platforms, constructing a time-series trajectory diagram, and predicting the target trajectory within a period of time in the future to generate a trajectory prediction map; identifying observation blind spots based on the prediction results to form an observation blind spot distribution map; constructing a collaborative compensation observation network based on the blind spot distribution to achieve multi-platform collaborative observation; and obtaining fused observation data for target status judgment and emergency response. This method improves observation coverage and data integrity and is suitable for scenarios such as maritime target monitoring and emergency management.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0127] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0128] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0129] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0130] Alternatively, if the integrated units described above are implemented as software modules and sold or used as standalone products, they can also be stored on a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product, stored on a storage medium, includes instructions for enabling a computer device (such as a personal computer, server, or network device) to execute all or part of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as removable storage devices, ROM, RAM, magnetic disks, or optical disks.

[0131] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for optimizing the collaborative observation of marine targets based on data fusion, characterized in that: The following steps are involved: Tracking a mobile target at sea based on a maritime observation platform, constructing a time-series trajectory diagram of the mobile target at sea, and predicting the movement trajectory of the mobile target at sea in a preset time period in the future based on the time-series trajectory diagram to obtain a trajectory prediction map; Determine the observation blind spots of the offshore observation platform for the offshore moving target based on the trajectory prediction map, and construct an observation blind spot distribution map; The collaborative compensation observation network is constructed according to the observation blind area distribution map, specifically as follows: Extracting spatial boundary information and time window information of each observation blind spot according to the observation blind spot distribution map, and calculating the coverage area, time window length and predicted moving speed of the maritime moving target within the blind spot of each observation blind spot; When the coverage area is smaller than the preset area threshold and the time window length is smaller than the preset time threshold, the corresponding observation blind spot is calibrated as a short-term local blind spot; when the coverage area exceeds the preset area threshold or the predicted moving speed of the mobile target at sea exceeds the preset speed threshold, the corresponding observation blind spot is calibrated as a long-term wide-area blind spot; For short-term local blind spots, the real-time position coordinates and endurance of dispatchable UAVs in the target sea area are obtained. Based on the spatial boundary coordinates and time window information of the short-term local blind spot, the shortest navigation path and navigation time of the UAV from the current position to the boundary of the short-term local blind spot are calculated; When the flight time is less than a preset ratio of the time window length and the drone's endurance meets the round-trip observation requirements, the corresponding drone is marked as an available compensation node, and a drone observation path planning instruction is generated to obtain a short-term blind spot compensation observation strategy; For long-term wide-area blind spots, obtain the historical transit time window data and coverage area data of Beidou satellites, match the coverage area and transit time window of Beidou satellites according to the spatial boundary information and time window information of the long-term wide-area blind spot, and screen out Beidou satellite nodes whose coverage area spatially overlaps with the long-term wide-area blind spot and whose transit time window intersects with the blind spot time window; According to the remote sensing image resolution of the screened Beidou satellite node, when the remote sensing image resolution meets the preset observation accuracy requirement, the corresponding Beidou satellite node is marked as an available compensation node, a satellite remote sensing data acquisition instruction is generated, and a long-term blind area compensation observation strategy is obtained; According to the short-term blind spot compensation observation strategy and the long-term blind spot compensation observation strategy, a collaborative compensation observation network including UAVs and BeiDou satellite nodes is constructed; Conducting collaborative observation of mobile targets at sea based on the collaborative compensation observation network to construct fused observation data; An emergency response is performed on mobile targets at sea based on the fused observation data.

2. The method for optimizing the coordinated observation of marine targets based on data fusion according to claim 1, characterized in that: The marine observation platform is used to track the marine mobile target, construct a time series trajectory diagram of the marine mobile target, and predict the movement trajectory of the marine mobile target in a preset time period in the future according to the time series trajectory diagram to obtain a trajectory prediction map, which is specifically: Acquire infrared video image data of a target sea area using an infrared camera device on a marine observation platform, extract frame image data of the infrared video image data, and model a static background area in the frame image data based on a Gaussian mixture model to generate a background model; Comparing the current frame image data with the background model, extracting the pixel area in the current frame image data whose difference from the background model exceeds a preset threshold as the foreground target area, generating a foreground target mask, performing a morphological opening operation on the foreground target mask, and constructing binary image data; Detecting a moving target at sea based on the binary image data, extracting the center coordinates and bounding box information of the moving target at sea, performing trajectory matching on the center coordinates and bounding box information of the moving target at sea between adjacent frames using the Hungarian algorithm, and calculating the Euclidean distance and bounding box overlap between the targets in adjacent frames; Associating the maritime mobile targets whose distances and overlaps satisfy preset condition values into the same trajectory sequence, generating a time-series trajectory point set of the maritime mobile targets, and constructing a time-series trajectory graph of the maritime mobile targets based on the time-series trajectory point set; A long short-term memory (LSTM) network is introduced, and the time series trajectory graph is learned and trained according to the LSTM network to construct a trajectory prediction model. The movement trajectory of the maritime mobile target in the target sea area within a preset time period in the future is predicted based on the trained trajectory prediction model to obtain a trajectory prediction map.

3. The method for optimizing the coordinated observation of marine targets based on data fusion according to claim 1, characterized in that: The method of determining the observation blind spots of the offshore observation platform for the offshore mobile target based on the trajectory prediction map and constructing the observation blind spot distribution map is specifically as follows: Obtain the geographical environment data of the target sea area, the geographical coordinates and appearance data of the offshore observation platform to construct a three-dimensional geographical model of the target sea area; Acquire effective observation distance and observation coverage angle data of the offshore observation platform, generate a visual ray emission model of the observation platform based on the effective observation distance and observation coverage angle, map the visual ray emission model to the offshore observation platform in the three-dimensional geographic model, and construct a blind spot scanning system; determining the apparent size of the maritime moving target based on bounding box information of the maritime moving target, constructing a simulation model of the maritime moving target based on the bounding box information and the apparent size, mapping the simulation model to the three-dimensional geographic model, and simulating navigation of the maritime moving target based on the trajectory prediction map; The blind spot scanning system emits simulated rays at preset angular intervals toward the three-dimensional geographic model of the simulated navigation, and determines, based on the coverage area and occlusion relationship of the simulated rays in the three-dimensional geographic model, blank areas of the target sea area not covered by the rays at each time point of the simulated navigation, and marks the blank areas as preselected observation blind spots; According to the predicted position coordinates of the maritime mobile target in the trajectory prediction map, determining whether there is an area of the preselected observation blind spot that overlaps with the predicted position of the maritime mobile target at each moment in a preset future time period; if there is an overlapping area, associating the overlapping area with the corresponding moment and marking it as an observation blind spot; According to the time series distribution and spatial coordinates of the observation blind spots, a distribution map of the observation blind spots evolving over time in a future preset time period is constructed, wherein the distribution map of the observation blind spots includes spatial boundary information and time window information of the observation blind spots.

4. The method for optimizing the coordinated observation of marine targets based on data fusion according to claim 1, characterized in that: The collaborative observation of the mobile target at sea based on the collaborative compensation observation network and the construction of fused observation data are specifically as follows: Acquire real-time observation video data of a mobile target at sea in a target sea area based on a marine observation platform, extract real-time observation video frames, and acquire compensated observation data of an observation blind spot based on the collaborative compensation observation network, wherein the compensated observation data includes UAV compensated observation video data and satellite remote sensing image data; Extracting local feature points of the real-time observation video frame and the compensated observation data based on a scale-invariant feature transformation algorithm, matching the local feature points, and establishing a spatial correspondence between the video frame and the compensated observation data; Performing an affine transformation on the pixel area covering the observation blind area in the compensated observation data according to the spatial correspondence to generate a compensated image block that is spatially aligned with the real-time observation video frame; The compensated image blocks are embedded into the corresponding blind spot positions of the real-time observation video frames using a weighted fusion algorithm to generate fused observation data of a continuous time series.

5. The method for optimizing the coordinated observation of marine targets based on data fusion according to claim 1, characterized in that: The emergency response to the mobile target at sea based on the fused observation data is specifically: Extracting a multi-dimensional motion feature vector of a mobile target at sea based on the fused observation data, including a target heading angle change rate, a speed fluctuation amplitude, and a trajectory deviation; Acquire historical motion feature data of mobile targets at sea during different abnormal events, build an abnormal event recognition model based on a decision tree, import the motion feature data of the different abnormal events into the abnormal event recognition model, and calculate the contribution of each motion feature to the classification of the abnormal event based on the information gain rate; According to the classification contribution, motion features with information gain rates exceeding a preset threshold are selected as split nodes of a decision tree, and a decision tree is constructed using the Gini coefficient as a node splitting criterion to obtain a trained abnormal event recognition model; Importing the multidimensional motion feature vector into the trained abnormal event recognition model to perform abnormal event recognition to obtain an abnormal event recognition result; The abnormal event location information is determined according to the abnormal event identification result, and emergency warning data is constructed by combining the abnormal event identification result and the abnormal event location information and sent to nearby rescue ships for emergency response.

6. A data fusion-based marine target collaborative observation optimization system, characterized in that: The data fusion-based marine target collaborative observation optimization system includes a storage and a processor. The storage includes a data fusion-based marine target collaborative observation optimization method program. When the data fusion-based marine target collaborative observation optimization method program is executed by the processor, the following steps are implemented: Tracking a mobile target at sea based on a maritime observation platform, constructing a time-series trajectory diagram of the mobile target at sea, and predicting the movement trajectory of the mobile target at sea in a preset time period in the future based on the time-series trajectory diagram to obtain a trajectory prediction map; Determine the observation blind spots of the offshore observation platform for the offshore moving target based on the trajectory prediction map, and construct an observation blind spot distribution map; The collaborative compensation observation network is constructed according to the observation blind area distribution map, specifically as follows: Extracting spatial boundary information and time window information of each observation blind spot according to the observation blind spot distribution map, and calculating the coverage area, time window length and predicted moving speed of the maritime moving target within the blind spot of each observation blind spot; When the coverage area is smaller than the preset area threshold and the time window length is smaller than the preset time threshold, the corresponding observation blind spot is calibrated as a short-term local blind spot; when the coverage area exceeds the preset area threshold or the predicted moving speed of the mobile target at sea exceeds the preset speed threshold, the corresponding observation blind spot is calibrated as a long-term wide-area blind spot; For short-term local blind spots, the real-time position coordinates and endurance of dispatchable UAVs in the target sea area are obtained. Based on the spatial boundary coordinates and time window information of the short-term local blind spot, the shortest navigation path and navigation time of the UAV from the current position to the boundary of the short-term local blind spot are calculated; When the flight time is less than a preset ratio of the time window length and the drone's endurance meets the round-trip observation requirements, the corresponding drone is marked as an available compensation node, and a drone observation path planning instruction is generated to obtain a short-term blind spot compensation observation strategy; For long-term wide-area blind spots, obtain the historical transit time window data and coverage area data of Beidou satellites, match the coverage area and transit time window of Beidou satellites according to the spatial boundary information and time window information of the long-term wide-area blind spot, and screen out Beidou satellite nodes whose coverage area spatially overlaps with the long-term wide-area blind spot and whose transit time window intersects with the blind spot time window; According to the remote sensing image resolution of the screened Beidou satellite node, when the remote sensing image resolution meets the preset observation accuracy requirement, the corresponding Beidou satellite node is marked as an available compensation node, a satellite remote sensing data acquisition instruction is generated, and a long-term blind area compensation observation strategy is obtained; According to the short-term blind spot compensation observation strategy and the long-term blind spot compensation observation strategy, a collaborative compensation observation network including UAVs and BeiDou satellite nodes is constructed; Conducting collaborative observation of mobile targets at sea based on the collaborative compensation observation network to construct fused observation data; An emergency response is performed on mobile targets at sea based on the fused observation data.

Images