Marine target collaborative observation optimization method and system based on data fusion

Abstract

The invention discloses a sea target collaborative observation optimization method and a sea target collaborative observation optimization system based on data fusion, which are used for improving the observation efficiency and emergency response capability of multiple platforms to sea moving targets. The method comprises the following steps: tracking a target through an offshore observation platform, constructing a time sequence trajectory diagram, predicting a target trajectory in a period of time in the future, and generating a trajectory prediction map; recognizing the observation blind area according to the prediction result to form an observation blind area distribution map; a cooperative compensation observation network is constructed based on blind area distribution, and multi-platform cooperative observation is achieved; and after fusion observation data is obtained, target state judgment and emergency response are carried out. The method improves the observation coverage rate and the data integrity, and is suitable for scenes such as marine target monitoring and emergency management.

Description

Technical Field

[0001] The present invention relates to the technical field of marine target tracking, and in particular to a marine target collaborative observation optimization method and system based on data fusion. Background Art

[0002] With the increasing number of marine resource development, maritime transportation and maritime security activities, the demand for dynamic observation and management of marine targets (such as ships, floating objects and illegal intrusion targets) is becoming increasingly urgent. Marine targets usually have complex motion trajectories, large environmental interference, and frequent target occlusion, which poses a great challenge to traditional observation and recognition systems.

[0003] Existing marine observations mainly rely on a single platform, such as fixed marine radar stations, shore-based camera equipment, drone cruise systems or satellite remote sensing platforms. Although these devices can monitor targets in specific scenarios, when the sea conditions are complex, the field of view is limited or blind spots occur frequently, observation blind spots or target information is easily lost, resulting in insufficient stability and continuity of the overall monitoring system.

[0004] In addition, some current studies have attempted to introduce multi-platform joint observation and target tracking technology, but they are mostly focused on the multi-sensor data fusion level, and have failed to effectively solve the problem of coordinated scheduling and observation resource optimization among multi-source observation equipment, especially in observation blind spot compensation, target trajectory prediction and emergency response. There is still much room for improvement. In particular, in the face of emergencies or abnormal behaviors, traditional systems are difficult to identify and respond in a coordinated manner in a timely manner, which poses certain safety risks.

[0005] Therefore, there is an urgent need for a technical solution that can integrate multi-source observation data, dynamically build target prediction models, automatically identify observation blind spots, and realize compensatory observation through a collaborative platform, so as to enhance the continuous monitoring capability, global perception capability and intelligent response capability of maritime targets. Summary of the invention

[0006] In order to solve at least one of the above technical problems, the present invention proposes a method and system for optimizing collaborative observation of marine targets based on data fusion.

[0007] A first aspect of the present invention provides a method for optimizing coordinated observation of marine targets based on data fusion, comprising: Tracking the mobile targets at sea based on the marine observation platform, constructing a time series trajectory diagram of the mobile targets at sea, and predicting the movement trajectory of the mobile targets at sea in a preset time period in the future according to the time series trajectory diagram to obtain a trajectory prediction map; Determine the observation blind area of the offshore observation platform for the offshore moving target according to the trajectory prediction map, and construct an observation blind area distribution map; Construct a collaborative compensation observation network according to the described observation blind area distribution map; Conduct collaborative observation on the maritime moving target according to the described collaborative compensation observation network, and construct integrated observation data; Conduct emergency response to the maritime moving target according to the described integrated observation data.

[0008] In this solution, the maritime moving target is tracked based on a maritime observation platform, a time-series trajectory map of the maritime moving target is constructed, and the moving trajectory of the maritime moving target in a future preset time period is predicted according to the time-series trajectory map to obtain a trajectory prediction map. Specifically: Obtain infrared video image data in the target sea area based on the infrared imaging device of the maritime observation platform, extract the frame image data of the infrared video image data, and model the static background area in the frame image data based on the Gaussian mixture model to generate a background model; Compare the current frame image data with the background model, extract the pixel area in the current frame image data with a difference degree from the background model exceeding a preset threshold as the foreground target area, generate a foreground target mask, and perform morphological opening operation processing on the foreground target mask to construct binary image data; Detect the maritime moving target according to the binary image data, extract the center coordinates and bounding box information of the maritime moving target, and use the Hungarian algorithm to perform trajectory matching on the center coordinates and bounding box information of the maritime moving target between adjacent frames, and calculate the Euclidean distance and bounding box overlap degree between adjacent frame targets; Associate the maritime moving targets with distance and overlap degree meeting the preset condition values to the same trajectory sequence, generate a set of time-series trajectory points of the maritime moving target, and construct a time-series trajectory map of the maritime moving target according to the set of time-series trajectory points; Introduce a long short-term memory network, learn and train the time-series trajectory map according to the long short-term memory network, construct a trajectory prediction model, and predict the moving trajectory of the maritime moving target in the target sea area in a future preset time period according to the trained trajectory prediction model to obtain a trajectory prediction map.

[0009] In this solution, the observation blind area of the maritime observation platform for the maritime moving target is determined according to the trajectory prediction map, and an observation blind area distribution map is constructed. Specifically: Obtain the geographical environment data of the target sea area, the geographical coordinates and appearance form data of the maritime observation platform to construct a three-dimensional geographical model of the target sea area; Obtain the effective observation distance and observation coverage angle data of the maritime observation platform, generate a visual ray emission model of the observation platform based on the effective observation distance and observation coverage angle, and map the visual ray emission model to the maritime observation platform in the three-dimensional geographical model to construct a blind area scanning system; Determining the apparent size of the marine mobile target according to the bounding box information of the marine mobile target, constructing a simulation model of the marine mobile target according to the bounding box information and the apparent size, mapping the simulation model to the three-dimensional geographic model, and simulating navigation of the marine mobile target according to the trajectory prediction map; According to the blind spot scanning system, simulated rays are emitted to the three-dimensional geographic model of the simulated navigation at preset angle intervals, and according to the coverage area and occlusion relationship of the simulated rays in the three-dimensional geographic model, blank areas of the target sea area not covered by the rays at each time point of the simulated navigation are determined, and the blank areas are marked as pre-selected observation blind areas; According to the predicted position coordinates of the mobile target at sea in the trajectory prediction map, it is determined whether the preselected observation blind area has an area overlapping with the predicted position of the mobile target at sea at each moment in the future preset time period; if there is an overlapping area, the overlapping area is associated with the corresponding moment and marked as the observation blind area; According to the time series distribution and spatial coordinates of the observation blind area, a distribution map of the observation blind area that evolves over time within a future preset time period is constructed, and the observation blind area distribution map includes spatial boundary information and time window information of the observation blind area.

[0010] In this solution, the collaborative compensation observation network is constructed according to the observation blind area distribution map, specifically: Extract spatial boundary information and time window information of each observation blind area according to the observation blind area distribution map, calculate the coverage area, time window length and predicted moving speed of the marine mobile target in the blind area of each observation blind area; When the coverage area is smaller than the preset area threshold and the time window length is smaller than the preset time threshold, the corresponding observation blind area is calibrated as a short-term local blind area; when the coverage area exceeds the preset area threshold or the predicted moving speed of the mobile target at sea exceeds the preset speed threshold, the corresponding observation blind area is calibrated as a long-term wide-area blind area; For short-term local blind spots, the real-time position coordinates and endurance of dispatchable UAVs in the target sea area are obtained, and the shortest navigation path and navigation time of the UAV from the current position to the boundary of the short-term local blind spot are calculated according to the spatial boundary coordinates and time window information of the short-term local blind spot. When the navigation time is less than a preset ratio of the time window length and the drone's endurance meets the round-trip observation requirements, the corresponding drone is marked as an available compensation node, and a drone observation path planning instruction is generated to obtain a short-term blind spot compensation observation strategy; For long - term wide - area blind spots, obtain the historical transit time window data and coverage area data of Beidou satellites. According to the spatial boundary information and time window information of the long - term wide - area blind spots, match the coverage area of Beidou satellites with the transit time window, and screen out Beidou satellite nodes whose coverage area has a spatial overlap with the long - term wide - area blind spot and whose transit time window has an intersection with the blind - spot time window; According to the remote - sensing image resolution of the screened - out Beidou satellite nodes, when the remote - sensing image resolution meets the preset observation accuracy requirements, mark the corresponding Beidou satellite nodes as available compensation nodes, generate a satellite remote - sensing data acquisition instruction, and obtain a long - term blind - spot compensation observation strategy; According to the short - term blind - spot compensation observation strategy and the long - term blind - spot compensation observation strategy, construct a collaborative compensation observation network including unmanned aerial vehicles and Beidou satellite nodes.

[0011] In this solution, the collaborative observation of marine mobile targets according to the collaborative compensation observation network and the construction of fused observation data are specifically as follows: Obtain the real - time observation video data of marine mobile targets in the target sea area from a marine observation platform, extract real - time observation video frames, and obtain compensation observation data for the observation blind spot based on the collaborative compensation observation network. The compensation observation data includes unmanned - aerial - vehicle compensation observation video data and satellite remote - sensing image data; Extract local feature points of the real - time observation video frames and the compensation observation data based on the Scale - Invariant Feature Transform (SIFT) algorithm, match the local feature points, and establish a spatial correspondence relationship between the video frames and the compensation observation data; Perform an affine transformation on the pixel area covering the observation blind spot in the compensation observation data according to the spatial correspondence relationship to generate a compensation image block that is spatially aligned with the real - time observation video frame; Use a weighted fusion algorithm to embed the compensation image block into the corresponding blind - spot position of the real - time observation video frame to generate fused observation data with a continuous time series.

[0012] In this solution, the emergency response to marine mobile targets according to the fused observation data is specifically as follows: Extract multi - dimensional motion feature vectors of marine mobile targets from the fused observation data, including the target heading - angle change rate, speed fluctuation amplitude, and trajectory deviation degree; Obtain the historical motion feature data of marine mobile targets in different abnormal events, construct an abnormal - event recognition model based on a decision tree, import the motion feature data of different abnormal events into the abnormal - event recognition model, and calculate the classification contribution degree of each motion feature to the abnormal events based on the information - gain ratio; Select the motion features with an information gain rate exceeding a preset threshold as the decision tree splitting nodes according to the classification contribution degree, and use the Gini coefficient as the node splitting criterion to construct a decision tree, obtaining a trained abnormal event recognition model; Import the multi-dimensional motion feature vector into the trained abnormal event recognition model for abnormal event recognition, obtaining an abnormal event recognition result; Determine the abnormal event location information according to the abnormal event recognition result, and construct emergency warning data with the abnormal event recognition result and the abnormal event location information and send it to nearby rescue ships for emergency response.

[0013] The second aspect of the present invention also provides an optimization system for collaborative observation of maritime targets based on data fusion. The system includes: a memory and a processor. The memory includes an optimization method program for collaborative observation of maritime targets based on data fusion. When the optimization method program for collaborative observation of maritime targets based on data fusion is executed by the processor, the following steps are implemented: Track the maritime mobile target based on a maritime observation platform, construct a time-series trajectory map of the maritime mobile target, and predict the movement trajectory of the maritime mobile target in a future preset time period according to the time-series trajectory map, obtaining a trajectory prediction map; Determine the observation blind area of the maritime observation platform for the maritime mobile target according to the trajectory prediction map, and construct an observation blind area distribution map; Construct a collaborative compensation observation network according to the observation blind area distribution map; Conduct collaborative observation on the maritime mobile target according to the collaborative compensation observation network, and construct fused observation data; Conduct emergency response on the maritime mobile target according to the fused observation data.

[0014] The present invention discloses an optimization method and system for collaborative observation of maritime targets based on data fusion, which is used to improve the observation efficiency and emergency response ability of multiple platforms for maritime mobile targets. The method includes: tracking a target through a maritime observation platform, constructing a time-series trajectory map, and predicting the target trajectory in a future period of time to generate a trajectory prediction map; identifying the observation blind area according to the prediction result to form an observation blind area distribution map; constructing a collaborative compensation observation network based on the blind area distribution to achieve multi-platform collaborative observation; after obtaining the fused observation data, it is used for target state judgment and emergency response. This method improves the observation coverage rate and data integrity, and is applicable to scenarios such as maritime target monitoring and emergency management. Description of the Drawings

[0015] Figure 1 Shows a flowchart of an optimization method for collaborative observation of maritime targets based on data fusion according to the present invention; Figure 2Shows the flowchart of constructing fused observation data in the present invention; Figure 3 Shows the flowchart of emergency response to maritime moving targets in the present invention; Figure 4 Shows the block diagram of an optimized system for collaborative observation of maritime targets based on data fusion in the present invention. Detailed implementation manners

[0016] In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments may be combined with each other.

[0017] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0018] Figure 1 Shows the flowchart of an optimized method for collaborative observation of maritime targets based on data fusion in the present invention.

[0019] As Figure 1 shown, the first aspect of the present invention provides an optimized method for collaborative observation of maritime targets based on data fusion, including: S102, tracking maritime moving targets based on a maritime observation platform, constructing a time-series trajectory map of the maritime moving targets, and predicting the moving trajectory of the maritime moving targets in a future preset time period according to the time-series trajectory map to obtain a trajectory prediction map; S104, determining the observation blind area of the maritime observation platform for the maritime moving targets according to the trajectory prediction map, and constructing an observation blind area distribution map; S106, constructing a collaborative compensation observation network according to the observation blind area distribution map; S108, performing collaborative observation on the maritime moving targets according to the collaborative compensation observation network, and constructing fused observation data; S110, performing emergency response on the maritime moving targets according to the fused observation data.

[0020] It should be noted that by constructing a time-series trajectory map, the accurate modeling and prediction of the motion law of maritime moving targets are realized, effectively improving the tracking accuracy and prediction reliability of moving trajectories under complex sea conditions, and solving the problem of trajectory loss caused by target mobility in traditional observation methods; through the dynamic fusion modeling of three-dimensional geographical environment and observation platform parameters, the spatio-temporal blind areas generated by geographical occlusion or observation equipment limitations during the movement of the target are accurately identified, significantly enhancing the coverage ability of continuous monitoring of the target; the multi-level collaborative compensation observation network constructed based on the spatio-temporal distribution characteristics of blind areas can intelligently schedule multi-source observation resources such as unmanned aerial vehicles and satellites, realizing the dynamic compensation of observation blind areas and the optimal allocation of resources, breaking through the bottleneck of limited observation capabilities of a single platform; through the spatio-temporal alignment and fusion processing of multi-source heterogeneous observation data, the information loss or contradiction caused by different observation perspectives is eliminated, forming a complete and reliable target state perception data chain; finally, combined with multi-dimensional motion characteristics and intelligent decision-making models, abnormal behaviors of the target are quickly identified and a hierarchical emergency response mechanism is triggered, greatly improving the timeliness of early warning and disposal of maritime emergencies.

[0021] According to an embodiment of the present invention, the tracking of a maritime moving target is performed based on a maritime observation platform, a time-series trajectory map of the maritime moving target is constructed, and the moving trajectory of the maritime moving target in a future preset time period is predicted according to the time-series trajectory map to obtain a trajectory prediction map. Specifically: Based on the infrared imaging device of the maritime observation platform, infrared video image data in the target sea area is obtained, the frame image data of the infrared video image data is extracted, and the static background area in the frame image data is modeled based on the Gaussian mixture model to generate a background model. The current frame image data is compared with the background model, and the pixel area in the current frame image data with a difference degree from the background model exceeding a preset threshold is extracted as the foreground target area, a foreground target mask is generated, and morphological opening operation processing is performed on the foreground target mask to construct binary image data. According to the binary image data, the maritime moving target is detected, the center coordinates and bounding box information of the maritime moving target are extracted, and the Hungarian algorithm is used to perform trajectory matching on the center coordinates and bounding box information of the maritime moving target between adjacent frames, and the Euclidean distance and bounding box overlap degree between adjacent frame targets are calculated. The maritime moving targets whose distance and overlap degree meet the preset condition values are associated with the same trajectory sequence, a set of time-series trajectory points of the maritime moving target is generated, and a time-series trajectory map of the maritime moving target is constructed according to the set of time-series trajectory points. The long short-term memory network is introduced, the time-series trajectory map is learned and trained according to the long short-term memory network, a trajectory prediction model is constructed, and the moving trajectory of the maritime moving target in a future preset time period in the target sea area is predicted according to the trained complete trajectory prediction model to obtain a trajectory prediction map.

[0022] It should be noted that, firstly, the Gaussian mixture model is used to model the background of the infrared video frame, effectively separate the static sea environment and the dynamic target, and the morphological opening operation is combined to eliminate the noise interference, generate robust binary target features, and significantly improve the detection accuracy of moving targets in complex sea conditions; then, the central coordinates and bounding box information of multi-frame targets are cross-frame associated through the Hungarian algorithm, and the precise matching of multi-target trajectories is achieved based on the dual criteria of Euclidean distance and bounding box overlap, and a time-series trajectory diagram with spatiotemporal continuity is constructed, which effectively solves the problem of continuous identity tracking in scenarios such as target occlusion and cross-trajectory. tracking problem; finally, by introducing the long short-term memory network (LSTM) to perform time series modeling on the historical trajectory data, the implicit characteristics of the target movement, such as speed, heading angle, acceleration, and their nonlinear evolution laws are fully learned, which can accurately capture the movement pattern of marine targets affected by environmental factors such as ocean currents and wind and waves, thereby generating a trajectory prediction map containing position distribution, movement trend and probability density in the future time window; the foreground target mask is a binary mask extracted from the dynamic target area by comparing the current frame image with the background model, in which the target area is white (1) and the background area is black (0).

[0023] According to an embodiment of the present invention, the observation blind area of the offshore observation platform for the offshore mobile target is determined according to the trajectory prediction map, and the observation blind area distribution map is constructed, specifically: Obtain the geographical environment data of the target sea area, the geographical coordinates and appearance data of the offshore observation platform to construct a three-dimensional geographical model of the target sea area; Acquire the effective observation distance and observation coverage angle data of the offshore observation platform, generate a visualized ray emission model of the observation platform based on the effective observation distance and observation coverage angle, map the visualized ray emission model to the offshore observation platform of the three-dimensional geographic model, and construct a blind spot scanning system; Determining the apparent size of the marine mobile target according to the bounding box information of the marine mobile target, constructing a simulation model of the marine mobile target according to the bounding box information and the apparent size, mapping the simulation model to the three-dimensional geographic model, and simulating navigation of the marine mobile target according to the trajectory prediction map; According to the blind spot scanning system, simulated rays are emitted to the three-dimensional geographic model of the simulated navigation at preset angle intervals, and according to the coverage area and occlusion relationship of the simulated rays in the three-dimensional geographic model, blank areas of the target sea area not covered by the rays at each time point of the simulated navigation are determined, and the blank areas are marked as pre-selected observation blind areas; Based on the predicted position coordinates of the maritime moving target in the trajectory prediction map, determine whether there is an area overlapping with the predicted position of the maritime moving target at each moment in the future preset time period in the preselected observation blind area. If there is an overlapping area, associate the overlapping area with the corresponding moment and label it as an observation blind area. Based on the time series distribution and spatial coordinates of the observation blind area, construct an observation blind area distribution map evolving with time within the future preset time period. The observation blind area distribution map includes the spatial boundary information and time window information of the observation blind area.

[0024] It should be noted that in the observation of maritime targets, due to the occlusion of the geographical environment by the observation platform, the occlusion between moving targets, the limitation of the detection distance, and the influence of the dynamic trajectory of the target, it is easy to generate observation blind areas, and it is difficult to predict the spatio-temporal coupling relationship between the target movement trajectory and the blind area in real time, resulting in a lag in emergency monitoring. Therefore, by simulating the navigation of maritime moving targets in a three-dimensional geographical model according to the trajectory prediction map, based on the effective detection distance and coverage angle parameters of the observation platform, the three-dimensional space coverage characteristics of the physical detection range of the sensor are simulated using ray tracing technology. The visual ray emission model takes the position of the observation platform as the origin and emits a dense ray cluster into the three-dimensional geographical model at a preset angular interval. Through the collision detection of the rays with terrain obstacles (such as islands and reefs) and the target simulation model, the occlusion effect and distance attenuation boundary in the ray propagation path are dynamically solved, so as to accurately quantify the geometric range of the area reachable by the sensor's line of sight. The blind area scanning system analyzes the coverage blind spots of the ray cluster in three-dimensional space frame by frame, combines the spatio-temporal evolution of the target simulation navigation trajectory, and captures in real time the continuous blank areas caused by terrain occlusion, insufficient detection distance, or target movement, and maps them to the preselected observation blind area associated with the time axis. Further, a spatio-temporal correlation analysis is performed between the preselected blind area and the future position of the target in the trajectory prediction map. Through the spatial coordinate overlap verification and time window matching, the blind area areas that may exist on the actual navigation path of the target are screened out, and the invalid blind area marks caused by the target not arriving or the path deviation are eliminated; through the spatial topology analysis of the three-dimensional ray cluster and the spatio-temporal coupling modeling of the dynamic trajectory, the limitation of the traditional two-dimensional plane projection method ignoring the terrain undulation and the target movement correlation is broken through, the high-precision dynamic calibration of the blind area boundary is realized, and at the same time, relying on the physical simulation characteristics of ray tracing, the attenuation law of the sensor detection ability under complex sea conditions is truly restored, effectively improving the reliability of blind area determination.

[0025] According to an embodiment of the present invention, the construction of the collaborative compensation observation network based on the observation blind area distribution map is specifically as follows: Extract the spatial boundary information and time window information of each observation blind area from the observation blind area distribution map, and calculate the coverage area, time window length of each observation blind area, and the predicted moving speed of the maritime moving target in the blind area. When the coverage area is less than the preset area threshold and the time window length is less than the preset time threshold, the corresponding observation blind area is calibrated as a short-term local blind area. When the coverage area exceeds the preset area threshold or the predicted moving speed of the marine mobile target exceeds the preset speed threshold, the corresponding observation blind area is calibrated as a long-term wide-area blind area; For the short-term local blind area, obtain the real-time position coordinates and endurance of the schedulable unmanned aerial vehicles (UAVs) in the target sea area. According to the spatial boundary coordinates and time window information of the short-term local blind area, calculate the shortest navigation path and navigation time for the UAVs to reach the boundary of the short-term local blind area from the current position; When the navigation time is less than a preset proportion of the time window length and the endurance of the UAVs meets the round-trip observation requirements, mark the corresponding UAVs as available compensation nodes, generate UAV observation path planning instructions, and obtain the short-term blind area compensation observation strategy; For the long-term wide-area blind area, obtain the historical transit time window data and coverage area data of the Beidou satellites. According to the spatial boundary information and time window information of the long-term wide-area blind area, match the coverage area of the Beidou satellites with the transit time window, and filter out the Beidou satellite nodes whose coverage area has a spatial overlap with the long-term wide-area blind area and whose transit time window has an intersection with the blind area time window; According to the remote sensing image resolution of the filtered Beidou satellite nodes, when the remote sensing image resolution meets the preset observation accuracy requirements, mark the corresponding Beidou satellite nodes as available compensation nodes, generate satellite remote sensing data acquisition instructions, and obtain the long-term blind area compensation observation strategy; According to the short-term blind area compensation observation strategy and the long-term blind area compensation observation strategy, construct a collaborative compensation observation network including UAVs and Beidou satellite nodes.

[0026] It should be noted that when the coverage area is less than the preset area threshold and the time window length is less than the preset time threshold, it indicates that the observation blind area has the characteristics of limited spatial range and short duration. Such blind areas are usually caused by local obstacles or short-term environmental interference, and their influence range is controllable and the existence time is short. It is suitable to use flexible means such as unmanned aerial vehicles for precise compensation observation. When the coverage area exceeds the preset area threshold or the predicted moving speed of the maritime moving target exceeds the preset speed threshold, it indicates that the blind area has the characteristics of broad spatial range or rapid expansion over time. Such blind areas are often caused by large-scale terrain occlusion, continuous bad sea conditions, or high-speed moving targets exceeding the conventional observation range, and their influence range is large, the duration is long or the dynamic changes are severe. It is necessary to rely on platforms with wide-area continuous observation capabilities such as satellites to achieve effective coverage. For short-term local blind areas, based on the dynamic path planning of the unmanned aerial vehicle's real-time position and endurance, through the matching verification of the shortest navigation time and the blind area time window, it is ensured that the unmanned aerial vehicle arrives at the designated position before the target enters the blind area, forming a fast-response and flexible near-field compensation observation ability, effectively solving the problem that traditional fixed observation equipment cannot cover sudden small-scale blind areas. For long-term wide-area blind areas, through the coupling and screening of the Beidou satellite transit time window and the spatio-temporal parameters of the blind area, combined with the remote sensing resolution threshold constraint, high-precision satellite remote sensing coverage of large-scale dynamic blind areas is achieved, breaking through the ability bottleneck of a single observation platform in the continuous monitoring of wide-area blind areas. The two strategies form a three-dimensional observation network through spatio-temporal dimension complementarity. Among them, the unmanned aerial vehicle strategy focuses on ensuring the real-time performance and positioning accuracy of blind area compensation, and the satellite strategy focuses on solving the large-scale spatial coverage and long-term monitoring requirements. By constructing a collaborative network through the strategy complementarity in the spatio-temporal dimension, not only the advantages of high-precision near-field observation of the unmanned aerial vehicle are exerted, but also the macro remote sensing ability of the satellite is utilized, significantly improving the data integrity of the blind area and the timeliness of emergency response.

[0027] Figure 2 Fig. shows a flowchart of constructing fused observation data according to the present invention.

[0028] According to an embodiment of the present invention, the collaborative observation of the maritime moving target according to the collaborative compensation observation network and the construction of fused observation data are specifically as follows: S202, obtain real-time observation video data of the maritime moving target in the target sea area according to the maritime observation platform, extract real-time observation video frames, and obtain compensation observation data of the observation blind area based on the collaborative compensation observation network, where the compensation observation data includes unmanned aerial vehicle compensation observation video data and satellite remote sensing image data; S204, extract local feature points of the real-time observation video frames and the compensation observation data based on the scale-invariant feature transform algorithm, match the local feature points, and establish a spatial correspondence relationship between the video frames and the compensation observation data; S206. Perform an affine transformation on the pixel regions in the compensated observation data that cover the observation blind area according to the spatial correspondence relationship to generate a compensated image block that is spatially aligned with the real-time observation video frame. S208. Embed the compensated image block into the corresponding blind area position of the real-time observation video frame by using a weighted fusion algorithm to generate fused observation data with a continuous time series.

[0029] It should be noted that, based on the UAV video and satellite remote sensing data obtained from the collaborative compensation observation network, local feature points of multi-source data are extracted through the scale-invariant feature transform algorithm and an accurate spatial correspondence relationship is established, effectively solving the data registration problem caused by different viewing angles and resolutions of different observation platforms; the compensated observation data and the subjective observation video frame are geometrically corrected by using an affine transformation to ensure the accurate alignment of the blind area compensated image block and the original video in spatial coordinates, eliminating the target deformation or position misalignment problems caused by coordinate deviation in traditional methods; the compensated image is embedded into the blind area position by using an adaptive weighted fusion algorithm, and through pixel-level fusion weight adjustment, both the observation continuity of the original video is maintained and the natural transition between the compensated area and the surrounding environment is ensured, avoiding artificial splicing traces. The fusion accuracy of multi-platform observation data is improved to the sub-pixel level, enabling the fused observation data to achieve an optimal balance in terms of time continuity and spatial consistency. It not only completely restores the movement trajectory of the target in the blind area but also enhances the recognition of target features through multi-source data complementarity. The weighted fusion algorithm includes a multi-scale fusion method based on the Gaussian-Laplacian pyramid. The fused observation data is fused observation video data.

[0030] Figure 3 The flowchart showing the emergency response of the present invention to a maritime moving target is presented.

[0031] According to an embodiment of the present invention, the emergency response to the maritime moving target based on the fused observation data is specifically as follows: S302. Extract a multi-dimensional motion feature vector of the maritime moving target from the fused observation data, including the target course angle change rate, speed fluctuation amplitude, and trajectory deviation degree. S304. Obtain the historical motion feature data of the maritime moving target in different abnormal events, construct an abnormal event recognition model based on a decision tree, import the motion feature data of the different abnormal events into the abnormal event recognition model, and calculate the classification contribution degree of each motion feature to the abnormal event based on the information gain rate. S306. Select the motion features with an information gain rate exceeding a preset threshold as decision tree splitting nodes according to the classification contribution degree, use the Gini coefficient as the node splitting criterion to construct a decision tree, and obtain a trained abnormal event recognition model. S308. Import the multi-dimensional motion feature vector into the trained abnormal event recognition model to recognize abnormal events and obtain the abnormal event recognition result; S310. Determine the abnormal event location information according to the abnormal event recognition result, and construct emergency warning data from the abnormal event recognition result and the abnormal event location information and send it to nearby rescue ships for emergency response.

[0032] It should be noted that the multi-dimensional motion feature vector extracted based on the fused observation data comprehensively depicts the abnormal behavior patterns of the target through dynamic parameters such as the rate of change of the course angle and speed fluctuations; the decision tree model is used to screen the most discriminative feature nodes based on the information gain rate, and the Gini coefficient is used to optimize the decision tree splitting process, significantly improving the accuracy of abnormal event recognition; by importing the real-time motion features into the trained model, various typical abnormal events such as collision risks and ship failures can be quickly recognized, and structured warning data can be generated in combination with the location information; the system automatically pushes the warning information to surrounding rescue ships, realizing the full-link automation processing from abnormal detection to emergency response, greatly shortening the response time required for traditional manual judgment, and at the same time effectively reducing the false alarm rate through the feature contribution degree ranking mechanism, significantly improving the timeliness and accuracy of maritime emergency handling. The different abnormal events include ship collision risks, illegal border-crossing behaviors, mechanical failure drifts, illegal fishing operations, and maritime search and rescue targets, etc.

[0033] According to an embodiment of the present invention, it further includes: Based on the historical communication delay data of each node in the cooperative compensation observation network, construct a communication delay time distribution model, and determine the delay time window for data transmission of each node according to the communication delay time distribution model; Collect the historical blind area state change data of each time window in the blind area distribution map, including the blind area spatial boundary, target motion vector, and compensation node observation parameters, and construct a blind area state time series training set; Introduce a long short-term memory network to perform time series learning on the blind area state time series training set and establish a blind area state prediction model; Predict the blind area state within a preset future time period according to the blind area state prediction model, generate a blind area state prediction sequence, and store the blind area state prediction sequence in the compensation data buffer; Real-time monitor the communication delay state of the current node in the cooperative compensation observation network, and extract the blind area state prediction data corresponding to the delay time point from the compensation data buffer according to the communication delay time window; Perform a spatial affine transformation on the compensated observation data based on the blind area state prediction data to generate a predicted compensation image block that is spatio-temporally aligned with the real-time observation video frame, and dynamically embed the predicted compensation image block into the corresponding blind area of the real-time observation video frame to achieve data synchronization fusion under latency compensation.

[0034] Determine the latency time window for data transmission of each node according to the communication latency time distribution model, specifically: Statistically analyze the historical communication latency data sequence of the UAV nodes and satellite nodes in the collaborative compensation observation network, and use the kernel density estimation method to fit the probability distribution of the historical communication latency data sequence to generate the latency time probability density function of each node; Calculate the latency time fluctuation interval under a preset confidence level according to the latency time probability density function, and divide the latency time fluctuation interval into multiple consecutive time windows; Based on the occurrence frequency and duration of the latency time within each time window, establish a Markov state transition matrix for node communication latency, and decode the most likely subsequent time window sequence of the current latency state through the Viterbi algorithm to generate a dynamic latency time window prediction result; Update the blind area state prediction sequence index table of the compensation data buffer according to the dynamic latency time window prediction result, and establish a mapping relationship between the latency time window and the predicted compensation image block.

[0035] It should be noted that in the scenario of collaborative observation of maritime targets, due to the instability of the communication links between UAVs and satellite nodes and the differences in transmission distances, the communication latency of the collaborative compensation observation network will cause the compensated observation data (such as satellite remote sensing images, UAV video streams) to be spatio-temporally misaligned with the real-time observation video frame, thereby causing data fusion out-of-sync problems. For example, the high latency of satellite data transmission may cause a time lag when the compensated image block is embedded in the real-time video, resulting in the failure of blind area compensation or the breakage of the target tracking trajectory, reducing the timeliness of emergency response. To address this issue, this technology constructs a communication latency probability model and a blind area state prediction mechanism, combines the long short-term memory network (LSTM) to perform temporal modeling on the dynamic evolution law of the blind area, and uses Markov state transition and dynamic time window prediction to generate a compensation data buffer that matches future latency periods in advance. The technical effects are as follows: By predicting the future blind area state and pre-generating compensation data, the time difference caused by communication latency is effectively offset, ensuring the spatio-temporal consistency between the compensated observation data and the real-time video frame; at the same time, by dynamically adjusting the compensation strategy, the robustness of multi-node collaborative observation is improved, ensuring the accuracy and timeliness of emergency response instructions, and ultimately optimizing the continuity of maritime target tracking and the global situation awareness ability.

[0036] Figure 4The block diagram of an optimization system for collaborative observation of maritime targets based on data fusion according to the present invention is shown.

[0037] In a second aspect of the present invention, an optimization system 4 for collaborative observation of maritime targets based on data fusion is further provided. The system includes: a memory 41 and a processor 42. The memory includes a program for the optimization method of collaborative observation of maritime targets based on data fusion. When the program for the optimization method of collaborative observation of maritime targets based on data fusion is executed by the processor, the following steps are implemented: Tracking a maritime moving target based on a maritime observation platform, constructing a time-series trajectory graph of the maritime moving target, and predicting the moving trajectory of the maritime moving target in a preset future time period according to the time-series trajectory graph to obtain a trajectory prediction graph; Determining the observation blind area of the maritime observation platform for the maritime moving target according to the trajectory prediction graph, and constructing an observation blind area distribution graph; Constructing a collaborative compensation observation network according to the observation blind area distribution graph; Performing collaborative observation on the maritime moving target according to the collaborative compensation observation network, and constructing fused observation data; Performing an emergency response on the maritime moving target according to the fused observation data.

[0038] The present invention discloses an optimization method and system for collaborative observation of maritime targets based on data fusion, which is used to improve the observation efficiency and emergency response ability of multiple platforms for maritime moving targets. The method includes: tracking a target through a maritime observation platform, constructing a time-series trajectory graph, predicting the target trajectory in a future period of time, and generating a trajectory prediction graph; identifying the observation blind area according to the prediction result to form an observation blind area distribution graph; constructing a collaborative compensation observation network based on the blind area distribution to achieve multi-platform collaborative observation; after obtaining the fused observation data, it is used for target state judgment and emergency response. This method improves the observation coverage rate and data integrity, and is applicable to scenarios such as maritime target monitoring and emergency management.

[0039] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed with each other can be through some interfaces. The indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0040] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0041] In addition, each functional unit in the embodiments of the present invention may be fully integrated into one processing unit, or each unit may be separately taken as one unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0042] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including the above method embodiments; and the foregoing storage medium includes: removable storage devices, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks, or optical discs and other various media that can store program codes.

[0043] Alternatively, if the above-mentioned integrated units of the present invention are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. And the foregoing storage medium includes: removable storage devices, ROM, RAM, magnetic disks, or optical discs and other various media that can store program codes.

[0044] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. An optimization method for collaborative observation of maritime targets based on data fusion, characterized in that Including the following steps: Tracking the maritime moving target based on the maritime observation platform, constructing a time-series trajectory map of the maritime moving target, predicting the moving trajectory of the maritime moving target in a future preset time period according to the time-series trajectory map, and obtaining a trajectory prediction map; Determining the observation blind area of the maritime observation platform for the maritime moving target according to the trajectory prediction map, and constructing an observation blind area distribution map; Constructing a collaborative compensation observation network according to the observation blind area distribution map; Conducting collaborative observation on the maritime moving target according to the collaborative compensation observation network, and constructing integrated observation data; Conducting emergency response on the maritime moving target according to the integrated observation data.

2. The optimization method for collaborative observation of maritime targets based on data fusion according to claim 1, wherein, The tracking of the maritime moving target based on the maritime observation platform, constructing a time-series trajectory map of the maritime moving target, predicting the moving trajectory of the maritime moving target in a future preset time period according to the time-series trajectory map, and obtaining a trajectory prediction map are specifically as follows: Obtaining infrared video image data in the target sea area based on the infrared imaging device of the maritime observation platform, extracting the frame image data of the infrared video image data, and modeling the static background area in the frame image data based on the Gaussian mixture model to generate a background model; Comparing the current frame image data with the background model, extracting the pixel area in the current frame image data with a difference degree from the background model exceeding a preset threshold as the foreground target area, generating a foreground target mask, and performing morphological opening operation processing on the foreground target mask to construct binary image data; Detecting the maritime moving target according to the binary image data, extracting the center coordinates and bounding box information of the maritime moving target, using the Hungarian algorithm to perform trajectory matching on the center coordinates and bounding box information of the maritime moving target between adjacent frames, and calculating the Euclidean distance and bounding box overlap degree between adjacent frame targets; Associating the maritime moving targets with distance and overlap degree meeting the preset condition values to the same trajectory sequence, generating a time-series trajectory point set of the maritime moving target, and constructing a time-series trajectory map of the maritime moving target according to the time-series trajectory point set; Introducing a long short-term memory network, learning and training the time-series trajectory map according to the long short-term memory network, constructing a trajectory prediction model, and predicting the moving trajectory of the maritime moving target in a future preset time period in the target sea area according to the trained complete trajectory prediction model to obtain a trajectory prediction map.

3. A method for optimizing collaborative observation of maritime targets based on data fusion according to claim 1, characterized in that, The determining the observation blind area of the maritime observation platform for the maritime moving target according to the trajectory prediction map, and constructing an observation blind area distribution map are specifically as follows: Obtaining the geographical environment data of the target sea area, the geographical coordinates and appearance morphology data of the maritime observation platform to construct a three-dimensional geographical model of the target sea area; Obtaining the effective observation distance and observation coverage angle data of the maritime observation platform, generating a visual ray emission model of the observation platform based on the effective observation distance and observation coverage angle, and mapping the visual ray emission model to the maritime observation platform in the three-dimensional geographical model to construct a blind area scanning system; Determine the appearance size of the maritime moving target according to the bounding box information of the maritime moving target, construct a simulation model of the maritime moving target based on the bounding box information and the appearance size, map the simulation model into the three-dimensional geographic model, and perform simulated navigation on the maritime moving target according to the trajectory prediction atlas; According to the blind area scanning system, emit simulated rays to the three-dimensional geographic model of the simulated navigation at a preset angular interval, and determine the blank area not covered by the rays in the target sea area at each time point of the simulated navigation according to the coverage area and occlusion relationship of the simulated rays in the three-dimensional geographic model, and label the blank area as the preselected observation blind area; According to the predicted position coordinates of the maritime moving target in the trajectory prediction atlas, judge whether there is an overlapping area between the preselected observation blind area and the predicted position of the maritime moving target at each moment in the future preset time period. If there is an overlapping area, associate the overlapping area with the corresponding moment and label it as the observation blind area; Construct a distribution map of the observation blind area evolving with time within a future preset time period according to the time series distribution and spatial coordinates of the observation blind area. The distribution map of the observation blind area includes the spatial boundary information and time window information of the observation blind area.

4. A method for optimizing collaborative observation of maritime targets based on data fusion according to claim 1, characterized in that, The construction of the collaborative compensation observation network according to the distribution map of the observation blind area is specifically as follows: Extract the spatial boundary information and time window information of each observation blind area according to the distribution map of the observation blind area, and calculate the coverage area, time window length of each observation blind area, and the predicted moving speed of the maritime moving target in the blind area; When the coverage area is less than the preset area threshold and the time window length is less than the preset time threshold, label the corresponding observation blind area as a short-term local blind area. When the coverage area exceeds the preset area threshold or the predicted moving speed of the maritime moving target exceeds the preset speed threshold, label the corresponding observation blind area as a long-term wide-area blind area; For the short-term local blind area, obtain the real-time position coordinates and endurance of the dispatchable drones in the target sea area, and calculate the shortest navigation path and navigation time for the drones to reach the boundary of the short-term local blind area from the current position according to the spatial boundary coordinates and time window information of the short-term local blind area; When the navigation time is less than the preset ratio of the time window length and the endurance of the drone meets the round-trip observation requirements, label the corresponding drone as an available compensation node, generate a drone observation path planning instruction, and obtain a short-term blind area compensation observation strategy; For the long-term wide-area blind area, obtain the historical transit time window data and coverage area data of the Beidou satellites, and match the coverage area and transit time window of the Beidou satellites according to the spatial boundary information and time window information of the long-term wide-area blind area, and screen out the Beidou satellite nodes whose coverage area has a spatial overlap with the long-term wide-area blind area and whose transit time window has an intersection with the blind area time window; According to the remote sensing image resolution of the selected Beidou satellite nodes, when the remote sensing image resolution meets the preset observation accuracy requirements, label the corresponding Beidou satellite nodes as available compensation nodes, generate a satellite remote sensing data acquisition instruction, and obtain a long-term blind area compensation observation strategy; Construct a collaborative compensation observation network including unmanned aerial vehicles (UAVs) and Beidou satellite nodes according to the short-term blind area compensation observation strategy and the long-term blind area compensation observation strategy.

5. The optimization method for collaborative observation of maritime targets based on data fusion according to claim 1, characterized in that, Perform collaborative observation on a maritime moving target according to the collaborative compensation observation network, and construct fused observation data, specifically as follows: Obtain real-time observation video data of a maritime moving target in a target sea area from a maritime observation platform, extract real-time observation video frames, and obtain compensation observation data for the blind area based on the collaborative compensation observation network. The compensation observation data includes UAV compensation observation video data and satellite remote sensing image data; Extract local feature points of the real-time observation video frames and the compensation observation data based on the Scale-Invariant Feature Transform (SIFT) algorithm, match the local feature points, and establish a spatial correspondence relationship between the video frames and the compensation observation data; Perform affine transformation on the pixel area covering the observation blind area in the compensation observation data according to the spatial correspondence relationship to generate a compensation image block spatially aligned with the real-time observation video frame; Use a weighted fusion algorithm to embed the compensation image block into the corresponding blind area position of the real-time observation video frame to generate fused observation data with a continuous time series.

6. The optimized method for collaborative observation of maritime targets based on data fusion according to claim 1, characterized in that Perform an emergency response to the maritime moving target according to the fused observation data, specifically as follows: Extract multi-dimensional motion feature vectors of the maritime moving target from the fused observation data, including the target heading angle change rate, speed fluctuation amplitude, and trajectory deviation degree; Obtain historical motion feature data of the maritime moving target in different abnormal events, construct an abnormal event recognition model based on a decision tree, import the motion feature data of the different abnormal events into the abnormal event recognition model, and calculate the classification contribution degree of each motion feature to the abnormal events based on the information gain ratio; Select motion features with an information gain ratio exceeding a preset threshold as decision tree splitting nodes according to the classification contribution degree, use the Gini coefficient as the node splitting criterion to construct a decision tree, and obtain a trained abnormal event recognition model; Import the multi-dimensional motion feature vectors into the trained abnormal event recognition model for abnormal event recognition to obtain an abnormal event recognition result; Determine the location information of the abnormal event according to the abnormal event recognition result, and construct emergency warning data from the abnormal event recognition result and the abnormal event location information and send it to nearby rescue ships for emergency response.

7. An optimized system for collaborative observation of maritime targets based on data fusion, characterized in that, The maritime target collaborative observation optimization system based on data fusion includes a memory and a processor. The memory includes a program for the maritime target collaborative observation optimization method based on data fusion. When the program for the maritime target collaborative observation optimization method based on data fusion is executed by the processor, the following steps are implemented: Track a maritime moving target based on a maritime observation platform, construct a time-series trajectory map of the maritime moving target, and predict the moving trajectory of the maritime moving target in a future preset time period according to the time-series trajectory map to obtain a trajectory prediction map; Determine the observation blind area of the maritime observation platform for the maritime moving target according to the trajectory prediction map, and construct an observation blind area distribution map; Construct a collaborative compensation observation network according to the observation blind area distribution map; Conduct collaborative observations on maritime mobile targets according to the described collaborative compensation observation network, and construct fused observation data; Conduct emergency responses to maritime mobile targets based on the described fused observation data.

Images