Beidou vision fusion-based ship monitoring method, medium and equipment

Abstract

The invention discloses a ship monitoring method based on Beidou vision fusion, a medium and equipment, and the ship monitoring method based on multi-level verification is constructed through deep fusion of a Beidou positioning terminal and vision AI monitoring equipment. Specifically, initial position information is determined by acquiring alarm data uploaded by a Beidou terminal; scheduling the intelligent cameras in a preset range, and calculating target accurate coordinates based on the reference positions and the ranging data; driving the camera to zoom and turn to a focusing target area to collect visual data; performing event identification on the visual data by utilizing the pre-training model; performing consistency comparison verification on the positioning alarm and the visual identification result; and finally generating a verification alarm including event types, accurate positions and image analysis. According to the technical scheme, the positioning precision and the alarm accuracy are effectively improved, the positioning precision and the event identification accuracy are remarkably improved, the false alarm condition is greatly reduced, and the emergency response time is effectively shortened.

Description

Technical Field

[0001] The present invention relates to the field of ship monitoring technology, and in particular to a ship monitoring method, medium and equipment based on Beidou vision fusion. Background Art

[0002] Existing ship monitoring systems primarily rely on satellite positioning devices for data collection and alarm feedback. However, due to the lack of video equipment installation on unpowered vessels, such as township vessels, existing systems suffer from low positioning accuracy, high false alarm rates, and difficulty in emergency command and decision-making. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to propose a ship monitoring method, medium and equipment based on Beidou vision fusion.

[0004] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0005] In a first aspect, the present invention provides a ship monitoring method based on Beidou visual fusion, comprising:

[0006] Obtain real-time alarm data uploaded by Beidou shipborne positioning terminals and Beidou intelligent life-saving positioning devices. The real-time alarm data includes the alarm event type, the latitude and longitude of the alarm device, and the alarm timestamp;

[0007] Based on the alarm latitude and longitude, the Beidou Vision AI monitoring camera within the preset range is dispatched. The precise latitude and longitude of the target is calculated based on the dual-antenna direction finding angle, the camera reference latitude and longitude of the dual-frequency RTK differential positioning, and the target distance of the laser ranging unit. The Beidou Vision AI monitoring camera within the preset range is driven to zoom and turn to focus on the target area corresponding to the precise latitude and longitude of the target, and the final visual data is collected;

[0008] Perform AI event recognition on the final visual data. This includes extracting image features through a convolutional neural network to obtain prediction results, and using a pre-trained ship accident detection model to determine the target event type. The target event type includes at least one of: person overboard, ship capsize, and shipwreck.

[0009] Compare the alarm event type with the target event type for consistency. If a match is found, a secondary verification alarm is generated. The secondary verification alarm includes the target event type, the target's precise latitude and longitude, and an AI analysis image.

[0010] A ship monitoring report is generated based on the secondary verification alarm. The ship monitoring report includes event level assessment, target location trajectory map and emergency response recommendations, and is pushed to the business terminal in real time through the situation awareness platform.

[0011] In some embodiments, the precise longitude and latitude of the target is calculated based on the dual-antenna direction finding angle, the camera reference longitude and latitude of the dual-frequency RTK differential positioning, and the target distance of the laser ranging unit, including:

[0012] Obtain the camera azimuth output by the dual-antenna direction-finding module and the camera reference longitude and latitude measured in real time by the dual-frequency RTK module;

[0013] Obtain the straight-line distance between the target object and the camera through the laser ranging unit;

[0014] Based on the spherical trigonometry model, the azimuth, reference longitude and latitude, and straight-line distance are used as input parameters to calculate the target longitude and latitude corresponding to the target precise longitude and latitude using formula (1). Formula (1) is as follows:

[0015]

[0016] In formula (1), long1 is the reference longitude, lat1 is the reference latitude, d1 is the target distance of the laser ranging unit, θ1 is the dual-antenna direction finding angle, R is the radius of the earth, long2 is the target longitude, and lat2 is the target latitude;

[0017] Perform Gauss-Krüger projection coordinate conversion on the target longitude and latitude, and output the precise longitude and latitude of the target corresponding to the geographic location coordinates.

[0018] In some embodiments, driving a Beidou vision AI monitoring camera within a preset range to zoom and turn to focus on a target area corresponding to the precise latitude and longitude of the target, and collecting final visual data, includes:

[0019] Calculate the gimbal steering control parameters based on the spatial relationship between the target precise longitude and latitude and the reference longitude and latitude;

[0020] Calculate zoom control parameters based on target distance measured by laser ranging unit and camera optical parameters;

[0021] Generate a camera driving instruction set including steering instructions and zoom instructions;

[0022] After executing the camera driving instruction set, the camera is controlled to collect visible light images and infrared images of the target area as first visual data;

[0023] Performing acquisition quality verification on the first visual data, determining it as valid visual data when the image clarity reaches a preset resolution threshold, the target object is located within a preset range of the image center, and the target detection confidence in consecutive acquisition frames exceeds a preset threshold, and is recorded as the second visual data;

[0024] If the acquisition quality verification fails, the steering control parameters and zoom control parameters are recalculated to perform supplementary acquisition until the second visual data is obtained;

[0025] The second visual data is output as final visual data.

[0026] In some embodiments, calculating the gimbal steering control parameters based on the spatial relationship between the target precise longitude and latitude and the reference longitude and latitude includes:

[0027] Based on the spherical trigonometry model, the reference longitude and latitude and the target precise longitude and latitude are used as input parameters, and the spherical distance between the reference position and the target position is calculated using formula (2). Formula (2) is as follows:

[0028] d2=R·arccos(sin(lat1)·sin(lat2)+cos(lat1)·cos(lat2)·cos(Δlong));

[0029] In formula (2), d2 is the spherical distance between the reference position and the target position, Δlong is the longitude deviation value, Δlong = |long2-long1|;

[0030] The direction angle from the reference position to the target position is calculated by formula (3), which is as follows:

[0031] θ2=arctan2(sin(Δlong)·cos(lat2),cos(lat1)·sin(lat2)-sin(lat1)·cos(lat2)cos(Δlong));

[0032] In formula (3), θ2 is the direction angle from the reference position to the target position;

[0033] The spherical distance between the reference position and the target position is converted into a zoom parameter of the camera, and the direction angle from the reference position to the target position is converted into an azimuth control parameter of the pan / tilt head;

[0034] Outputs pan / tilt control instructions containing zoom parameters and azimuth angle control parameters to drive the camera pan / tilt to perform zoom and steering actions.

[0035] In some embodiments, AI event recognition is performed on the final visual data. The AI ​​event recognition includes extracting image features through a convolutional neural network to obtain a prediction result, including:

[0036] The final visual data is input into the pre-trained convolutional neural network model for multi-scale feature extraction to obtain the first image feature vector;

[0037] Performing spatial pyramid pooling on the first image feature vector to generate a second image feature vector;

[0038] The feature vector of the second image is weighted by the attention mechanism to highlight the key area features of the ship and obtain the weighted feature vector;

[0039] The weighted feature vector is input into the feature fusion layer, and cross-modal feature fusion is performed with the thermal radiation feature of the infrared image to obtain the fused feature;

[0040] Perform dimension reduction on the fused features to obtain the final image feature vector;

[0041] The final image feature vector is input into the classifier to predict the type of ship event, and the output includes the prediction results of the probability of people falling into the water, the probability of ship capsizing, and the probability of ship sinking.

[0042] In some embodiments, a pre-trained ship accident detection model is used to determine the target event type, where the target event type includes at least one of a person falling overboard, a ship capsizing, and a shipwreck, including:

[0043] According to the prediction results, the probability of people falling into the water, the probability of ship capsizing and the probability of ship sinking are extracted;

[0044] When the probability of a person falling into the water exceeds a first threshold, it is determined that a person falling into the water event has occurred;

[0045] When the probability of the ship capsizing exceeds a second threshold, it is determined that a ship capsizing event has occurred;

[0046] When the probability of a shipwreck exceeds a third threshold, it is determined that a shipwreck has occurred;

[0047] The first threshold, the second threshold, and the third threshold are configured to be obtained through training of historical accident data.

[0048] In some embodiments, the alarm event type is compared with the target event type for consistency. If a match is found, a secondary verification alarm is generated. The secondary verification alarm includes the target event type, the precise latitude and longitude of the target, and an AI analysis image, including:

[0049] Extract alarm event type and corresponding alarm timestamp from real-time alarm data;

[0050] Obtain the target event type and corresponding AI analysis timestamp output by the ship accident detection model;

[0051] When the alarm event type is consistent with the target event type and the time difference between the alarm timestamp and the AI ​​analysis timestamp is less than the preset time tolerance threshold, the alarm event type is determined to match the target event type;

[0052] Extract the precise latitude and longitude of the target calculated by dual-frequency RTK differential positioning;

[0053] Obtain AI analysis images containing ship features from the AI ​​event recognition process;

[0054] Generates a secondary verification alarm that includes the target event type, the target's precise latitude and longitude, and AI analysis images.

[0055] In some embodiments, a vessel monitoring report is generated based on the secondary verification alarm. The vessel monitoring report includes an event level assessment, a target location trajectory map, and emergency response recommendations, including:

[0056] Obtain the target event type and AI analysis image in the secondary verification alarm;

[0057] Determine the event level assessment result according to the preset event level mapping table corresponding to the target event type;

[0058] Extract historical positioning data of Beidou shipborne positioning terminals and Beidou intelligent life-saving positioning devices within a preset time period;

[0059] Generate a target location trajectory map including the time dimension based on historical positioning data and the target's precise latitude and longitude;

[0060] According to the event level assessment results and the target event type, the corresponding emergency response suggestions are matched from the preset emergency response knowledge base;

[0061] Integrate event level assessment results, target location trajectory maps and emergency response recommendations into a ship monitoring report.

[0062] In a second aspect, the present invention further provides a computer-readable storage medium storing computer program instructions, which implement the method described in the first aspect when executed by a processor.

[0063] In a third aspect, the present invention further provides an electronic device comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method described in the first aspect.

[0064] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art:

[0065] Different from the existing technology, the above technical solution has built a multi-level verification ship monitoring method by deeply integrating the Beidou positioning terminal with the visual AI monitoring equipment. Specifically, the preliminary location information is determined by obtaining the alarm data uploaded by the Beidou terminal; the smart camera within the preset range is dispatched to calculate the precise coordinates of the target based on its reference position and ranging data; the camera is driven to zoom and focus on the target area to collect visual data; the visual data is used for event recognition using a pre-trained model; the positioning alarm is compared and verified for consistency with the visual recognition results; and finally a verification alarm is generated that includes the event type, precise location, and image analysis. This technical solution effectively improves positioning accuracy and alarm accuracy, significantly improves positioning accuracy and event recognition accuracy, greatly reduces false alarms, and effectively shortens emergency response time. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0067] Figure 1 It is a schematic diagram of the steps of the ship monitoring method;

[0068] Figure 2 It is a flow chart of the ship monitoring method. DETAILED DESCRIPTION

[0069] The present invention will be described in further detail below with reference to the accompanying drawings and examples. It is particularly noted that the following examples are intended only to illustrate the present invention and are not intended to limit the scope of the present invention. Similarly, the following examples are only some embodiments of the present invention and are not intended to be exhaustive. All other embodiments obtained by those of ordinary skill in the art without creative effort are intended to fall within the scope of protection of the present invention.

[0070] See also Figure 1 and Figure 2 In a first aspect, this embodiment provides a ship monitoring method based on Beidou visual fusion, comprising:

[0071] S101. Acquire real-time alarm data uploaded by the Beidou shipborne positioning terminal and the Beidou intelligent lifesaving positioning device, where the real-time alarm data includes the alarm event type, the latitude and longitude of the alarm device, and the alarm timestamp;

[0072] S102: Based on the alarm latitude and longitude, dispatch the Beidou vision AI monitoring camera within the preset range, calculate the target's precise latitude and longitude based on the dual-antenna direction finding angle, the camera's reference latitude and longitude using dual-frequency RTK differential positioning, and the target distance of the laser ranging unit, and drive the Beidou vision AI monitoring camera within the preset range to zoom and turn to focus on the target area corresponding to the target's precise latitude and longitude, and collect final visual data;

[0073] S103. Perform AI event recognition on the final visual data. The AI ​​event recognition includes extracting image features through a convolutional neural network to obtain a prediction result, and using a pre-trained ship accident detection model to determine the target event type. The target event type includes at least one of a person falling overboard, a ship capsizing, and a shipwreck.

[0074] S104: Compare the alarm event type with the target event type for consistency. If they match, a secondary verification alarm is generated. The secondary verification alarm includes the target event type, the precise latitude and longitude of the target, and an AI analysis image.

[0075] S105. Generate a ship monitoring report based on the secondary verification alarm. The ship monitoring report includes an event level assessment, a target location trajectory map, and emergency response recommendations, and is pushed to the business terminal in real time through the situation awareness platform.

[0076] In step S101, real-time alarm data uploaded by the Beidou shipborne positioning terminal and the Beidou intelligent life-saving positioning device is received. The real-time alarm data contains three key elements: the alarm event type (such as man overboard, ship capsize, etc.), the latitude and longitude coordinates of the alarm device, and accurate timestamp information.

[0077] In step S102, the Beidou visual AI monitoring camera within the preset range is automatically dispatched based on the alarm longitude and latitude. The target orientation is determined by the dual-antenna direction finding angle. Combined with the camera's own reference position (obtained through dual-frequency RTK differential positioning) and the target distance data provided by the laser ranging unit, the precise longitude and latitude of the target is calculated using a spatial geometry algorithm. The camera is then controlled to perform zoom and steering operations so that the monitoring image accurately focuses on the target area and completes the collection of high-quality visual data. In some preferred embodiments, the laser ranging unit can control the ranging error to the centimeter level to ensure the accuracy of coordinate solution.

[0078] In step S103, the final visual data captured by the camera undergoes intelligent analysis. A convolutional neural network is used to extract image features, and a pre-trained ship accident detection model is used to identify and determine the target event type. Optional identification types include typical marine accidents such as man overboard, ship capsize, and shipwreck. The model utilizes multi-layer feature fusion to enhance recognition robustness in complex scenarios.

[0079] In step S104, the alarm event type reported by the Beidou terminal is cross-validated against the target event type identified by the visual AI. When the two match, a secondary verification alarm is generated. This secondary verification alarm contains three core elements: the visually confirmed event type, the precise latitude and longitude coordinates calculated through multi-sensor fusion, and an image of the scene annotated with AI analysis. This achieves dual verification of satellite positioning and visual evidence, effectively preventing false alarms from a single sensor.

[0080] In step S105, the secondary verification alarm is converted into a structured vessel monitoring report. This report integrates an event severity assessment, a target trajectory map generated based on historical location data, and emergency response recommendations based on the event type. Ultimately, the report is pushed in real time to maritime regulatory and rescue command centers via the situational awareness platform, forming a complete closed-loop monitoring system.

[0081] This implementation leverages the deep collaboration of Beidou positioning and visual AI to build a comprehensive monitoring system, from initial alarm triggering, multi-sensor data fusion, intelligent visual verification, to decision support. This significantly improves positioning accuracy and event recognition, effectively reduces the risk of false alarms through a dual verification mechanism, and enhances emergency response efficiency through structured reporting, providing reliable technical support for maritime safety supervision.

[0082] In some embodiments, the precise longitude and latitude of the target is calculated based on the dual-antenna direction finding angle, the camera reference longitude and latitude of the dual-frequency RTK differential positioning, and the target distance of the laser ranging unit, including:

[0083] Obtain the camera azimuth output by the dual-antenna direction-finding module and the camera reference longitude and latitude measured in real time by the dual-frequency RTK module;

[0084] Obtain the straight-line distance between the target object and the camera through the laser ranging unit;

[0085] Based on the spherical trigonometry model, the azimuth, reference longitude and latitude, and straight-line distance are used as input parameters to calculate the target longitude and latitude corresponding to the target precise longitude and latitude using formula (1). Formula (1) is as follows:

[0086]

[0087] In formula (1), long1 is the reference longitude, lat1 is the reference latitude, d1 is the target distance of the laser ranging unit, θ1 is the dual-antenna direction finding angle, R is the radius of the earth, long2 is the target longitude, and lat2 is the target latitude;

[0088] Perform Gauss-Krüger projection coordinate conversion on the target longitude and latitude, and output the precise longitude and latitude of the target corresponding to the geographic location coordinates.

[0089] In this embodiment, the dual-frequency RTK module simultaneously provides real-time measurement data of the camera's reference longitude and latitude, forming the basic parameters for coordinate calculation. Optionally, the reference longitude and latitude measurements employ dual-frequency RTK technology, achieving centimeter-level positioning accuracy. The laser ranging unit operates independently to obtain the linear distance between the target object and the camera. In preferred embodiments, the laser ranging unit utilizes a phase-based ranging principle, achieving a ranging error of no more than 3 centimeters within a range of 500 meters, providing highly accurate distance parameters for coordinate calculation.

[0090] Based on the spherical trigonometry model, the azimuth, reference longitude and latitude, and straight-line distance are input into formula (1) for coordinate calculation. Formula (1) establishes spatial geometric transformations through trigonometric relationships, where the Earth's radius R is set to 6,371,000 meters. The calculation of the longitude component requires consideration of latitude cosine compensation. The calculation process strictly maintains dimensional consistency to ensure consistency between the mathematical model and physical reality.

[0091] The target longitude and latitude output by formula (1) are transformed into Gauss-Krüger projection coordinates. The spherical coordinates are converted into geographic location coordinates in a plane rectangular coordinate system. The conversion process uses a three-dimensional zone projection method. The central meridian is automatically selected according to the base longitude. The final output is the target precise latitude and longitude that meets the requirements of the GIS system.

[0092] This embodiment establishes a high-precision conversion method from relative bearing to absolute geographic coordinates through multi-sensor data fusion and rigorous spatial geometry calculations. Dual-antenna direction finding provides a bearing reference, laser ranging ensures distance accuracy, spherical trigonometry models enable rigorous coordinate calculations, and Gauss-Krüger projections achieve coordinate standardization. This effectively improves target positioning accuracy, provides a reliable geospatial data foundation for ship monitoring, and reduces the impact of single sensor errors on the overall system.

[0093] In some embodiments, driving a Beidou vision AI monitoring camera within a preset range to zoom and turn to focus on a target area corresponding to the precise latitude and longitude of the target, and collecting final visual data, includes:

[0094] Calculate the gimbal steering control parameters based on the spatial relationship between the target precise longitude and latitude and the reference longitude and latitude;

[0095] Calculate zoom control parameters based on target distance measured by laser ranging unit and camera optical parameters;

[0096] Generate a camera driving instruction set including steering instructions and zoom instructions;

[0097] After executing the camera driving instruction set, the camera is controlled to collect a visible light image and an infrared image of the target area as first visual data;

[0098] Performing acquisition quality verification on the first visual data, determining it as valid visual data when the image clarity reaches a preset resolution threshold, the target object is located within a preset range of the image center, and the target detection confidence in consecutive acquisition frames exceeds a preset threshold, and is recorded as the second visual data;

[0099] If the acquisition quality verification fails, the steering control parameters and zoom control parameters are recalculated to perform supplementary acquisition until the second visual data is obtained;

[0100] The second visual data is output as final visual data.

[0101] In this embodiment, based on the spatial geometric relationship between the precise longitude and latitude of the target and the reference longitude and latitude of the camera, the control parameters required for pan-tilt steering are calculated by azimuth angle difference and pitch angle. The optional steering control uses a PID algorithm to achieve smooth motion to prevent mechanical jitter from affecting the image acquisition quality. The zoom control parameters are calculated by combining the target distance measured by the laser ranging unit with the optical characteristics of the camera. Taking into account hardware parameters such as lens focal length and sensor size, the perspective projection model is used to determine the optimal zoom factor to ensure that the target object occupies an appropriate proportion in the picture. In some preferred embodiments, the zoom control accuracy can reach 0.1 times the optical zoom step.

[0102] The camera's drive command set includes two types of control signals: steering and zoom commands. Steering commands specify the pan / tilt and tilt angles, while zoom commands set the lens' focal length and aperture. The command set is transmitted to the camera control unit via RS-485 or Ethernet. After executing the drive commands, the camera simultaneously captures visible light and infrared images of the target area as primary visual data. The visible light image provides color and texture information, while the infrared image complements the thermal radiation signature. Together, these two data types form the multispectral observation results. Hardware triggering is used during the acquisition process to ensure millisecond-level image synchronization accuracy.

[0103] The first visual data must pass the acquisition quality verification process. The verification criteria include three core indicators: image resolution of at least 1920×1080 pixels, target object center position offset of no more than 15% of the frame range, and target detection confidence of 0.85 or higher in five consecutive frames. First visual data that meets all conditions is marked as second visual data. If the first visual data fails quality verification, the steering and zoom control parameters are recalculated. The recapture process adjusts the gimbal angle and lens focal length, and iterative optimization is performed based on the previous acquisition results until second visual data that meets quality requirements is obtained. The maximum number of recaptures is set to three to avoid invalid acquisition cycles. The second visual data is output as the final visual data to the subsequent processing module. The final visual data contains the quality-verified visible light image, infrared image, and corresponding metadata. The metadata records information such as acquisition time, geographic location, and sensor parameters.

[0104] This embodiment achieves reliable image acquisition of target areas in complex environments through precise geometric calculations and closed-loop quality control. Spatial relationship calculations ensure accurate initial positioning, multi-parameter joint control optimizes imaging, and rigorous quality verification ensures data availability. This effectively improves the success rate of image acquisition in adverse weather conditions, and a closed-loop feedback mechanism reduces resource waste caused by ineffective acquisition.

[0105] In some embodiments, calculating the gimbal steering control parameters based on the spatial relationship between the target precise longitude and latitude and the reference longitude and latitude includes:

[0106] Based on the spherical trigonometry model, the reference longitude and latitude and the target precise longitude and latitude are used as input parameters, and the spherical distance between the reference position and the target position is calculated using formula (2). Formula (2) is as follows:

[0107] d2=R·arccos(sin(lat1)·sin(lat2)+cos(lat1)·cos(lat2)·cos(Δlong));

[0108] In formula (2), d2 is the spherical distance between the reference position and the target position, Δlong is the longitude deviation value, Δlong = |long2-long1|;

[0109] The direction angle from the reference position to the target position is calculated by formula (3), which is as follows:

[0110] θ2=arctan2(sin(Δlong)·cos(lat2),cos(lat1)·sin(lat2)-sin(lat1)·cos(lat2)cos(Δlong));

[0111] In formula (3), θ2 is the direction angle from the reference position to the target position;

[0112] The spherical distance between the reference position and the target position is converted into a zoom parameter of the camera, and the direction angle from the reference position to the target position is converted into an azimuth control parameter of the pan / tilt head;

[0113] Outputs pan / tilt control instructions containing zoom parameters and azimuth angle control parameters to drive the camera pan / tilt to perform zoom and steering actions.

[0114] In this embodiment, formula (2) uses the reference longitude and latitude and the target precise longitude and latitude as inputs, and calculates the spherical distance between the reference position and the target position using the inverse cosine function. The calculation process takes into account the influence of the earth's curvature, where R takes the earth's average radius of 6371000 meters, and the direction angle is calculated using formula (3). This formula uses the four-quadrant inverse tangent function arctan2 to process direction determination. The numerator contains the product of the sine function of the longitude difference and the cosine of the target latitude, and the denominator is composed of the product of the sine of the reference latitude and the cosine of the target latitude. The calculation result eliminates directional ambiguity and accurately reflects the azimuth relationship from the reference position to the target position.

[0115] The spherical distance is converted into the camera's zoom factor. This conversion takes into account the camera's optical characteristics, establishing a distance-zoom factor mapping relationship. In some preferred embodiments, a piecewise linear interpolation algorithm is used to achieve a precise distance-to-zoom factor mapping, with a minimum adjustment step of 0.05x optical zoom.

[0116] The direction angle is directly converted into the azimuth control parameter of the gimbal. It should be noted that the calculated θ2 is in radians and needs to be converted to degrees:

[0117]

[0118] If the result is negative, add 360 degrees to keep it in the range [0,360).

[0119] The conversion process retains the original angle accuracy, with the pan range covering 0-360 degrees and the vertical pitch range from -30 degrees to +90 degrees. Control parameters are transmitted with 16-bit precision, ensuring a pan-tilt angle resolution better than 0.01 degrees.

[0120] The resulting pan / tilt control commands contain two types of data: zoom parameters and azimuth control parameters. These commands are encoded in binary format and transmitted to the camera's pan / tilt control unit via the RS-485 bus at 115,200 bps. Once executed, these commands drive the camera's pan / tilt control unit to achieve precise zoom and steering movements.

[0121] This embodiment achieves precise conversion of spatial position into control parameters through rigorous spherical geometry calculations. The spherical distance formula accurately reflects the Earth's curved surface, the azimuth angle calculation eliminates azimuth ambiguity, and the parameter conversion process maintains measurement accuracy. This effectively improves camera alignment accuracy, ensuring the target object is always in the optimal viewing position, and reduces the number of mechanical adjustments through mathematical model optimization.

[0122] In some embodiments, AI event recognition is performed on the final visual data. The AI ​​event recognition includes extracting image features through a convolutional neural network to obtain a prediction result, including:

[0123] The final visual data is input into the pre-trained convolutional neural network model for multi-scale feature extraction to obtain the first image feature vector;

[0124] Performing spatial pyramid pooling on the first image feature vector to generate a second image feature vector;

[0125] The feature vector of the second image is weighted by the attention mechanism to highlight the key area features of the ship and obtain the weighted feature vector;

[0126] The weighted feature vector is input into the feature fusion layer, and cross-modal feature fusion is performed with the thermal radiation feature of the infrared image to obtain the fused feature;

[0127] Perform dimension reduction on the fused features to obtain the final image feature vector;

[0128] The final image feature vector is input into the classifier to predict the type of ship event, and the output includes the prediction results of the probability of people falling into the water, the probability of ship capsizing, and the probability of ship sinking.

[0129] In this embodiment, the processing flow of AI event recognition takes the final visual data as input. The pre-trained convolutional neural network model adopts the ResNet50 architecture and extracts multi-scale image features through five convolution stages. Each convolution stage contains batch normalization and ReLU activation function, and outputs a 1024-dimensional first image feature vector. Spatial pyramid pooling performs multi-level pooling operations on the first image feature vector. The pooling level is set to three grid sizes of 4×4, 2×2, and 1×1, and local features of different scales are aggregated into a second image feature vector of fixed length. The second image feature vector dimension is expanded to 4096 dimensions, retaining the multi-level spatial information of the original image.

[0130] The attention mechanism uses a channel attention module to process the second image feature vector. Global average pooling is used to generate channel weights, and feature channels in key areas such as the ship's mast and side are weighted by a factor of 1.5-3. This weighted feature vector maintains a 4096-dimensional structure, improving the signal-to-noise ratio of key features by a factor of 2-3. The feature fusion layer performs a cross-modal fusion of the weighted feature vectors of the visible light image with the thermal radiation features of the infrared image. The fusion process uses element-by-element addition, and the infrared features are resized to 4096 dimensions using a 1×1 convolution kernel.

[0131] The fused features contain both visible light texture and thermal radiation intensity information, maintaining a constant 4096-dimensional dimensionality. Dimensionality reduction is performed using a fully connected layer to compress the 4096-dimensional fused features to 512 dimensions. These fully connected layers use a linear activation function, combined with a dropout layer to prevent overfitting. The final image feature vector retains over 95% of the original feature information, improving computational efficiency by eightfold. The classifier consists of three parallel fully connected layers, each handling the probability predictions for three types of events: man overboard, vessel capsize, and shipwreck. Each fully connected layer outputs a probability value between 0 and 1, using a sigmoid activation function for multi-label classification. The prediction results include independent probability estimates for each of the three event types.

[0132] This embodiment achieves high-precision ship event recognition through multi-stage feature processing and cross-modal fusion. Multi-scale feature extraction preserves image details, an attention mechanism emphasizes key areas, and thermal radiation features supplement environmental information. This effectively improves the accuracy of identifying ship hazards in complex sea conditions. Collaborative analysis of visible and infrared data enhances detection capabilities at night and in foggy conditions. The multi-label classification structure meets the needs of parallel judgment of different event types.

[0133] In some embodiments, a pre-trained ship accident detection model is used to determine the target event type, where the target event type includes at least one of a person falling overboard, a ship capsizing, and a shipwreck, including:

[0134] According to the prediction results, the probability of people falling into the water, the probability of ship capsizing and the probability of ship sinking are extracted;

[0135] When the probability of a person falling into the water exceeds a first threshold, it is determined that a person falling into the water event has occurred;

[0136] When the probability of the ship capsizing exceeds a second threshold, it is determined that a ship capsizing event has occurred;

[0137] When the probability of a shipwreck exceeds a third threshold, it is determined that a shipwreck has occurred;

[0138] The first threshold, the second threshold, and the third threshold are configured to be obtained through training of historical accident data.

[0139] In this embodiment, the ship accident detection model determines the event type based on the prediction results. The model input includes three independent prediction values: the probability of people falling overboard, the probability of ship capsizing, and the probability of shipwreck. Each probability value ranges from 0 to 1. The probability data comes from the feature extraction and classifier output of the convolutional neural network in the previous stage. The judgment condition for a people-overboard event is that the probability of people falling overboard exceeds a first threshold. The first threshold is determined by analyzing the ratio of positive and negative samples in historical people-overboard accidents, and the optional threshold is set to 0.78. When the probability exceeds the threshold, the model outputs a people-overboard event flag and triggers the corresponding alarm protocol.

[0140] The basis for determining a ship capsize event is when the probability of the ship capsizing exceeds a second threshold. This second threshold is derived from training on the correlation between the ship's tilt angle and accidents, with a typical value of 0.85. During the judgment, the spatial distribution changes of the ship's contour feature points are simultaneously checked to enhance the reliability of the judgment. The trigger condition for a shipwreck event is when the probability of the shipwreck exceeds a third threshold. This third threshold is calculated based on the rate of change of the ship's draft depth and the proportion of the visible area of ​​the hull, and the default configuration is 0.91. In some preferred embodiments, the shipwreck judgment is combined with the prediction results of multiple consecutive frames for time series verification.

[0141] Optionally, the three thresholds are optimized using machine learning. The training dataset contains 5,000 historical accident samples and 15,000 normal flight samples. Receiver-operating characteristic (ROC) curve analysis is used to determine the optimal balance between the thresholds. Threshold parameters are stored in the model configuration file and can be dynamically adjusted based on actual application scenarios.

[0142] This embodiment achieves accurate identification of ship accidents through a multi-threshold judgment mechanism. Independent threshold settings take into account the characteristic differences between different accident types, and historical data training ensures the scientific nature of the thresholds. This effectively distinguishes between real accidents and suspected cases, reducing false alarms. Multi-type parallel judgment meets monitoring needs in complex sea conditions, and the configurable parameter design adapts to different application scenarios in different waters.

[0143] In some embodiments, the alarm event type is compared with the target event type for consistency. If a match is found, a secondary verification alarm is generated. The secondary verification alarm includes the target event type, the precise latitude and longitude of the target, and an AI analysis image, including:

[0144] Extract alarm event type and corresponding alarm timestamp from real-time alarm data;

[0145] Obtain the target event type and corresponding AI analysis timestamp output by the ship accident detection model;

[0146] When the alarm event type is consistent with the target event type and the time difference between the alarm timestamp and the AI ​​analysis timestamp is less than the preset time tolerance threshold, the alarm event type is determined to match the target event type;

[0147] Extract the precise latitude and longitude of the target calculated by dual-frequency RTK differential positioning;

[0148] Obtain AI analysis images containing ship features from the AI ​​event recognition process;

[0149] Generates a secondary verification alarm that includes the target event type, the target's precise latitude and longitude, and AI analysis images.

[0150] In this embodiment, the generation of the secondary verification alarm is based on a dual verification mechanism of the alarm event type and the target event type. The alarm event types extracted from the real-time alarm data include classification information such as people falling overboard, ship capsizing or sinking. The alarm timestamp is accurate to the millisecond level and records the moment the event is triggered. The target event type output by the ship accident detection model is synchronized with the AI ​​analysis timestamp. The AI ​​analysis timestamp marks the time node when the convolutional neural network completes feature extraction and classification prediction, and the time synchronization accuracy is controlled within 50 milliseconds. The preset time tolerance threshold is set to 200 milliseconds to ensure that the time window during comparison covers data transmission and processing delays.

[0151] Dual-frequency RTK differential positioning provides centimeter-level accuracy for precise target latitude and longitude data. Positioning data is updated 10 times per second, with longitude values ​​retained to 7 decimal places and latitude values ​​retained to 7 decimal places. When generating a secondary verification alarm, the positioning data frame closest to the AI ​​analysis timestamp is automatically selected. The AI ​​analysis image is extracted from the intermediate buffer during the event recognition process. The image resolution remains at 1920×1080 pixels and includes a vessel outline marker and a heat map overlay. Image storage uses JPEG compression with a quality parameter set to 90, balancing file size and detail preservation. The data structure of the secondary verification alarm contains three mandatory fields: the target event type field is encoded using a 4-byte integer, the target precise latitude and longitude fields are stored using a 16-byte floating-point array, and the AI ​​analysis image field is embedded in binary format. The alarm data packet is appended with an 8-byte timestamp and a 4-byte checksum to ensure transmission integrity.

[0152] This embodiment improves the reliability of alarm information through multi-level data verification. Timestamp comparison eliminates asynchronous data interference, precise location information enhances event traceability, and visual analysis images provide intuitive judgment. This effectively filters out false alarms, providing comprehensive, temporally and spatially accurate composite alarm data for rescue decision-making. This dual verification mechanism significantly reduces the risk of single sensor failure.

[0153] In some embodiments, a vessel monitoring report is generated based on the secondary verification alarm. The vessel monitoring report includes an event level assessment, a target location trajectory map, and emergency response recommendations, including:

[0154] Obtain the target event type and AI analysis image in the secondary verification alarm;

[0155] Determine the event level assessment result according to the preset event level mapping table corresponding to the target event type;

[0156] Extract historical positioning data of Beidou shipborne positioning terminals and Beidou intelligent life-saving positioning devices within a preset time period;

[0157] Generate a target location trajectory map including the time dimension based on historical positioning data and the target's precise latitude and longitude;

[0158] According to the event level assessment results and the target event type, the corresponding emergency response suggestions are matched from the preset emergency response knowledge base;

[0159] Integrate event level assessment results, target location trajectory maps and emergency response recommendations into a ship monitoring report.

[0160] In this embodiment, the generation of the ship monitoring report is based on the data of the secondary verification alarm. The target event type and AI analysis image are directly parsed and obtained from the secondary verification alarm data packet, the event type field is converted into a standard event code, and the image data maintains the original resolution unchanged. The event level assessment is completed by querying the preset event level mapping table. The mapping table divides the three types of events, namely, people falling overboard, ship capsizing and shipwreck, into three levels: emergency, major and extremely major. The incident of people falling overboard is assessed as emergency by default, the incident of ship capsizing is assessed as major, and the incident of shipwreck is automatically classified as extremely major.

[0161] Historical positioning data is collected from Beidou shipborne positioning terminals and Beidou intelligent life-saving positioning devices. The data acquisition time range is set from 30 minutes before the event to the current moment. Positioning data includes three fields: longitude, latitude, and UTC timestamp. Positioning data sampling interval is 10 seconds, and positioning accuracy is better than 5 meters. The target location trajectory map is generated using vector graphics. The horizontal axis represents time, and the vertical axis represents geographic location. The trajectory lines use different colors to distinguish between the shipborne terminal and the life-saving device. In some preferred embodiments, the trajectory map is overlaid with an electronic nautical chart as a base map, and key locations are annotated with precise time information.

[0162] Emergency response recommendations are dynamically generated by matching from a pre-set knowledge base. The knowledge base maintains a tree-like index structure based on incident type and severity. Each recommendation includes three components: handling procedures, contact units, and equipment requirements. The matching process prioritizes recommendations that fully match the current incident severity. Vessel monitoring reports are output in a structured document format. The incident severity assessment results are prominently displayed at the report header, with a target location trajectory diagram presented as an embedded chart. Emergency response recommendations are sorted in descending order of priority. The report file also includes metadata such as the incident number, generation time, and issuing unit.

[0163] This embodiment generates a comprehensive ship monitoring report through multi-dimensional data analysis. Event level assessment quantifies risk, trajectory maps visualize the event development process, and emergency recommendations provide targeted response plans. This solution transforms discrete alarm information into complete decision-making support documents. Standardized report formats facilitate multi-party collaboration, historical trajectory analysis helps predict event trends, and a knowledge base matching mechanism ensures the professionalism and timeliness of emergency recommendations.

[0164] In a second aspect, this embodiment further provides a computer-readable storage medium storing computer program instructions, which implement the method described in the first aspect when executed by a processor.

[0165] The computer program involved in this embodiment can be stored in a computer device readable storage medium, which includes but is not limited to a disk, a magnetic tape, a magnetic card, a floppy disk, a flash memory, an optical disc, an optical card, a read-only memory (ROM), a random access memory (RAM), an erasable programmable ROM (EPROM) and an electrically erasable programmable ROM (EEPROM), etc., and also includes other biological, physical or chemical structures that can achieve similar or equivalent functions to the storage media listed above, such as DNA, RNA, proteins and other units with information storage capabilities. In a specific embodiment, the storage medium involved can be one of the above-mentioned media types or a combination of the above-mentioned media types. In different embodiments, the computer program involved in the embodiment can be stored in a single medium in a centralized manner or in a distributed manner in multiple media. The memory containing the computer device readable storage medium can be a non-volatile memory or a random access memory. These computer device readable storage media can be built into the device or can be connected to the device involved in the embodiment as an external device or part of an external device. In some embodiments, a memory having a computer-readable storage medium is deployed locally; in other embodiments, a solution of deploying the memory away from the processor may also be adopted, such as a network-attached memory accessed via an RF circuit or an external port and a communication network, wherein the communication network may be the Internet, one or more intranets, a local area network (LAN), a wide area network (WLAN), a storage area network (SAN), etc., or a suitable combination thereof, as long as the computer device can access the memory. In addition, the computer program involved in the embodiment can be stored in plaintext / ciphertext form, or can be designed as training data, which can be integrated and reorganized through model training and implicitly stored in the parameter state of a deep neural network or other machine learning model.

[0166] In a third aspect, this embodiment further provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method described in the first aspect.

[0167] The processor described in this embodiment can be implemented by hardware, firmware, software or a combination thereof, and can use a circuit, a single or multiple application-specific integrated circuits (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, or at least one of a microprocessor. It also includes other physical, biological or chemical structures that can achieve similar or equivalent functions to the processors listed above, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute some or all of the steps in the computer program or method involved in each embodiment of the present application, or any combination of the steps mentioned therein.

[0168] Different from the existing technology, the above technical solution has built a multi-level verification ship monitoring method by deeply integrating the Beidou positioning terminal with the visual AI monitoring equipment. Specifically, the preliminary location information is determined by obtaining the alarm data uploaded by the Beidou terminal; the smart camera within the preset range is dispatched to calculate the precise coordinates of the target based on its reference position and ranging data; the camera is driven to zoom and focus on the target area to collect visual data; the visual data is used for event recognition using a pre-trained model; the positioning alarm is compared and verified for consistency with the visual recognition results; and finally a verification alarm is generated that includes the event type, precise location, and image analysis. This technical solution effectively improves positioning accuracy and alarm accuracy, significantly improves positioning accuracy and event recognition accuracy, greatly reduces false alarms, and effectively shortens emergency response time.

[0169] The above descriptions are only some embodiments of the present invention and do not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A ship monitoring method based on Beidou visual fusion, characterized in that: include: Obtaining real-time alarm data uploaded by the Beidou shipborne positioning terminal and the Beidou intelligent life-saving positioning device, the real-time alarm data including the alarm event type, the latitude and longitude of the alarm device, and the alarm timestamp; Based on the alarm latitude and longitude, the Beidou Vision AI monitoring camera within the preset range is dispatched. The precise latitude and longitude of the target is calculated based on the dual-antenna direction finding angle, the camera reference latitude and longitude of the dual-frequency RTK differential positioning, and the target distance of the laser ranging unit. The Beidou Vision AI monitoring camera within the preset range is driven to zoom and turn to focus on the target area corresponding to the precise latitude and longitude of the target, and the final visual data is collected; Performing AI event recognition on the final visual data, the AI ​​event recognition includes extracting image features through a convolutional neural network to obtain prediction results, and using a pre-trained ship accident detection model to determine the target event type, the target event type including at least one of a person falling overboard, ship capsizing, and shipwreck; Compare the alarm event type with the target event type for consistency. If they match, a secondary verification alarm is generated. The secondary verification alarm includes the target event type, the target's precise latitude and longitude, and an AI analysis image. A ship monitoring report is generated based on the secondary verification alarm, which includes an event level assessment, a target location trajectory map, and emergency response recommendations, and is pushed to the business terminal in real time through the situation awareness platform.

2. The ship monitoring method based on Beidou visual fusion according to claim 1 is characterized in that: Calculate the precise longitude and latitude of the target based on the dual-antenna direction finding angle, the camera reference longitude and latitude of the dual-frequency RTK differential positioning, and the target distance of the laser ranging unit, including: Obtain the camera azimuth output by the dual-antenna direction-finding module and the camera reference longitude and latitude measured in real time by the dual-frequency RTK module; Acquire the straight-line distance between the target object and the camera by the laser ranging unit; Based on the spherical trigonometry model, the azimuth, reference longitude and latitude, and straight-line distance are used as input parameters to calculate the target longitude and latitude corresponding to the target precise longitude and latitude using formula (1). Formula (1) is as follows: In formula (1), long1 is the reference longitude, lat1 is the reference latitude, d1 is the target distance of the laser ranging unit, θ1 is the dual-antenna direction finding angle, R is the radius of the earth, long2 is the target longitude, and lat2 is the target latitude; Perform Gauss-Krüger projection coordinate conversion on the target longitude and latitude, and output the precise longitude and latitude of the target corresponding to the geographic location coordinates.

3. The ship monitoring method based on Beidou visual fusion according to claim 2 is characterized in that: Drive the Beidou vision AI monitoring camera within the preset range to zoom and turn to focus on the target area corresponding to the precise latitude and longitude of the target, and collect the final visual data, including: Calculate the gimbal steering control parameters based on the spatial relationship between the target precise longitude and latitude and the reference longitude and latitude; Calculate zoom control parameters based on target distance measured by laser ranging unit and camera optical parameters; Generate a camera driving instruction set including steering instructions and zoom instructions; After executing the camera driving instruction set, the camera is controlled to collect a visible light image and an infrared image of the target area as first visual data; Performing acquisition quality verification on the first visual data, determining it as valid visual data when the image clarity reaches a preset resolution threshold, the target object is located within a preset range of the image center, and the target detection confidence in consecutive acquisition frames exceeds a preset threshold, and is recorded as the second visual data; If the acquisition quality verification fails, the steering control parameters and zoom control parameters are recalculated to perform supplementary acquisition until the second visual data is obtained; The second visual data is output as final visual data.

4. The ship monitoring method based on Beidou visual fusion according to claim 3 is characterized in that: Calculate the gimbal steering control parameters based on the spatial relationship between the target's precise latitude and longitude and the reference latitude and longitude, including: Based on the spherical trigonometry model, the reference longitude and latitude and the target precise longitude and latitude are used as input parameters, and the spherical distance between the reference position and the target position is calculated by formula (2), which is as follows: d2=R·arccos(sin(lat1)·sin(lat2)+cos(lat1)·cos(lat2)·cos(Δlong)); In formula (2), d2 is the spherical distance between the reference position and the target position, Δlong is the longitude deviation value, Δlong = |long2-long1|; The direction angle from the reference position to the target position is calculated by formula (3), which is as follows: θ2=arctan2(sin(Δlong)·cos(lat2),cos(lat1)·sin(lat2)-sin(lat1)·cos(lat2)cos(Δlong)); In formula (3), θ2 is the direction angle from the reference position to the target position; Converting the spherical distance between the reference position and the target position into a zoom parameter of the camera, and converting the direction angle from the reference position to the target position into an azimuth control parameter of the pan / tilt head; Outputs pan / tilt control instructions containing zoom parameters and azimuth angle control parameters to drive the camera pan / tilt to perform zoom and steering actions.

5. The ship monitoring method based on Beidou visual fusion according to claim 1 is characterized in that: Perform AI event recognition on the final visual data. The AI ​​event recognition includes extracting image features through a convolutional neural network to obtain prediction results, including: Inputting the final visual data into a pre-trained convolutional neural network model to perform multi-scale feature extraction to obtain a first image feature vector; performing spatial pyramid pooling processing on the first image feature vector to generate a second image feature vector; Performing feature weighting on the second image feature vector through an attention mechanism to highlight key area features of the ship and obtain a weighted feature vector; The weighted feature vector is input into the feature fusion layer, and cross-modal feature fusion is performed with the thermal radiation feature of the infrared image to obtain the fused feature; Perform dimension reduction on the fused features to obtain the final image feature vector; The final image feature vector is input into a classifier to predict the type of ship event, and the prediction results including the probability of people falling into the water, the probability of ship capsizing and the probability of ship sinking are output.

6. The ship monitoring method based on Beidou visual fusion according to claim 1 is characterized in that: The pre-trained ship accident detection model is used to determine the target event type, where the target event type includes at least one of a person falling overboard, a ship capsizing, and a shipwreck, including: According to the prediction results, the probability of people falling into the water, the probability of ship capsizing and the probability of ship sinking are extracted; When the probability of a person falling into the water exceeds a first threshold, it is determined that a person falling into the water event has occurred; When the probability of the ship capsizing exceeds a second threshold, it is determined that a ship capsizing event has occurred; When the probability of a shipwreck exceeds a third threshold, it is determined that a shipwreck has occurred; The first threshold, the second threshold, and the third threshold are configured to be obtained through training of historical accident data.

7. The ship monitoring method based on Beidou visual fusion according to claim 1 is characterized in that: The alarm event type is compared with the target event type for consistency. If a match is found, a secondary verification alarm is generated. The secondary verification alarm includes the target event type, the target's precise latitude and longitude, and an AI analysis image, including: Extracting the alarm event type and the corresponding alarm timestamp from the real-time alarm data; Obtain the target event type and corresponding AI analysis timestamp output by the ship accident detection model; When the alarm event type is consistent with the target event type and the time difference between the alarm timestamp and the AI ​​analysis timestamp is less than a preset time tolerance threshold, it is determined that the alarm event type matches the target event type; Extract the precise latitude and longitude of the target calculated by dual-frequency RTK differential positioning; Obtaining an AI analysis image containing vessel features from the AI ​​event recognition process; Generate a secondary verification alarm containing the target event type, the target's precise latitude and longitude, and AI analysis images.

8. The ship monitoring method based on Beidou visual fusion according to claim 1 is characterized in that: Generate a ship monitoring report based on the secondary verification alarm, which includes an event level assessment, a target location trajectory map, and emergency response recommendations, including: Obtaining the target event type and AI analysis image in the secondary verification alarm; Determine the event level assessment result according to the preset event level mapping table corresponding to the target event type; Extracting historical positioning data of the Beidou shipborne positioning terminal and the Beidou intelligent life-saving positioning device within a preset time period; Based on the historical positioning data and the precise latitude and longitude of the target, a target position trajectory map including a time dimension is generated; Matching corresponding emergency response suggestions from a preset emergency response knowledge base based on the event level assessment result and the target event type; The event level assessment results, target location trajectory map and emergency response recommendations are integrated into a ship monitoring report.

9. A computer-readable storage medium storing computer program instructions, characterized in that: The computer program instructions implement the method according to any one of claims 1 to 8 when executed by a processor.

10. An electronic device comprising a memory and a processor, characterized in that: The memory is configured to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 8.

Images