Sea current monitoring method, device and equipment based on video image and storage medium

Abstract

The application discloses a sea current monitoring method and device based on video images, equipment and a storage medium, relates to the technical field of sea current monitoring, and the method comprises the following steps: preprocessing video images of a target sea area, the preprocessing comprising frame processing and optical flow calculation of the video images; extracting spatial features of water surface wave patterns and floating object distribution of the preprocessed video images through a spatial flow network; capturing time sequence features of tidal periods and vortex evolution of the preprocessed video images through a time flow network; fusing the spatial features and the time sequence features as space-time features, and then calculating sea current parameters based on motion vector integration; eliminating noise interference of the sea current parameters based on multi-scale data fusion, and then generating a sea current field distribution model; and monitoring the sea current of the target sea area by using the sea current field distribution model. The application can finely monitor the sea current of the target sea area on a large scale, and the sea current field distribution model can accurately and quickly monitor the sea current, which is more efficient than the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of ocean current monitoring, and in particular to an ocean current monitoring method and device based on video images, equipment and a storage medium. BACKGROUND

[0002] Limitations of traditional ocean current monitoring technology Traditional ocean current monitoring methods mainly rely on contact or single-point measurement technology, which has the following bottlenecks: mechanical ocean current meter: flow rate is measured by propeller or rotor, but low flow rate (<0.2 m / s) or turbulent flow cannot be detected, and it is easily affected by sediment abrasion and biological attachment. Electromagnetic ocean current meter: requires seawater conductivity support, which is limited in deep sea applications, and has high energy consumption and complex maintenance. Acoustic Doppler current profiler (ADCP): relies on suspended matter reflection signals, which is invalid in clear water or very deep water (>500 m), and has a measurement blind area. Buoy drift method: only large-scale flow field information can be obtained, and local fine monitoring cannot be achieved. SUMMARY

[0003] Therefore, the embodiments of the present application provide an ocean current monitoring method and device based on video images, equipment and a storage medium to achieve fine and efficient ocean current monitoring.

[0004] An aspect of the embodiments of the present application provides an ocean current monitoring method based on video images, which comprises the following steps:

[0005] The video images of the target sea area are preprocessed, and the preprocessing includes frame processing and optical flow calculation of the video images;

[0006] The spatial features of water surface wave patterns and floating object distribution of the preprocessed video images are extracted by a spatial flow network;

[0007] The time sequence features of tidal period and vortex evolution of the preprocessed video images are captured by a time flow network;

[0008] The spatial features and the time sequence features are fused as spatiotemporal features, and then ocean current parameters are calculated based on motion vector integration;

[0009] Based on multi-scale data fusion, noise interference of the ocean current parameters is eliminated, and an ocean current field distribution model is generated;

[0010] The ocean current field distribution model is used to monitor the ocean current of the target sea area.

[0011] In some embodiments, the preprocessing of the video images of the target sea area comprises the following steps:

[0012] The video images are segmented into sliding windows composed of a preset number of consecutive frames;

[0013] The dense optical flow algorithm is used to calculate the displacement field between adjacent frames, and the window size of the optical flow calculation is dynamically adjusted based on the real-time wave height prediction value, wherein the window size is positively correlated with the wave height.

[0014] In some embodiments, the step of extracting the spatial features of water surface ripples and floating object distribution from the preprocessed video images by the spatial stream network comprises the following steps:

[0015] An improved convolutional neural network structure containing a dilated convolution layer is used as the spatial stream network.

[0016] The multi-scale spatial features of water surface ripples and floating object distribution are extracted by the spatial stream network.

[0017] In some embodiments, the step of capturing the time sequence features of tidal period and vortex evolution from the preprocessed video images by the time stream network comprises the following steps:

[0018] A combination structure of three-dimensional convolutional neural network and deformable convolution layer is used as the time stream network.

[0019] The multi-level time sequence features of tidal period and vortex evolution are captured by the time stream network.

[0020] In some embodiments, the step of fusing the spatial features and the time sequence features as spatiotemporal features and then calculating the ocean current parameters based on the motion vector integration comprises the following steps:

[0021] A spatiotemporal feature interaction weight matrix is constructed to realize the cross-modal fusion of the spatial features and the time sequence features to obtain spatiotemporal features.

[0022] The spatiotemporal features are subjected to time series analysis by a time sequence modeling unit, and the local area flow rate is calculated in combination with the motion vector integration algorithm.

[0023] In some embodiments, the step of eliminating the noise interference of the ocean current parameters based on multi-scale data fusion and then generating an ocean current field distribution model comprises the following steps:

[0024] Visible light video data, radar data and inertial navigation data are fused to establish multi-source complementary information.

[0025] A multi-scale spatial decomposition algorithm is used to separate effective ocean current features and noise interference from the multi-source complementary information.

[0026] A confidence-based dynamic weight allocation mechanism is used to realize cross-modal feature fusion of the effective ocean current features and then generate the ocean current field distribution model.

[0027] In some embodiments, the method further comprises the following steps:

[0028] The video image is compressed by using a video coding standard optimization algorithm, and the coding rate is dynamically adjusted according to the channel quality.

[0029] The multi-priority network slice is configured, and differentiated transmission resources are allocated for control instructions, the video image and management signals.

[0030] Another aspect of the embodiment of the present application further provides a sea current monitoring device based on a video image, and the device comprises:

[0031] A video preprocessing unit is configured to pre-process a video image of a target sea area, and the preprocessing comprises frame processing and optical flow calculation of the video image.

[0032] A spatial feature extraction unit is configured to extract spatial features of water surface wave patterns and floating object distribution of the pre-processed video image by using a spatial flow network.

[0033] A time sequence feature extraction unit is configured to capture time sequence features of tidal cycles and vortex evolution of the pre-processed video image by using a time flow network.

[0034] A sea current parameter calculation unit is configured to fuse the spatial features and the time sequence features as spatio-temporal features and then calculate sea current parameters based on motion vector integration.

[0035] A model generation unit is configured to eliminate noise interference of the sea current parameters based on multi-scale data fusion and then generate a sea current field distribution model.

[0036] A sea current monitoring unit is configured to monitor sea currents of the target sea area by using the sea current field distribution model.

[0037] Another aspect of the embodiment of the present application further provides an electronic device comprising a processor and a memory.

[0038] The memory is configured to store a program.

[0039] The processor executes the program to implement the method of any one of the above aspects.

[0040] Another aspect of the embodiment of the present application further provides a computer readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method of any one of the above aspects.

[0041] The present application at least has the following beneficial effects:

[0042] The application can preprocess video images of a target sea area, the preprocessing including frame processing and optical flow calculation of the video images; spatial features of water surface wave patterns and floating object distribution are extracted from the preprocessed video images through a spatial flow network; time sequence features of tidal period and vortex evolution are captured from the preprocessed video images through a time flow network; spatial features and time sequence features are fused as spatio-temporal features, and then sea current parameters are calculated based on motion vector integration; noise interference of the sea current parameters is eliminated based on multi-scale data fusion, and then a sea current field distribution model is generated; and the sea current field distribution model is used to monitor the sea current of the target sea area. The application can monitor the sea current of the target sea area in a large range and in detail, and the sea current field distribution model can accurately and quickly monitor the sea current, which is more efficient than the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0044] Figure 1 A flowchart of a sea current monitoring method based on video images provided by the embodiments of the application is shown in the figure.

[0045] Figure 2 A structure block diagram of a sea current monitoring device based on video images provided by the embodiments of the application is shown in the figure. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the application more clear, the application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application.

[0047] Before the embodiments of the application are described in detail, some related technologies involved in the embodiments of the application will be described first, as follows:

[0048] 1. Video flow measurement system core principle: improved image analysis method of buoy method, technical process as follows:

[0049] Feature extraction: identify texture features such as water surface rigid floating objects (such as buoys), wave patterns or bubbles.

[0050] Motion tracking: use optical flow method or deep learning model (such as U-Net) to match the displacement of feature points between consecutive frames.

[0051] Flow velocity calculation: Calculate surface flow velocity based on physical distance of feature points (converted by calibration parameters) and time difference between frames.

[0052] Flow estimation: Estimate cross-section flow by integral method or machine learning model combined with cross-section terrain data.

[0053] 2. System hardware architecture:

[0054] Typical equipment composition includes: imaging unit: equipped with high-definition camera (resolution ≥4K), polarization filter (reduces water surface reflection) and automatic gimbal (adapts to different lighting conditions).

[0055] Edge computing module: embedded GPU (such as NVIDIA Jetson) for real-time processing of video stream, reducing dependence on the cloud.

[0056] Communication module: support 4G / 5G or Tianhong satellite transmission to ensure data backhaul in remote areas.

[0057] Power supply system: solar panels + lithium batteries to meet the long-term operation needs in the field.

[0058] 3. Technical challenges and improvement direction:

[0059] Complex water flow disturbance: Turbulence and vortex cause feature point matching errors, need to introduce time sequence optical flow analysis and multi-sensor fusion (such as pressure gauge).

[0060] Calibration parameter drift: camera distortion and installation angle change affect accuracy in long-term deployment, need to develop adaptive calibration algorithm (such as based on permanent landmark matching).

[0061] Algorithm generalization ability: different water environments need customized models, promote the application of transfer learning in monitoring systems.

[0062] Summary of existing technology shortcomings:

[0063] (1) Monitoring failure under complex water flow conditions, problem performance: traditional video flow measurement method (such as wave peak and trough pixel polarization processing) fails to extract features due to water surface fragmentation in extreme wind and wave (such as typhoon weather) or turbulent environment, unable to accurately quantify flow velocity or flow. Technical limitations: rely on static feature matching, lack of dynamic environment adaptability, unable to handle irregular water flow patterns.

[0064] (2) Dependence on manual intervention and deployment restrictions, problem performance: existing wireless sea current monitoring systems require manual deployment of sensors to fixed structures (such as jacket), and require regular maintenance of equipment, resulting in high operation risk and increased cost. Technical limitations: device positioning relies on manual operation, unable to adaptively adjust monitoring area, poor flexibility.

[0065] (3) Low flow rate and turbidity water monitoring accuracy is insufficient, the problem is: the traditional buoy method or ADCP in low flow rate (<0.2 m / s) or high turbidity water body, the error increases significantly. Technical limitations: single-point measurement is easily affected by local disturbance, and the accuracy cannot be improved through multi-source data fusion.

[0066] (4) Insufficient real-time and data continuity, problem performance: satellite remote sensing and unmanned aerial vehicle monitoring are limited by revisit period and airspace, and cannot realize high-frequency continuous monitoring; traditional ADCP needs ship sailing, and data acquisition is lagging. Technical limitations: low data update frequency, difficult to capture short-term hydrological changes (such as tidal mutation).

[0067] (5) High system energy consumption and maintenance cost, problem performance: existing wireless equipment relies on solar power supply, and the endurance decreases in rainy weather; mechanical sensors are easily affected by biological attachment and need to be cleaned or replaced frequently. Technical limitations: insufficient hardware reliability, high long-term deployment cost.

[0068] The sea current monitoring technology based on video images solves the following technical problems through non-contact optical sensing:

[0069] Non-contact: avoid equipment corrosion caused by immersion, suitable for turbid water areas (such as the environment with high suspended solids concentration in the Jiangxi water conveying ditch case).

[0070] All-weather capability: combined with infrared or thermal imaging technology, it can operate at night or in low light conditions.

[0071] Large-scale coverage: a single device can monitor several kilometers of river section, which is significantly better than the single-point profile measurement of ADCP.

[0072] Data fusion potential: combined with spatiotemporal hydrological big data algorithm, it can simultaneously invert multiple parameters such as flow rate and water depth.

[0073] Reference Figure 1 The embodiments of the present application provide a sea current monitoring method based on video images, which specifically includes the following steps S100-S150:

[0074] S100: pre-processing the video images of the target sea area, the pre-processing includes frame processing and optical flow calculation of the video images;

[0075] S110: extracting the spatial features of water surface wave patterns and floating object distribution of the pre-processed video images through a spatial flow network;

[0076] S120: capturing the time sequence features of tidal period and vortex evolution of the pre-processed video images through a time flow network;

[0077] S130: fuse the spatial features and the temporal features as spatio-temporal features to calculate the ocean current parameters based on motion vector integration;

[0078] S140: eliminate noise interference of the ocean current parameters based on multi-scale data fusion to generate an ocean current field distribution model;

[0079] S150: monitor the ocean current of the target sea area by using the ocean current field distribution model.

[0080] Optionally, the video image of the target sea area is preprocessed, including the following steps:

[0081] The video image is divided into a sliding window composed of a preset number of consecutive frames;

[0082] The dense optical flow algorithm is used to calculate the displacement field between adjacent frames, and the optical flow calculation window size is dynamically adjusted based on the real-time wave height prediction value, wherein the window size is positively correlated with the wave height.

[0083] Optionally, the spatial features of water surface ripple pattern and floating object distribution are extracted from the preprocessed video image by the spatial stream network, including the following steps:

[0084] An improved convolutional neural network structure containing a hollow convolutional layer is used as the spatial stream network;

[0085] Multi-scale spatial features of water surface ripple pattern and floating object distribution are extracted by the spatial stream network.

[0086] Optionally, the temporal features of tidal period and vortex evolution are captured from the preprocessed video image by the time stream network, including the following steps:

[0087] A combination structure of three-dimensional convolutional neural network and deformable convolutional layer is used as the time stream network;

[0088] Multi-level temporal features of tidal period and vortex evolution are captured by the time stream network.

[0089] Optionally, the spatial features and the temporal features are fused as spatio-temporal features to calculate the ocean current parameters based on motion vector integration, including the following steps:

[0090] A spatio-temporal feature interaction weight matrix is constructed to realize cross-modal fusion of the spatial features and the temporal features to obtain spatio-temporal features;

[0091] The spatio-temporal features are analyzed by a time series modeling unit, and the local area flow rate is calculated by combining the motion vector integration algorithm.

[0092] Optionally, the multi-scale data fusion-based elimination of noise interference of the sea current parameter in turn generates a sea current field distribution model, including the following steps:

[0093] Fusion of visible light video data, radar data and inertial navigation data to establish multi-source complementary information;

[0094] A multi-scale spatial decomposition algorithm is used to separate effective sea current features and noise interference from the multi-source complementary information;

[0095] A confidence-based dynamic weight allocation mechanism is used to implement cross-modal feature fusion of the effective sea current features to in turn generate the sea current field distribution model.

[0096] Optionally, the method further includes the following steps:

[0097] A video encoding standard optimization algorithm is used to compress the video image, and the encoding rate is dynamically adjusted according to the channel quality;

[0098] A multi-priority network slice is configured, and differentiated transmission resources are allocated for control instructions, the video image and management signals.

[0099] Next, the scheme of the embodiments of the present application will be described and explained in detail in combination with specific application examples.

[0100] Specifically, the embodiments can include the following technical solutions:

[0101] (1) Robust monitoring in complex environments is achieved:

[0102] Technical path: A space-time dual-flow convolutional neural network (ST-CNN) is used to simultaneously extract water surface texture temporal changes and spatial form features, breaking through the feature failure bottleneck under extreme conditions such as typhoon breaking waves and turbulence. Target effect: In a typhoon wave with a wave height > 3m, the flow rate measurement error is ≤5%, which is 40% more reliable than traditional methods. DETAILED DESCRIPTION

[0104] ① System architecture:

[0105] Hardware composition:

[0106] Imaging unit: 4K resolution camera (supports HDR), polarization filter, 8x optical zoom lens.

[0107] Edge computing module: NVIDIA Jetson AGX Xavier module (computing power 32TOPS) is installed.

[0108] Communication module: 5G-A slice network (uplink bandwidth 1Gbps).

[0109] Power supply system: Dual-axis solar panel (500W) + lithium iron phosphate battery (200Ah).

[0110] Software architecture:

[0111] Data acquisition layer: Real-time video stream acquisition (25fps).

[0112] Feature extraction layer: ST-CNN model (spatio-temporal dual-stream parallel processing).

[0113] Decision output layer: Flow rate / flow calculation and visualization.

[0114] ② Spatio-temporal dual-stream convolutional neural network (ST-CNN) implementation:

[0115] Step 1: Input data preprocessing.

[0116] Video frame division: Continuous video is divided into 5-frame sliding windows (time interval Δt=0.2s).

[0117] Optical flow calculation: Improved Farneback dense optical flow algorithm is used to calculate the displacement field of adjacent frames:

[0118] Step 2: Spatial stream network (Spatial Stream).

[0119] Network structure: Based on ResNet50 improvement, adding dilated convolution layer (expansion rate r=2).

[0120] ;

[0121] .

[0122] Output features: Extract spatial features such as water surface ripple patterns and floating object distribution (dimension: B×1024×H / 16×W / 16).

[0123] Step 3: Temporal stream network (Temporal Stream).

[0124] Network structure: 3D-CNN + deformable convolution (Deformable Convolution):

[0125] ;

[0126] Where ΔFt is the deformable offset, and σ is the Sigmoid function.

[0127] Output features: Capture time sequence features such as tidal cycle and vortex evolution (dimension: B×32×T / 2×H / 8×W / 8).

[0128] Step 4: Feature fusion and decision.

[0129] Cross-modal attention: Calculate spatio-temporal feature interaction weights:

[0130]

[0131] LSTM time series modeling:

[0132]

[0133] Flow rate calculation: Based on motion vector integration:

[0134]

[0135] where Ai is the area of the feature region, u i is the local displacement vector.

[0136] ③ Key technology innovation.

[0137] Dynamic optical flow compensation mechanism:

[0138] Wave height adaptive filtering: dynamically adjust the optical flow calculation window according to the CNN predicted wave height H:

[0139]

[0140] Hmax is the maximum monitoring wave height of the device.

[0141] Multi-scale feature pyramid:

[0142] Spatial flow: Build a 4-level feature pyramid (1 / 4, 1 / 8, 1 / 16, 1 / 32 resolution).

[0143] Temporal flow: Design a time series pyramid (1 frame, 2 frames, 4 frame intervals).

[0144] Hardware co-optimization:

[0145] Edge computing acceleration: use TensorRT quantized model, inference speed is improved by 3 times (up to 18.6FPS).

[0146] Energy management: dynamically adjust the camera frame rate (1-25fps), prolong the endurance to 6 months.

[0147] (2) Fully autonomous deployment and adaptive calibration, technical path: integrate multispectral imaging module and edge computing unit, automatically identify water body boundary and suspended matter distribution through AI algorithm, dynamically adjust exposure parameters and feature extraction threshold. Target effect: deployment time is shortened to 1 hour (traditional 3-5 days), manual intervention is reduced by 90%. ① Fully autonomous deployment system

[0148] Hardware architecture: ​​​​

[0149] Positioning module: integrates a Beidou / GPS dual-mode positioning chip (positioning accuracy ±5cm) and an inertial measurement unit (IMU, sampling rate 200Hz).

[0150] Environmental perception module: equipped with lidar (scanning frequency 10Hz) and multispectral camera (wavelength range 400-1000nm).

[0151] Energy system: Foldable solar panel (conversion efficiency 22%) + lithium iron phosphate battery (200Ah), supporting operation from -30℃ to 60℃.

[0152] Self-deployment process:

[0153] Step 1: Environmental scanning and anchor point location.

[0154] The lidar scans an area with a radius of 50m and generates a point cloud map (resolution 0.1m).

[0155] Based on the YOLOv5 algorithm, identify permanent landmarks (such as navigation marks and reefs) and calculate the initial coordinates of the device:

[0156] ;

[0157] Where w i represents the confidence weight of the landmark, and n represents the number of landmarks detected.

[0158] Step 2: Equipment adaptively deploys.

[0159] The dual-axis gimbal automatically adjusts the pitch angle (range -30° to +90°) according to the terrain slope, ensuring that the camera's field of view covers more than 80% of the water surface. If an ocean current velocity >1.5m / s is detected, anchoring mode is activated.

[0160] ;

[0161] k=0.8 is the drag coefficient, ρ=1025kg / m 3 Where A is the density of seawater, A = 2m 2 For anchorage area.

[0162] Step 3: Establish communication link. 5G-A network (latency <10ms) is selected first. If the signal is weaker than -110dBm, switch to Tiantong satellite communication (uplink rate 1.2kbps).

[0163] ② Adaptive calibration system.

[0164] Hardware-level calibration:

[0165] Camera distortion correction:

[0166] The Brown-Conrady model is used to calculate the distortion parameters through the chessboard calibration plate (20 mm interval):

[0167] ;

[0168] r2=x2+y2, k1, k2 are radial distortion coefficients, and p1, p2 are tangential distortion coefficients.

[0169] Dynamic exposure compensation:

[0170] Based on the Retinex algorithm, the illumination component is separated, and the exposure parameter is adjusted in real time:

[0171] ;

[0172] ΔL is the light difference between the current frame and the historical mean, and σ=0.2 is the adjustment intensity.

[0173] Software level calibration:

[0174] Dynamic fitting of water surface reference surface:

[0175] The RANSAC algorithm is used to remove wave interference and fit the static water surface equation:

[0176] ;

[0177] ϵ∼N(0,σ2) is a noise term, and the parameters a, b, and c are optimized by least squares method.

[0178] Flow rate calibration feedback loop:

[0179] The float trajectory inversion model is introduced to establish the error mapping between the measured flow rate and the video flow measurement:

[0180] ;

[0181] λ=0.05h −1 is the drift attenuation coefficient, and t is the running time.

[0182] When δv>0.1 m / s, the parameter self-tuning is triggered:

[0183] ;

[0184] α=0.3 is the proportional gain adjustment factor.

[0185] ③ Core process implementation:

[0186] Abnormal environment adaptive strategy:

[0187] Typhoon wave scene: activate the frequency domain filtering module to suppress wave high-frequency noise:

[0188] ;

[0189] Cutoff frequency fc=2Hz, reserve 0.1-1.5Hz effective flow signal.

[0190] Low light scene: switch to long exposure mode (up to 10s), combined with HDR imaging technology:

[0191] ;

[0192] K=5 frame synthesis, weight wk is assigned based on signal-to-noise ratio, γ=0.45 is the logarithmic transformation coefficient.

[0193] ④Deployment and calibration experiment verification:

[0194] Calibration accuracy verification:

[0195] Attitude angle calibration: roll / pitch angle error <0.5° (traditional method ≥2°).

[0196] Flow rate calibration: 24-hour drift <0.03m / s (traditional equipment ≥0.15m / s).

[0197] (3) Multi-scale data fusion to improve accuracy.

[0198] Technical path: Construct a "monocular vision + pressure sensor + inertial navigation" multi-source data fusion framework, and use Kalman filtering algorithm to eliminate noise interference such as ship body sway and biological attachment.

[0199] Target effect: In water with turbidity >100 NTU, the relative error of flow rate measurement is ≤8% (traditional method ≥20%). DETAILED DESCRIPTION

[0201] ①Multi-scale data fusion system architecture:

[0202] Hardware composition:

[0203] Multi-modal sensor array: visible light camera (1080P@25fps), infrared thermal imager (640×512@60fps), millimeter wave radar (24GHz, resolution 0.1°×0.1°), edge computing unit: NVIDIA Jetson Orin (computing power 64TOPS)

[0204] Storage module: 1TB NVMe SSD (supports real-time data caching);

[0205] ②Multi-scale data fusion implementation process.

[0206] Step 1: Temporal and spatial alignment of heterogeneous data.

[0207] Spatial registration: improved RANSAC algorithm is used to eliminate the perspective difference and calculate the projection transformation matrix:

[0208] ;

[0209] Optimize the parameters aij by least squares method, the reprojection error <0.5 pixels.

[0210] Step 2: multi-scale feature decomposition.

[0211] Spatial dimension: using non-subsampled shearlet transform (NSST), decomposed into multi-level directional subband:

[0212] ;

[0213] J=4 level decomposition, K=8 directional filter.

[0214] Temporal dimension: construct 3D tensor T∈R T×H×W , through tensor decomposition.

[0215] ;

[0216] U, V, W are factor matrices, and E is the residual tensor.

[0217] Step 3: cross-modal feature interaction.

[0218] Attention guided fusion: calculate the correlation weight between modalities:

[0219] ;

[0220] hi is the i-th modal feature, and si is the spatial context vector.

[0221] Gating feature selection:

[0222] ;

[0223] σ is the Sigmoid function, and ⊙ is the element-wise multiplication.

[0224] Step 4: error compensation and reconstruction.

[0225] Residual learning module: construct residual network to compensate for fusion error:

[0226] ;

[0227] Update the residual network parameters by back propagation.

[0228] Dynamic range mapping: adaptive histogram equalization is used:

[0229] ;

[0230] Enhancing dark channel detail visibility.

[0231] ③ Key technical innovation.

[0232] Hybrid multi-scale decomposition strategy:

[0233] Space-time joint decomposition: combine NSST with 3D tensor decomposition to construct a spatio-temporal feature cube:

[0234] ;

[0235] For tensor product operation, preserve local details and global correlation.

[0236] Adaptive weight distribution mechanism:

[0237] Uncertainty quantification: calculate confidence of each modality based on Bayesian neural network:

[0238] ;

[0239] θ is the fusion weight, f BN / g BN is the batch normalization layer.

[0240] Cross-scale feature enhancement:

[0241] Residual dense connection: design cross-layer skip connection structure:

[0242] ;

[0243] M=3 is the number of connection paths, to improve gradient propagation efficiency.

[0244] (4) High-frequency continuous monitoring and real-time transmission.

[0245] Technical path: develop lightweight video compression algorithm (optimized version of H.266 standard), combined with 5G-A slice network, realize 1080P video stream 25 frames per second lossless transmission, delay <200ms.

[0246] Target effect: support minute-level flow field dynamic visualization, capture second-level mutation events within the tidal cycle. DETAILED DESCRIPTION

[0248] ① High-frequency continuous monitoring system architecture.

[0249] Hardware composition:

[0250] Video acquisition module: 4K resolution global shutter camera (frame rate 25fps, exposure time ≤5ms).

[0251] Edge computing unit: NVIDIA Jetson AGX Orin (64 TOPS, supporting TensorRT acceleration).

[0252] Transmission module: 5G-A slice network (uplink bandwidth 1 Gbps, latency <10 ms).

[0253] ②High-frequency continuous monitoring implementation process.

[0254] Step 1: Ultra-high-speed video acquisition.

[0255] Hardware trigger synchronization: Use hardware clock synchronization protocol (IEEE 1588 PTP) to ensure multi-camera timestamp error <1 μs.

[0256] Dynamic exposure control: Based on scene brightness adaptive adjustment, the formula is as follows:

[0257] ;

[0258] L avg is the average brightness of the current frame, L target =120 is the standard brightness, L std =30 is the brightness standard deviation.

[0259] Step 2: Real-time preprocessing at the edge.

[0260] Noise reduction processing: Use non-local mean filtering (NL-Means) to calculate the weighted average of similar blocks in the window:

[0261] ;

[0262] ;

[0263] Dynamic range mapping: Based on the Retinex theory to separate the illumination component and enhance the dark channel details:

[0264] ;

[0265] Step 3: Key frame selection and compression.

[0266] Motion saliency detection: Calculate the inter-frame difference energy:

[0267] ;

[0268] When E diff >3σ (σ is the historical noise standard deviation), mark it as a key frame.

[0269] Step 4: 5G-A real-time transmission.

[0270] The network slice configuration is as follows: priority 1 is for control slice, bandwidth 10 Mbps, and latency 2 ms; priority 2 is for video slice, bandwidth 1 Gbps, and latency 5 ms; and priority 3 is for management slice, bandwidth 50 Mbps, and latency 10 ms.

[0271] Adaptive bit rate adjustment: dynamically switch bit rate according to channel quality:

[0272] ;

[0273] ③ Key technology innovation.

[0274] Zero-copy data pipeline.

[0275] GPU pass-through technology: transmit camera data directly to GPU memory through DMA engine, avoiding CPU copy overhead.

[0276] Forward error correction (FEC) mechanism:

[0277] LDPC code correction: use (1024, 512) LDPC code, error correction capability up to 10 −6 bit error rate:

[0278] ;

[0279] Re-transmission strategy: based on ACK / NACK feedback, selectively retransmit lost packets:

[0280] ;

[0281] Edge-cloud collaborative computing:

[0282] Model distillation: compress the ST-CNN model trained on the cloud to the edge:

[0283] ;

[0284] Load balancing: dynamically allocate computing tasks to edge nodes:

[0285] ;

[0286] T k is the length of the task queue of node k, U k is the CPU utilization, and λ = 0.3 is the weight factor.

[0287] (5) Low power consumption and self-maintenance design.

[0288] Technical path: replace traditional mechanical components (such as brushless motor gimbals) with MEMS micro-electromechanical systems, reduce power consumption by 70%; introduce self-cleaning coating and AI anomaly detection module, reduce 90% of manual maintenance demand.

[0289] Target effect: The device can run continuously for ≥6 months without external power supply, and the operation and maintenance cost is reduced by 60%. DETAILED DESCRIPTION

[0291] ① Low-power system architecture design:

[0292] Hardware composition:

[0293] Energy harvesting module:

[0294] Piezoelectric ceramic sea wave energy collector (conversion efficiency ≥15%).

[0295] Solar thin-film battery (conversion efficiency 22%, area 0.5m²).

[0296] Main control unit: RISC-V architecture processor (main frequency 1GHz, power consumption 0.2W).

[0297] Sensor array:

[0298] 4K global shutter camera (power consumption 1.5W, supporting hardware level sleep).

[0299] MEMS gyroscope (0.1mA@100Hz).

[0300] Water quality multi-parameter probe (pH / salinity / turbidity, 0.3mA).

[0301] ② Low-power implementation process.

[0302] Step 1: Energy dynamic allocation.

[0303] Wave energy collection:

[0304] Adopting bistable piezoelectric structure, energy conversion efficiency improvement formula:

[0305] ;

[0306] k is the electromechanical coupling coefficient, ξ is the damping ratio, and ω is the wave frequency.

[0307] Solar energy management, dynamic adjustment of charging current:

[0308] ;

[0309] Step 2: Intelligent sleep and wake-up.

[0310] Motion triggered wake-up:

[0311] Detecting picture changes based on improved ViBe algorithm:

[0312] ;

[0313] Tthresh=30 is the luminance threshold, N(x,y) is the neighborhood window.

[0314] Timing wake-up mechanism:

[0315] RTC real-time clock is adopted, error <±2ppm, sleep power consumption <10μW.

[0316] Step 3: hardware-level power consumption optimization.

[0317] Camera dynamic resolution, automatically switch resolution according to sea conditions, for calm sea conditions, wave height changes in the range of 0-0.5m, resolution is 1080P, frame rate is 5fps; for moderate sea conditions, wave height changes in the range of 0.5-2.0m, resolution is 720P, frame rate is 15fps; for rough sea conditions, wave height changes in the range of greater than 2.0m, resolution is 480P, frame rate is 5fps.

[0318] ③Self-maintenance system design.

[0319] 3.1 Hardware self-checking module:

[0320] Sensor health assessment, abnormality detection based on Kalman filtering:

[0321] ;

[0322] Residual covariance matrix S=HP k H T +R, if ∥S∥>3σ trigger alarm.

[0323] Power system monitoring, battery health (SOH) estimation:

[0324] ;

[0325] Cnom=20000mAh is the nominal capacity, Vnom=12V.

[0326] Software self-repair mechanism:

[0327] Firmware hot update, differential OTA upgrade is adopted, only differential data block is transmitted:

[0328] ;

[0329] CRC check ensures data integrity.

[0330] Fault isolation strategy, build module dependency graph, automatically switch standby unit.

[0331] Mechanical self-cleaning design:

[0332] Anti-biofouling structure: hydrophobic nano-coating (contact angle > 150°) combined with a vibrating motor (frequency 20 Hz) to remove attached material.

[0333] Lens self-cleaning uses an ultrasonic atomizing cleaning unit, with a working cycle of:

[0334] ;

[0335] ④ Low power consumption and self-maintenance implementation verification.

[0336] When the working mode is high load, the energy supply is solar energy and wave energy; when the working mode is medium load, the energy supply is mainly wave energy; when the working mode is low load, the energy supply is battery power supply.

[0337] Self-maintenance effect verification, fault recovery time: <30 seconds (traditional equipment >10 minutes), false alarm rate: <0.5% (LSTM anomaly detection model based on historical data training).

[0338] In summary, the embodiment includes the following technical means:

[0339] The embodiment provides a sea current monitoring method based on video images, comprising synchronously extracting the space-time features of water surface texture through a space-time double-flow convolutional neural network (ST-CNN), calculating a displacement field by using an improved Farneback optical flow algorithm, and combining a wave height adaptive filtering mechanism. The ST-CNN comprises a spatial flow network, an improved cavity convolution module based on a ResNet50;

[0340] a time flow network, a time sequence modeling module combined with a deformable convolution, and a cross-modal attention mechanism, which dynamically fuses space-time features.

[0341] (2) A sea current monitoring system based on video images is provided, comprising: an edge computing module (NVIDIA Jetson AGX Xavier); a dynamic optical flow compensation unit (integrating a wave height prediction model); and a 5G-A slice communication module (uplink bandwidth ≥1Gbps).

[0342] (3) A sea current monitoring system based on video images comprises: an autonomous navigation module integrating Beidou / GPS and a laser radar to realize centimeter-level positioning; a dynamic calibration unit using a RANSAC algorithm to fit a water surface reference plane and compensate for distortion in real time; and a multi-source fusion engine integrating optical flow, IMU and buoy data through Kalman filtering.

[0343] (4) An autonomous deployment process of a sea current monitoring system based on video images comprises: landmark recognition and coordinate calculation based on YOLOv5; a double-shaft gimbal adaptive pitch angle adjustment algorithm; and a sea current adaptive anchoring force calculation model.

[0344] (5) A sea current monitoring method based on video images includes constructing a spatiotemporal feature cube through non-subsampled shearlet transform and 3D tensor decomposition, calculating multi-modal data fusion weights using a Bayesian neural network, and introducing a residual learning module to supplement fusion errors.

[0345] (6) In a sea current monitoring method based on video images, spatiotemporal cube construction includes spatial dimension non-subsampled shearlet transform (NSST) and time dimension tensor decomposition.

[0346] (7) A sea current monitoring device based on video images includes a multi-modal sensor array (visible light, infrared, millimeter wave radar, etc.), an edge computing unit module, and a dynamic range mapping module (adaptive histogram equalization).

[0347] (8) A sea current monitoring method based on video images includes achieving 25 frames per second lossless video transmission through a 5G-A slice network, an end-to-end delay of less than 200 ms, using H.266 optimized encoding and adaptive code rate adjustment strategy, and dynamically matching network bandwidth.

[0348] (9) Real-time transmission in a sea current monitoring method based on video images includes a forward error correction mechanism and a key frame selection algorithm, and key frames are determined by inter-frame difference energy threshold.

[0349] (10) A sea current monitoring system device based on video images includes an edge computing unit, a 5G-A slice network module, and an adaptive code rate controller.

[0350] (11) A sea current monitoring system based on video images includes a pressure point ceramic wave energy collector, a dynamic power consumption controller, and a hardware self-test module.

[0351] (12) Energy dynamic allocation in a sea current monitoring system based on video images includes wave energy collection efficiency calculation and solar charging current adaptive adjustment strategy.

[0352] (13) A self-maintenance method for a sea current monitoring system based on video images includes hardware health assessment based on Kalman filter residual covariance analysis and firmware hot flow update based on differential over-the-air (OAT) upgrade and cyclic redundancy check (CRC) verification mechanism.

[0353] Referring Figure 2 , the embodiments of the present application provide a sea current monitoring device based on video images, comprising:

[0354] a video preprocessing unit for preprocessing video images of a target sea area, the preprocessing including frame processing and optical flow calculation of video images;

[0355] a spatial feature extraction unit configured to extract spatial features of water surface ripples and distribution of floating objects from the preprocessed video images through a spatial stream network;

[0356] a temporal feature extraction unit configured to capture temporal features of tidal period and vortex evolution from the preprocessed video images through a temporal stream network;

[0357] a current parameter calculation unit configured to fuse the spatial features and the temporal features as spatio-temporal features and calculate current parameters based on motion vector integration;

[0358] a model generation unit configured to eliminate noise interference of the current parameters based on multi-scale data fusion and generate a current field distribution model;

[0359] a current monitoring unit configured to monitor the currents of the target sea area by using the current field distribution model.

[0360] It can be understood that the contents in the method embodiments are applicable to the device embodiments, the device embodiments specifically implement the functions of the method embodiments, and achieve the same beneficial effects as the method embodiments.

[0361] In some alternative embodiments, the functions / operations mentioned in the block diagram can not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, two blocks shown in succession can actually be executed substantially simultaneously or the blocks can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flowcharts of the present application are provided by way of example, and the purpose is to provide a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of larger operations are independently executed.

[0362] Furthermore, although the present application is described in the context of functional modules, it is understood that one or more of the functions and / or features described can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary to an understanding of the present application. Rather, the actual implementation is within the routine skill of engineers familiar with the property, function and internal relationships of the various functional modules disclosed herein. Accordingly, the present application is not limited to the specific details of the functional modules described herein. Rather, it is understood that the skilled artisan, using ordinary skill in the art, can implement the application described in the claims without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and that the scope of the present application is determined by the appended claims and their equivalents.

[0363] If the functions are implemented in software, the functions can be stored in or implemented as one or more computer program products, which can be incorporated into a computer-readable medium for use by or in connection with an apparatus, method, or system as described herein. The computer-readable medium can be a computer- only medium, a partially software and partially hardware medium, or a medium that contains both software and hardware. The computer-readable medium can be any medium that can contain, store, or transport the program for use by or in connection with the computer. The computer-readable medium can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or a propagation medium. The computer-readable medium can be a computer program product. The computer program product can be packaged on an apparatus-based medium or transmitted by a device programmed to perform the process steps of the application.

[0364] Logic and / or steps represented in flow diagrams or otherwise described herein, for example, can be embodied in non-transitory computer-readable media, which can direct or instruct an instruction execution system, apparatus, or device to implement the steps or logic represented therein. For purposes of this specification, "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a computer storage medium or a computer communication medium.

[0365] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

[0366] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, or combinations thereof, can be used: a discrete logic circuit having logic gates for implementing logic functions upon data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0367] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or more embodiments or examples.

[0368] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions, and alterations can be made hereto without departing from the principles and spirit of the application, and the scope of the present application is defined by the claims and their equivalents.

[0369] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the described embodiments, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A method for monitoring ocean currents based on video imagery, characterized in that, The method includes the following steps: The video images of the target sea area are preprocessed, including frame-by-frame processing and optical flow calculation of the video images; Spatial features of water surface ripple morphology and floating object distribution are extracted from the preprocessed video images using a spatial flow network. The temporal characteristics of tidal cycles and vortex evolution are captured by a time-stream network on the preprocessed video images. The spatial features and temporal features are integrated as spatiotemporal features, and then ocean current parameters are calculated based on motion vector integrals. The ocean current field distribution model is generated by eliminating noise interference from the ocean current parameters based on multi-scale data fusion. The ocean currents in the target sea area are monitored using the aforementioned ocean current field distribution model; The process of fusing the spatial features and the temporal features as spatiotemporal features and then calculating ocean current parameters based on motion vector integrals includes the following steps: Construct a spatiotemporal feature interaction weight matrix to achieve cross-modal fusion of the spatial features and the temporal features to obtain spatiotemporal features; The spatiotemporal features are analyzed using a time-series modeling unit, and the flow velocity in the local region is calculated using a motion vector integral algorithm. The process of eliminating noise interference from ocean current parameters and generating an ocean current field distribution model based on multi-scale data fusion includes the following steps: By integrating visible light video data, radar data, and inertial navigation data, multi-source complementary information can be established; A multi-scale spatial decomposition algorithm is used to separate effective ocean current features from noise interference from the multi-source complementary information. A dynamic weight allocation mechanism based on confidence level is used to achieve cross-modal feature fusion of the effective ocean current features, thereby generating the ocean current field distribution model.

2. The ocean current monitoring method based on video imagery according to claim 1, characterized in that, The preprocessing of video images of the target sea area includes the following steps: The video image is divided into a sliding window consisting of a preset number of consecutive frames; The dense optical flow algorithm is used to calculate the displacement field between adjacent frames, and the size of the optical flow calculation window is dynamically adjusted based on the real-time wave height prediction value, wherein the window size is positively correlated with the wave height.

3. The ocean current monitoring method based on video imagery according to claim 1, characterized in that, The step of extracting spatial features of water surface ripple morphology and floating object distribution from the preprocessed video image using a spatial flow network includes the following steps: An improved convolutional neural network structure containing dilated convolutional layers is used as the spatial flow network; Multi-scale spatial features of water surface ripple morphology and floating object distribution are extracted using the spatial flow network.

4. The ocean current monitoring method based on video imagery according to claim 1, characterized in that, The step of capturing the temporal features of tidal cycles and vortex evolution from the preprocessed video images using a time-stream network includes the following steps: A combination of a three-dimensional convolutional neural network and deformable convolutional layers is used as the temporal flow network; The time-flow network is used to capture the multi-level temporal characteristics of tidal cycles and vortex evolution.

5. The ocean current monitoring method based on video imagery according to any one of claims 1 to 4, characterized in that, The method further includes the following steps: The video image is compressed using a video coding standard optimization algorithm, and the coding bitrate is dynamically adjusted according to channel quality. Configure multi-priority network slices to allocate differentiated transmission resources for control commands, video images, and management signals.

6. A video-based ocean current monitoring device, characterized in that, The device is applied to the ocean current monitoring method based on video imagery as described in claim 1, and the device comprises: The video preprocessing unit is used to preprocess video images of the target sea area, and the preprocessing includes frame-by-frame processing of the video images and optical flow calculation. The spatial feature extraction unit is used to extract spatial features of water surface ripple morphology and floating object distribution from the preprocessed video image through a spatial flow network. The temporal feature extraction unit is used to capture the temporal features of tidal cycle and vortex evolution from the preprocessed video image through a time-stream network. The ocean current parameter calculation unit is used to fuse the spatial features and the temporal features as spatiotemporal features and then calculate the ocean current parameters based on motion vector integrals. The model generation unit is used to eliminate noise interference from the ocean current parameters based on multi-scale data fusion, thereby generating an ocean current field distribution model. The ocean current monitoring unit is used to monitor ocean currents in the target sea area using the ocean current field distribution model.

7. An electronic device, characterized in that, The electronic device includes a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the method as described in any one of claims 1 to 5.

Images